 Before I start, I just want to say I'm not an engineer anymore, but I just want to thank Engineer Art for having essentially funded the Rails 3 effort. In addition to Rails 3 getting funded and obviously me and Carl, there's a lot of community multiplier effect that happens. People like Jose, Kaleem and Aaron Patterson in the audience and Santiago Pasemillo in the audience. So we got a bunch of new people that Engineer Art didn't have to pay for but through Engineer Art funding of full-time people to sit around and take questions and help people get started really, I think, made the difference to Rails 3 and will make the difference moving forward. I think Rails 3 is so strong now in terms of the community that not having as many real-time people, although Aaron is full-time modern now, I think that it will not be... Basically we did our job, right? We did the job that we set out to do a couple of years ago. I'm really excited, I want to thank Engineer Art. Engineer Art is going to continue putting some time, energy and money into Rails 3.1 and moving forward, so that's great and I just want to thank him. So my original talk title was Extending Rails 3. There's a much better place to find out about it because it's actually not a 30-minute topic. Jose wrote a book that he's going to be releasing really soon called Rails 3 Internals for Daily Use. You can tell he's from Brazil. It has all these topics and a lot more and again, every one of these topics is easily a 30-minute talk if you just talk to surface, so again, I could have done it but I figured that you guys might want to know a little bit about Rails 3.1 so we're going to talk about that. So the first thing is engines have just undergone a very large overall and this guy, Drogas, you can give him some love on Twitter if you want. He actually did this as a Ruby Summer Code project and like Jose's project last year, the overall generators, it was a pretty ambitious project and he basically did everything we had hoped that he would do in some more pretty much just delivering on the promise of the mouthful apps idea that we had a few years ago. So I'm going to talk about sort of what engines have now for a few minutes but just give him some love. So, first of all, migration. This is the last feature that wasn't in Rails 3.0 Migrations from the engines plugin. So the way it works is that you do rate, real-time, so any engine can just shift to the migration directory and you copy that and then it goes and it takes the, for instance, O01.adcolum.rb and moves it into a time-stamped add column with the name of the engine so that if you run that over and over again as the update of the jam, it won't continually copy out old ones but you have to wrap a record of the migration. There was a lot of discussion, probably by shedding level discussion about exactly how this feature needs to be implemented and the copying approach specifically solving how you can make sure you don't do copy things turned out to be by far the best solution and we finally have one that works well that will shift to Rails 3.0. The second really big thing that there are solutions for but there's no canonical solution for in Rails 3.0 with engines is static assets. Specifically, it's nice to have static assets that you don't have to hash the time as per engine, ask for copying or simulinking. Right now there are solutions but every engine basically provides some mechanism for copying the assets and what you really want is that the LTAB engines just put their stuff in a public directory and just have that work correctly without any additional work. The way this is going to work is that there's a public directory which is like a regular public directory and then you might have two engines that have public directories in development mode. The static, actions, dispatch, static, middleware will serve all three of them so you don't have to have like engine access or something like that serving your static assets. In production mode, the static middleware will still serve assets from things like device and hope but engine access will serve the main public directory. Something that we're doing just to make sure that that doesn't start to become a performance problem is the actions, dispatch, static, middleware will become a pre-populated app so instead of having to scan the directory as it does right now for every single request which is why it's not on my default to Rails 3 right now we scan the directory as upfront once it's in your production mode after all we get a hash and that way determining whether or not there's an asset is a hash lookup and we can do this performance and do that in problem and we actually just finalize the details for how this is going to work this morning, so great. The second really major feature in the new engines is the idea of a self-contained engine so in Rails 2.3 and Rails 3 and the old engine plug-in engines were basically a hash that let you take part of your application and put it somewhere else but at the end it basically became one application in one and you couldn't for instance have middleware that was around your engine you couldn't, if you namespace things then you had to deal with the effects of namespacing everywhere so everybody knows sort of how namespacing affects your application if you have an added namespace it changes a bunch of things and that's sort of fine for an added namespace but if you have an engine you don't want to have to modify all this stuff and understand how namespaces work just to be able to use them so Rails 3.1 adds the idea of a separate namespace engine and a namespace engine is essentially a standalone Rails application without the boot up stuff so you only need to boot up and initialize an app one time but you may have applications that are standalone that you maybe want to wrap mouth of rack applications or just use as a rack application anywhere in your app and this, the isolated engine does that one really obvious, this is from Drogos Peter's blog one really obvious thing is that you don't want to be including your main application's helpers in your engine you only want to include your engine helpers so that's like the first thing that's sort of important and sort of illustrates the kind of problems that you have when you start thinking about this another thing is that now if you use namespace and I think the name namespace will probably change to be more revealing about what's going on that's sort of the first implication of this every engine that is an isolated engine gets its own namespace and so its own route which means that you can do stuff like posturl.points at the engines you still might want a point at the main application one and so you can do mainapp.posturl and that goes to the mainapp and vice versa in the main application you can do blog.posturl another thing is by default if you do this it does blog.postpath.post and that's because it flattens out the namespace and makes the url helper but what you really want is postpath so basically if you're in an isolated engine and you form for a post it sees that you're in an isolated engine with a namespace and switches to use this postpath and all these things are really just designed to make it easier to build isolated engines almost like they were regular applications so you don't have to be thinking about the namespace all the time in a way that's unusual for building a regular application similarly if you did this you would get blog.postpath and you got to use params blogpost in an isolated engine it's just postpath and you can use params postpath and that's because you only get one post at a time so you have a bunch of forms that are from different engines only one of them will be submitted at a time that way you don't need to be namespace the way this will work is that you can mount an engine like this and this is actually the same API for mounting any rack application so the same way that you can say you can mount some synapse to blog you can say mount blog engine that's because blog engine is just a rack application and in your application then you can say blog postpath like I said before but you can change that name so blog is just whatever you can find the namespace to be if you want a different name to be used for routing helpers you can pick one by using as and then you get my blog postpath you can also call for all the helpers that take objects to make URLs out of them allow you to pass in as the first parameter now of an application so the main app this is how you make a form for the main app in an engine and similarly this is how you make a form for the blog in the main application so that basically we've made the idea of applications for a class there is only one application but the only thing that an application does that's different from what an engine does is that it foods up and initializes and has a bunch of middleware that only makes sense at once like the static middleware but engines at this point they actually applications in character from the engine class and engines are basically full fledged first class applications just without the boot up there's a bunch of other things that you have to do like image path doing the right thing DOM names rendering partials ITN and that all just works almost the same way that I showed before basically pretend there's no namespace because you've told us that you have namespace and then everything just works so again give this guy some love I think this is already in Rails 3.1 I'm not really excited about it and couldn't have happened to leave him to do the work so that's great the next really big thing is even a further shift toward HTTP so this is hgashing in Rails 3.0 this is the cap-stage method you can see stuff like fileUtils.vaker and that's because it's hard coded to essentially the file system and end-to-next-package strategies so the idea is that you put them in the public directory and then end-to-next-to-package and that works great for a lot of cases but doesn't work great for a lot of other cases and in general it is kind of a weird strategy especially as Rails 3.0 has gotten more as Rails 3.0 has gotten more agnostic it's kind of weird that we still have sort of a core piece of functionality hard coded to the file system especially as you embrace HTTP or else HTTP actually has mechanisms for this so for instance if you wanted to optimize your caching suddenly you have to do a whole other thing because we built something that only applies to the file system so Rails 3.0 has built an HTTP caching but we're going to make it even more hard coded which is weird but no we're going to make Rails 3.0 more associated with HTTP which is actually a pretty big deal so how many people know what these methods are? Raise your hand if you know what these methods are so more than a lot of other places I've asked that question but not a lot so basically what these methods are is they let you do something like this so you can say ifstale and then say itag and point it at a post and what that will do is Rails will automatically generate itag and that's just basically a unique identifier for that object without you having to sort of know how to do that so Rails has heuristic for figuring that out and then any subsequent requests and I'll show in a minute how the HTTP protocol works around this any subsequent request will not go and render the template so that's usually the most expensive part the expensive part isn't like finding something from your database so this allows you to say I will serve this template once and then any subsequent time I'm just going to basically bail at this point and tell the browser that I can just continue using whatever it was using just FYI you probably don't know yet people aren't that familiar with custom responders yet you can easily take this and make every single user respond with automatically do that detecting if you want, it's quite easy and Jose actually has a responders gem that doesn't so I expect that as people get more familiar with how the respond with API works they'll start building things like itags into how every one of their controllers works so let me talk about caching a little bit so this is the normal way things work every time you want something you add a fresh one it works like this instead the client says get hello that goes to the server and the server generates an ETag that ETag is just a unique identifier for that object and it sends back the ETag to the client and the next time the client wants hello for whatever reason it says if not modified it sends the ETag through goes to the server I already, that's the same because you said fresh went to ETag and it returns back to 304 and that means that you didn't have to do most of the expensive logic with that action you just have to do the work of going and getting that one object from the database and having rails generate ETag so now you only have to do one hit for slash hello plus a little Ruby logic but not a lot of Ruby logic so that actually works pretty nicely the problem is that if you have a bunch of browsers which everybody does and since the browser is really the only thing in the equation that has HTTP semantics for most people all of a sudden you get n hits as many hits as you have browsers and that stops being as useful so one really good solution is to put something in the middle like Occamai, Occamai is the thing that's getting all the hits and then it's doing the ETag work so it basically becomes the browser and now your Rails app only gets hit once with a full request another way to do that is with varnish so Baroque has a varnish layer in the project you can install it pretty much on your application but again varnish is not necessarily something that everybody has in all cases so it's something that is nice and has this nice effect but isn't necessarily built in and because of the fact that varnish is so unusual for people and because of the fact that Occamai is really expensive people haven't really embraced FreshWend and Stale clusterfarm the APIs that Rails gives you to make this work as much as they should have there's a really nice way to do caching validation built into Rails, all you have to do is say FreshWend, ETag, give me the object done, it's like a one-liner where in Java it would be like a 20-liner to do the same thing we have a really nice way but because it's sold, it's not so useful people don't use it so much so what we're going to be doing in Rails 3.1 is shipping Rack Cache together with Rails and now you have the same semantic so in case you're wondering, Rack Cache it sounds like, oh that's nice for development but you would never do that in production because it sounds like it would store all that caching in memory Rack Cache is actually pretty smart it stores all the data in any caching later has an API for that and the Rack Cache that Rails is going to ship with is just going to use you whatever your access port caches so whatever you said your Rails cache to be we'll use map caches that have said the policies and we'll use that, if you have a special ready store and you use that, we'll use that so essentially we're going to be storing things for you using htd semantics and that means that if you just decide you set fresh one or a scale question mark in your actions you get caching for free and the really nice thing is that unlike page caching, which you can still use still use page caching but unlike page caching where you have to actively invalidate the cache and let's say this is no longer valid with htd caching you don't have to do that because the eTag system is pretty flexible and you get to actually look at the eTag do a very little amount of Ruby code and return right away if you need to which means you don't have to do invalidation so that's pretty nice and it gives you like 90% of the performance impact before a page caching with a lot less caching one thing that this does out of the box is it lets us, we're going to build in and talk about this in a second compilation of static assets and we allows us to edge cache things like scss so right now you want to take static assets and compile them upfront because you don't want to be having to render them all the time but if what we do is we have caching baked in now we can actually just make scss for coffee, script or whatever a thing that gets rendered right away on the first take and we don't have to be sending it every time the caching layer is automatically handling so we don't have to worry about great compiling in this part of our deployment process or anything like that we can use htd caching to handle how strong or weak we want our caching to be and we can build that into rails so a really nice thing about this again is that it means that fresh ones which are relatively unknown can become relatively known because they'll suddenly have a really big impact they'll be a powerful way of doing something like page caching but really cheaply in terms of mental over it so the next thing is the next really big feature is assets and the thing about assets is that erb and scss are really conceptually very similar there you have something and you want to serve something else in the browser and you have to make it go through something first before you can do that and unfortunately erb is something that you don't have to do anything for you just make it that erb file and scss is something where you either have to have a very very slow weird middleware that they don't like and only turn on development or you run like great scss compilers like that in your deploy script but you can't just, it doesn't just work like the erb and the solution for this is to build in the idea of schub basically compiling css and compiling JavaScript so instead of it being public a public directory which is basically a junk drawer for everything if you have compiled assets if you have scss if you want to minify it if you want to combine a bunch of assets together all this becomes part of the process just like compiling erb is part of the process so you can move your assets in the Rails 3.1 to App Assets I think we'll still support public directories for non-compiled assets but for assets that are scss or coffee script or less you can move them to App Assets and then they'll work exactly the same as erb or app.css.scss scss will be a rendering engine we'll ship with scss so it'll basically be just like erb and if you want to support if you want to use less or something like that it'll be really easy for the less guys to make a plug-in just like to optimize middle plug-in to support it and then you'll also have multiple directories so the way scss works today like I said is that we compile at a run time compile at a run time goes to the file system like page caching and then we have an engine exit this kind of falls apart again with a read-only file system right so as you can see it's a similar problem to what we were talking about before page caching right and the solution is the same the solution is instead of having an scss compile phase we have scss template handler that serves an hd response and then you can drop and rise or optimize it just works so again there's a lot going on here but the short version of all this is scss becomes like erb scss is a thing you can say at the end of app food.css.scss and then we'll have the notion of template handler so everybody who does this doesn't have to make their own mechanism for compiling all that of course isolated engines get their own app assets so isolated engines you can just put things in there and then it will work just like I talked about isolated engines and the cool thing about this is that it comes really easy to package up something like j4ui or any one of these widget libraries that has scss, javascript and images all in one place and right now you have to take that out you have to put it somewhere you may have to go through it and change the urls because maybe you can put them in a different location right so there's sort of a process of installing j4ui it becomes really easy for us to ship j4ui as a gem and then because of the fact that like I said before assets are just assets are just controlled by the rack patch layer add that like 500 slides ago now you don't have to do any copying right so you have images going to public directory javascript and css going to have assets directory in your engine and you just do gem install j4ui or rails j4ui and now you can just do javascript and clutag j4ui.js and everything just works so I'm sort of talking about the mechanisms that that will be used from that but it's a nice it will work nicely out of this it's quite in so DHH talked about this a little bit Chris announced last week I guess that compass and laminate are merging and I'm going to talk about what that looks like but the important thing is that it will be built into rails j3ui so basically the way it will work is that your images directory will have a new sprites directory inside of it and then each every sub directory the sprites directory is a bunch of images that should be combined into sprites the basic usage is that you will just import sprites slash toolbar and then you include everything and you get this css item and so you can just add classical to the school bar new to anything and it will just work so basically this means that instead of that if you have an image like toolbar slash new you get a bunch of css classes that you can apply to whatever and the sizes and background positions and all that are built you probably don't want to do that though you probably don't want to have to be forced to use that school bar new in your markup you want to have the markup in semantics so more conventional usage might work something like this you can import sprites school bar so sprites school bar is just a name that is the directory for your images and then you include tool bar sprite new, tool bar sprite open for every class and then you get this item and now you can now those classes get automatically sprites so now you have a div with an idea of tool bar and classes new and it just works I talked about auto flushing a little bit on my blog I'll just reiterate sort of so we render the template first template populates a bunch of content cores and then we render the layout the layout gets the content cores and then when we're all the way done we send to the browser ideally you don't want to do that ideally you want to be flushing to the browser as quickly as possible so Rails 3.0 looks more, Rails 3.1 will look more like this where you basically you start by doing the layout first and as soon as you say yield JavaScript you go into the template until you get a content core JavaScript and then you go back to render the layout it doesn't actually change the programming model so this is a nice thing about this the details of how this works aren't hugely important because it doesn't really change the programming model what you're already doing today what you're doing today is you have a template that has a bunch of content cores in it and those content cores you have to render the entire template first because that's how we do it and then you have to render the entire layout in order to get those things this is just a way of modifying the control flow so that we can jump back in for it but from your perspective you can still make a template the template can still have content for it you can still make a layout that's still yield and we just will automatically behind the scenes flush whatever we think we can I wrote a blog post about this that is a lot more details and has an actual sample code for how this could work the important thing about this though is that there are some caveats that are not accepting handling so once you flush you obviously can't take the flush back so I think a lot of people when they saw this they were like surely there's a way they're kind of busy because you were ready to flush the whole point is that we flush before we've done some other code so there's an exception handling problem that we really can't get around and that will mean that we will have to say that it's optional and something that you have to turn on not it's not something you have to turn on and then you have to say flush all the time so this is something that we spent a lot of time thinking about in PHP there's a flush method that you can run and that has a couple of issues one of them is it's just annoying to have to have to know about what you should be doing but another problem is that it's sort of a one-time deal with Rails you'll flush and then we'll have to jump into the layout for the template execution where this solution lets us jump back and forth and it lets us not change the programming model at all while giving you a really nice experience for your users so it's cool but we're definitely going to have to make it something that you turn on and sort of accept the consequences because there are things like exceptions not being able to return a 500 status and you just have to live with it we're going to use trunk encoding which is an HTTP spec around this sort of thing that lets us send headers at the end and it has some more flexibility than I think a lot of people realize but that doesn't let us change the status of it so we'll never be able to make a 500 if there was a 200 and that's what yes I'm going to ask about the exceptions in the earlier talk sure I'm running out of time as long as you fill the exception in the controller yeah so obviously controller exceptions will be 500 the problem is that if you have in your template you have the exception that gets called because you're lazy you did a query lazily which is good now we can set up our queries in the controller using an API that looks a lot like what we did before that wasn't lazy now we do auto flushing and so all we're doing later is we're iterating over a query and we think we've already gotten it but we didn't because we have basically all this infrastructure now to let us do everything really lazy and flushing really takes advantage of that because now we do almost nothing in the controller you might result in an error for some reason you might have made the query wrong and then you get the exception and there's really not nothing you can do about it so I think the infrastructure is really nice that if people either write reasonable code or don't do yes or don't do or can put up with the consequences and we come up with some good solutions I think that would be great but we can't turn it on for everybody because there are consequences and I think such a product is probably the API I also don't know if we'll get it to 3.1 but it's something that we're actively working on and we'll get into the next release in which we can have something that works I'll just there's a couple more things I have in one minute acceptron this is basically MIRV's exception controller so you can rescue exceptions with the controller that has access to everything so you can respond with, render, whatever a lot of people in we've got this convenient in MIRV and you kind of just want to do something that you can do with a regular rail controller you want to render a template and have it have access to the request but you can't really easily with rails 2, 3 or rails 3 and this makes it really easy to do again there's some risks involved because you may end up raising another exception you may have an infill loop of exception so this is definitely also something that people want to opt into if they know what they're doing and are really careful about now raising another exception or weird things might happen MIRV had like a, if it happens five times we time out but seriously it's a good feature it's a nice feature but it's arbitrary code so don't have another exception again it's something that won't necessarily be there by default but it's a nice feature that will be there if you want it in rails 3.1 a couple last things performance we're going to be working on performance in a lot of rails 3.1 Aaron's going to be working on an arrow having to work on an arrow should be awesome I'm going to spend some time on an action pack mainly because I did a lot of work on action pack and then Dink for the last three months and I suspect there's some regressions and I also know that there's a lot of stuff I didn't do that I know that I could do so just spending some time on an action pack would be great and Emilio's been doing some work on action record so between Aaron and Emilio action record should be fast in 3.1 and I should be getting action pack for some other people as well and I think this is a possible goal just because we did that much more than that in some parts of rails 3 and we just need to deal with some hot spots and that's it