 So, JRuby on Rails in 2009, I'm the third developer and engineer, my name is Nick Seeger, and I want to start today by asking how many of you have heard of JRuby, you might have heard of it, it's a Ruby implementation and I went on top of the Java virtual, whoa, wait a second here, I guess I don't need that slide at this conference. So, I thought I'd take a step back, sort of like Charlie and Tom did, and kind of ask where we are with Rails running on JRuby, and also take a look at where some of the pain points are, some of the things we can improve upon, and what we're able to do well, and take that and kind of paint a picture of where we're going to move forward. So JRuby on Rails in 2009, where are we? How does it go? It works. Alright, thanks. Alright, I suppose I should go on. I thought, I wasn't planning on this right away, but I thought I'd take a moment and just kind of recap the actual JRuby on Rails experience from a demo standpoint, so I'm just going to go through the basic act of creating a Rails app, and deploying it on a job app server, just kind of reset the stage for everybody. Before I do that, can I ask how many people are deploying JRuby on Rails apps? So, it looks like a good third, or almost half of the room. So, this won't be new for all of you, but it'll just give you an idea of kind of where we stand. This might don't. So, I did a little bit of prep, so I'll show you that, the first thing you need to do is, after you install JRuby, you would want to install the gems, including Rails, and there are a couple of the gems that you need, and I'll just show you that I had those installed. So, you need the JDBC gems, and then you need the JRuby gem and Warbler, and I'll be using all these tools a little bit later. So, I'll just show you that these are in fact installed already. So, there are those. Yeah, I'll get to that later. So, of course, the first step with any Rails app is no different with JRuby. If you've got Rails installed, you just run the gem, and you can just create your Rails app that way. And so, now we've got our Rails app here, as you're all familiar, and the next step we would do is a one-time step that you need to do just for JRuby apps where you want to use a database. Then the latest ActiveRecord JDBC release, there's a special JDBC generator that inserts a couple of files into your Rails app that lets you use the JDBC connectors. Can you hit the bottom of the screen? Yes, let's cut it off, so let's do it like this. Right, so we've added the JDBC bits in there, and at this point we can actually do a rake db create all. And this is the standard step in Rails where you create the databases that your application will use. Notice that I created the Rails app with just the default settings, so what it actually did was it created three SQLite databases for me, which are, of course, empty now, there's nothing in them. So, in order to actually add something to our app, we'll just add a scaffold. We'll just add a person's scaffold, we'll just give them a name field, just so we have something to show that we're storing stuff in the database. And the main point with this so far is, I mean, obviously this is nothing new. The main point being that, up until this point, the experience has been identical to the normal Rails development experience, except for that one little generator step, and I'll show you that it continues that way. So then we can run rake to migrate the database, created the people table, and then we can actually, should be able to start the server at this point. And I have a module installed, module runs fine on JRuby as well as probably a few of you know that are aware. So I can go to the people page. Here's the people app, and you can go and add people into the app as you would expect. And everything's working as expected, so great. You can also notice that you can actually go into the console view, and that works just fine as well. So the IRB console that comes with Rails pre-loaded into it works great in JRuby as well, and we can do it first in the hall. Of course, we see the two records we just added. Okay, so the next step, we've got the app running, and I want to actually take the app and convert it into a war file so we can deploy it in the app server. To do this, we need to add a couple additional steps. The first thing I'm going to do is I'm going to change my production database to use MySQL instead of SQLite. So down here, MySQL, database, JRubyCon. Now I'll just go, I'll create that database. So that database gets created, and in fact we do, we look in the MySQL console, we see the JRubyCon databases there. So then we can run dvmigrate in production as well, and that works just fine with MySQL. And now, one more step we need to do is I'm going to create a war file of the application with Warbler, and the one thing I need to do is add some gems to my environment IRB to tell Warbler that I need those gems in my workflow. So I'll go in here, I'll add config gem, active quicker dvc adapter, and I'll add the MySQL driver. I don't need it, Rails doesn't need to actually load anything, they just need to be present in the gem repository's environment setup. And then I can simply just call the Warbler command that comes with Warbler, and this will do all the work to create a war file of the Rails application. And all I'm going to do is I have a Glassfish server running in the background, I'll just deploy the Warfile to Glassfish. That's the Glassfish deploy command. How big is that Warfile? How big is the Warfile? The Warfile is 14 megs. It's a little hefty, but considering that it's completely self-contained, it's actually a pretty nice package. You can take that and drop that into any app server on any OS that has a Java virtual machine running. Maybe it's not so bad. So the Warfile is deployed, then we can actually go to the Glassfish server, which is running on 8080 by default, and the Warfile got deployed with a prefix of jrubyconf, can go to people, and here's the app running inside of the Glassfish application server. Of course our database is empty, so we can add people to it again, and everything works just as we expect, same thing. You can see that the main point of this was to show you that even though this is a very simple app that I showed, the JRubyand Rails experience is very much similar to the standard Rails development experience, and I don't think there's any reason not to consider using JRuby during your normal daily development, especially if you're tired in the Java environment. So quick word about what works with JRubyand Rails or what makes it a really good choice. Obviously the JVM is a rock solid virtual machine that very rarely crashes. The JVM works by, you set a memory limit, number limit on the amount of memory the VM can use, and when it hits that point, rather than crashing like the CRuby interpreter will just tell your application that it doesn't have any more memory and you'll have to figure out how to deal with it. This makes for running long servers, long running servers much more, makes it much more stable and much easier to have trouble free production environments. And of course the deployment choices aspect that I just demonstrated, you can run it on any Java application server anywhere. Having those choices is helpful when you run into environments that are a little bit more locked down in terms of what kinds of software you can run there. And then we'd like to say that JRubyand Rails is fast enough. I'll show you some actual numbers in a little bit here. And you'll get an idea of what that means. So what doesn't work? Well, it's not fast enough. That's not to say that it's not tolerable. I think it definitely works well. In fact, I think those of you who are running JRubyand Rails apps say that it's not a problem usually. But we take performance seriously and we'd like to think that we could find new optimizations, new ways of running Rails apps, new things we can do on the JRuby side of things to make running Rails really sippy. You didn't see this during the demo and it was sort of on purpose. So the happy path development works great with JRuby. But then once you start to stray off the path and actually do normal development tasks like use image processing libraries, use JSON libraries, use typical plugins that you would actually want to use during Rails development, you start to feel some pain because there's a little bit of mismatch in some of those. And I'll list out what some of those are later and I'm sure some of you have run into a few of those from time to time. That goes as well with incompatible plugins. Some people Rails plugin authors just don't target JRuby at all and they may be using a few of the corners of the JRuby VM that are not compatible with CRuby implementation. And backgrounding, forking and scheduling and running tasks in the background is an area that frequently comes up on the mailing list and the IRC channel. People wonder how to do these things with JRuby and again we're limited somewhat by the Java virtual machine and its inability to give us full control over the operating system process launching. So there's a lot of backgrounding solutions that you typically run across with Rails apps don't translate over easily to JRuby. So another way to look at how JRuby is doing with Rails is to actually look at the Rails test suite itself and this is a graph of JRuby running all the Rails tests except for the action active record is not on the slide I'll show those to you in a minute. I have test results for 234 and Edge and you can see we still do have a few errors but there are a few zeros in there as well so we are running a couple of tests we do have a few issues in active support a few issues in action pack and there's a larger number of errors on the action pack Edge and there's one bug with some HTML escaping that happens with rendering but I just ran into just noticed the other day I haven't tracked down what the problem is so that should not be I would think within a few days of looking at that problem that number 80 for the action pack on Edge would go down significantly so we're actually doing pretty well with Rails tests at least on this side of the equation So then we move on to active record JDBC of course this is the active record adapter that allows you to talk to databases using Java's JDBC interface and we actually started working on this way back in 2006 about the time we first started running Rails we started to write a little ruby bits of ruby code that used JRuby's Java integration to talk to JDBC and basically write an active record shim or an active record connection adapter that would do the work for you and we've been basically steadily developing that library ever since and so here's a look at where it is today so here are the active record JDBC here's how active record JDBC performs on the Rails unit tests as Tom mentioned in the keynote my SQL is pretty close but still not quite clean we have about 19 to 20 errors I should say about 15 to 20 errors depending on which version you run on Postgres and SQLite go up from there Surprisingly the existence of these errors is not enough to prevent you actually running Rails you saw that I ran SQLite with the scaffolding just fine there probably will be some cases where that will bite you and if you do find those you'll be happy to hear about them so we can get them fixed now this is so that's the unit test I showed this in the demo how with the latest release there's a little JDBC generator the process for getting JDBC built before this require much more non-stator workflow this makes things quite a bit easier this is the only step you have to do now and of course as far as what databases we claim to support this is the list of open source databases how many people are using these databases in their applications this is the list of proprietary DBs that we've had patches for one time or another I can't claim that any of these are nearly 100% more than likely these are probably lagging behind the open source databases how many people are using one of these databases in their applications it's my opinion that we actually the target market that we're looking for with developing Rails apps is actually going to be people that have shops that use one of these databases it's a little bit tricky for us that we don't necessarily have customers that have the domain knowledge and expertise required to get them running cleanly so this is an area that hopefully in the future we can find a way to shore up so as far as pain points with Active Record JDBC the obvious ones we have to keep up with Active Record we have to follow edge we have to make sure that it runs clean with whatever changes are happening in Active Record and fortunately Active Record is pretty stable so that hasn't been too big of an issue lately but it certainly has bit us in the past there's also the issue of trying to maintain compatibility with multiple versions of Active Record with a single version of Active Record JDBC and our approach for that has lately been to just punt on the older versions so I run tests going back to Rails 2.0 Active Record right now when I develop Active Record JDBC but even those tests aren't completely clean so your best bet is to pick the latest version of Rails and the latest version of Active Record JDBC the lack of prepared statements of course is something that you're probably aware of as just a de facto issue of the way Active Record is built or recoded and this hurts on JDBC quite a bit we can't necessarily fix this either not easily and this is a common complaint we hear from people especially on Oracle and some of the other big iron databases where prepared statements are really the way to get extra also performance out of databases and to not use prepared statements is to blow out the query optimizer on those databases and that's a problem as far as actually developing with Active Record JDBC if you want to use your own database that maybe I didn't have on that list before the learning curve for the code base is a little bit high it's not clear how to plug it in and how to build a new adapter for a new kind of database and of course there are just too many to keep track of and we can only support so many who need the community and help out with those so what I'm hoping for AR JDBC 1.0 I don't have a time frame in mind for it yet but I'm hoping probably within the next 6 months we can move towards a 1.0 which shouldn't be that revolutionary it should just hopefully contain a simplification of the code base make it easier to understand easier to see where things are laid out so that people can dive in we want to have a mechanism so that people can declare their metadata and their type mappings for their databases externally to the actual adapters so you can have a configuration file of some sort that's in your application that defines those mappings give the application developers more control because we're just not going to be able to cover all the cases if somebody's using the the XML type in Oracle database or the DB2 database we're probably not going to have direct support for that built into AR JDBC and hopefully with that simplification and that process of externalizing those database mappings we can attract more patches more committers and hopefully attract fewer bug reports as well so next item to consider in the retrospective is Warbler I'm going to speed up a little bit here so what works with Warbler you saw it in action it makes building a war on a Rails application a breeze it's pretty straightforward to configure in this happy path case I actually didn't have to do any custom config in Rails environment today I had a couple gems what you didn't see underneath there is we've actually recently extracted out a separate gem that contains just JRuby itself so that when you upgrade Warbler you can upgrade JRuby independently and the actual JRuby JAR file that gets put in your war file can be versioned separately so that's a nice addition what doesn't work is that it's above and beyond what I showed you and the way that Warbler builds the war file that actually creates a copy of all the files in your Rails app in a little staging area and uses the Java JAR file to create the war file and there's a duplication aspect of that it makes development turnaround difficult and there's also an issue with Warbler where if you have custom break tasks in your application that generate more files to be included in your Rails app one example is the Asset Packager plugin or similar plugins that do minification and compression of your Rails assets it's a little tricky to get those to actually get injected into your application because the way Warbler builds the war file is to do a scan when the rake file is loaded of all the files that need to be included so if you actually run rake tasks later it's missed the boat on telling Warbler which files should be put into the war file so I'd like to try to correct that so the general idea for the next step of Warbler is to combine it with Bundler and if you don't know about Bundler that's the new gem sort of vendor and utility that you hood cats and Carl Lursche have been working on for Rails 3 and I've been starting to use it more and more with a lot of my projects not just Rails apps it's a really slick way to just basically build in a little sandbox specific to your one application and the other applications are isolated to that project so it seems natural to have Warbler take advantage of Bundler for taking care of gems so what do we call this? I don't know maybe you call it Wunderbar or something like that name changes you have to be determined maybe it won't be any different but anyway so the idea is to use Bundler and maybe get rid of the copying try to assemble the app in place and then maybe you need to use RubyGems at runtime so that Rails start up can be a lot faster and try to be smarter try to make it so that we don't have to do as much configuration even for a real world non-trivial Rails app JRubyRack is if you don't know it's actually a little circular filter library that Warbler includes in your app for you and that's what allows your Rails app to actually be modded inside a Java app server I've been working on this for about a year and a half now too and the general idea or the general thought about JRubyRack is it works and that's a good thing and it even works on App Engine believe it or not we didn't have to do very much work at all just to get JRuby on Rails apps running directly on App Engine and JRubyRack is the library that allows that to happen it does probably need a little bit more love I feel like the code that's there is getting a little heavy for my taste I have never really taken the time to do some heavy work on the code so I'd like to do that and then there's some other issues with how other servers handle Rails apps around whether it can load files from the files through direct file access or through resources or class loader resources some application servers prefer to run Warfiles completely unpacked so things actually stay zipped up and that can cause problems for Rails if there aren't files on the file system we've learned a few things about how Rack works in the process of building JRubyRack and there's some things I'd like to go back to the Rack community and talk about possible changes around lazy loading of the environment hash where it doesn't seem like it's necessary to build up a full Rack environment for every single request we don't need to use all those values and then finally again the App Engine guys have been communicating with me quite a bit and I've been ignoring them sometimes so I'd like to get back and make sure I can roll back some of those feedback into the product so as far as JRubyRack 1.0 no big plans right now but hopefully we can continue to improve it so performance of course performance has been the big elephant in the room for us with Rails for a while so if you look at, this is just some simple act record benchmarks that we created in the act record JRubyC project they're nothing fancy the labels at the bottom, find all create model.new.valid it does these things in a big loop basically and just measures time so it's not doing anything extensive but it gives you some idea of what kind of overhead the database access layers provide we show that act record JRubyC is actually pretty competitive for most of these it loses a little bit on the right side of things it seems to be pretty competitive and then if it's only in memory operations like validation it has no problems no problems there and of course we're probably not really touching the JDBC code at all there so that's why that is the way it is so but that doesn't mean there isn't room for improvement of course the next part of the equation for Rails is there's this notion of dark matter Ola Beanie has probably tried to profile and optimize Rails on JRuby once every three months or so once every quarter and it always comes back with an email to the core group and says oh I ran some new numbers and I have no idea where the time is being spent and we've done this repeatedly and maybe it's just a function of Rails itself but we'd like to think that there's something that we can do to crack the nut of Rails being slower than we'd like I guess and so if we look at this is just a graph of a mean average median and max and you can request the latency and I ran these numbers on the Spree commerce application which is a decent non-trivial web app and when you hit the homepage of Spree it does product lookups and category lookups and things like that and renders out a product listing page for you and so it's actually doing some stuff and you see that JRuby is pretty competitive here the error bars on there is actually the standard deviation and we show that if Ruby gets a chance to warm up the standard deviation is actually slower for JRuby than it is for Ruby 186 so that's generally a good thing but we'd still like to see the numbers get even more aggressive in this here's a breakout of how the request time is spent approximately between the three components of Rails and you can see that model actually the model stuff is about this with controllers hmm I thought the mouse and controllers switched around so what you see here is the controller is taking a bit more time than the model here on the JRuby side the controller is taking more time on JRuby than on Ruby 186 here there's something to dig into there but there's still the fact that there's this huge swath of rendering but that's the case for both of JRuby so what do we want to go from here with JRuby on Rails now we know where we are what comes next is the first standard cliche cliche is the enterprise but what does that really mean so I think the first step we need to do is really take a categorization of all the broken windows we have and I kind of hinted at some of those so far here's a little more concrete listing so I talked about ActiveRecord JVC how we've got an ever increasing number of bugs there we seem to be falling behind on we've got things like the JSON library issue there's a pure Ruby version of JSON there's a C extension and there's a JRuby version and they all have different names and there's no easy way to just make sure if your app uses JSON you get the right one you have to know to install the JRuby version versus the C version that's a barrier to new users another one that a few of those have found out who have been doing JRuby on Rails development is that the pure Ruby memcache client really sucks on JRuby it's problems with well just threading and the way sockets are handled and it doesn't perform very well at all and so one of the LinkedIn guys I don't know if you're hearing Kai but he worked on a memcache client using a Java memcache library and that has worked pretty well for the few of us who are actually employing that production or else apps we used it at the previous project I worked at it's on Project Kenai uses that there are choices like this where it's another case where if you're using memcache with JRuby you really want to use this but there's no easy way that you would know that that's what you need and so imageboot I mentioned here that's the image processing library that uses the Java 2D to do image processing and it's interface compatible with image boot image science for C Ruby but again it's another of those things that you have to know about so if you're running on a Java app server we have a version of the Rails session store that can be stored put in the Java servlet session that's not available by default it's not always the best choice but it's something that people might find useful especially if you're integrating with a Java web application and then of course we have an open SSL gem which gets used for a lot of stuff and it's sold separately it doesn't come installed at JRuby and it's shyed away from trying that because we're unsure of the crypto issues surrounding packaging a crypto library in JRuby and having it downloaded and so we've been kind of waiting for a lawyer to step up and clear the fact that we can do that Q's I mentioned backgrounding and I think this is something where we really need to have an out of the box solution that says if you're doing typical messaging with your Rails app this is what you want to use there's a number of things out there there are things like active messaging there's a number of BJ and DJ and delayed job queues and things like that but I'd like to see one or two standard solutions pop up that we can kind of bless and say try this first and if it doesn't suit your needs then you can go on from there out of curiosity do you notice this pickle monster on that side he's about to go eat that poor family queuing up for the Tower of Regina okay so I'm running well I'm going to run a little bit over I want to show you this quick demo of something that I have here and I'll breeze through the rest of the slides so one thing I've had in JRuby for a while is JMS support and I've got an app wired up here and I'll just show you what's going on so I have an initializer I actually load a couple of job files into the JRuby process you can see that here and basically those are the JRuby rack jar and an active MQ jar and then I have required a couple of Ruby libraries that JRuby rack has bundled in and then I'm just going to set up an active MQ, a configure an active MQ instance and by default what this does is it sets up the it sets up some state inside the job virtual machine so that you can use JMS and by default it uses an in-memory active Q server then I wired up a little Q controller and I have a couple of little class methods that I inject into action controller so you can say that your controller acts as a subscriber or a publisher in this case I'm doing both and then I just have a simple index method where when the index method is hit with a get I publish a message and then this little thing G do you know G? it's like B I'll just hit I'll just hit the Q controller with curl there you go okay so that's a simple example that I've worked on for a little while but never really got a chance to fully play with there are actually a couple of guys in the community that are using this particular support and I haven't gotten full feedback from them yet but this is the kind of thing where I think running with something like active MQ is certainly not a solution for everyone's problems but I think it's a good starting point especially if you're doing development it's really nice to have that queuing server running in memory and then active MQ may not be the most stable and most scalable messaging server but you can take it out of process and kind of move from there I think we need to get to the point where we realize that we're not all running websites like facebook and twitter maybe someday you will but right now you're not and if these lesser grade messaging queues are good enough for you why not use them right now and if we can make them available in a way that's really easy to use that's even better so certainly maybe this is talking about sneaking rails in the back door and what I want to propose to you as you go forward and as you look for new gigs and you maybe work with clients that have job environments you want to try to use Ruby and Rails so you start to think about proposing to them that what about rails alongside your existing system so what would it look like to have an app that has rails running inside next to Stratzer or Spring or an existing Serlet app with EJVs and JSPs or does a google web toolkit app with rails in it make any sense at all maybe not and what about rails I mean that seems maybe that just seems like a bad idea but if you saw the talk yesterday if you saw Aaron's talk yesterday he presented the idea of embedding a PHP runtime inside of Ruby which is preposterous but I think I like the idea from the standpoint of sometimes we need to look to other technologies and look for uncommon ways of putting things together or putting tools together as a way of learning about what we could do better so we can go over to Grails and these other Java web frameworks that are getting some mind share and look at what they're doing and see if there's something we can steal to so the idea I want to start to run with going forward is the idea of actually having a JRuby on Rails stack and what this means exactly I don't know for sure we don't have an organization built up around this yet but I want to start to think about like I said having sort of go to components that are part of the JRuby on Rails experience that people can just rely on that help people build JRuby on Rails apps build them in Java environments and be successful and make it easy to get going so at the base of that stack of course is Rails 3 and so as you know a lot of you know of me probably reading about Rails 3 basic idea is to take the existing MVC architecture of Rails pre Rails 3 and split it up a bit so you put some walls between the actual controller views and models making sure they're not directly coupled to each other but instead they're coupled through these vertical APIs that are very thin that are being built on top of those components so now we have active model, we have abstract controller we have the notion of a view context and we can we can basically take those thin layers and we can start to write new new wrappers for new back ends for those things and we can play around with them so I'd like to propose the idea of having wrappers for hibernate for JPA there's any graph database out there that's in pure Java called Neo4j so if you're looking for non-standard there's actually a guy who's got a Ruby based wrapper on that that's already ready to go, it's just waiting to be integrated into active model as far as views go it might seem like a bastard child but what about velocity, what about JSP what about using Java tag lines inside of your Rails views yes it sounds horrible it sounds awful but this is going to be the reality of the Java world so why not embrace it and as far as rack middleware we want to have easily available rack middleware for things like hibernate sections, database transactions maybe a way to implement sort of the filters inside of rack might be kind of interesting so to call the arms for one component would be hibernate I think putting hibernate behind active model is a really good idea so for example, here's the list of databases we support in ActiveRecord JDBC here's a list of databases that a hibernate proports to and I don't know about you but I maintain the ActiveRecord JDBC library and much rather get hibernate working if it means that I can just assume that all these work and I don't have to do anything else so the idea with hibernate for me is that you get all this database support you get a rock solid ORM if we can fit in an active model it means we can make it look just like ActiveRecord access to us today and maybe we can even build it with the goal of being able to work with existing hibernate models that are already written in Java so you don't even have to think about treating those any differently from any other active model like object in your system so Q's messaging I mentioned this before but it'd be nice to have JMS and MQP wrappers built in I guess Rails itself doesn't really have a standard API for this sort of thing but maybe that's the direction the community needs to be in NoSQL stores, things like this interesting area of course there's a link there Terracotta is another interesting technology that Fabio Kuhn wrote an interesting demo for but that's fallen out of repair for now but you have to get some of these ideas going and explore those as well scheduling there's a pretty solid Java library called Quartz which allows you to do crown-like scheduling amongst other things and you can do it in process I think scheduling is also a pretty typical component of Rails applications some enterprises will work better knowing that their schedule can happen inside their app alongside all their other code rather than having to maintain crown tabs and other sorts of things Soap, I don't know if anybody who does Soap anymore I personally don't like the idea of having to build Soap but I know for example that the JBoss guys have built Soap into their TorqueBox platform and sometimes you're just going to have to deal with it in an enterprise situation so I don't know if that's something we need to build or not and then legacy stuff I kind of hinted at that earlier it's probably something that we need to look at another call for documentation so say we actually get some of these stack pieces built let's get some unified guides and manuals and screencasts around how to use this stuff so we can get people going so going forward I want you to think about embracing your inner Java it's really not so bad don't think of the enterprise as a big Rube Goldberg machine with lots of sensitive fragile bits kind of string together with bailing wear and duct tape instead think about it as a system with components that are meant to fit together well now the reality is you say well most enterprises don't have a nice interlocking system but I'd like to give you another concept to think about and that's the idea of Watabi which is the Japanese sort of philosophy about embracing simplicity but also being comfortable with the idea that things aren't perfect and so you're going to go into these enterprises but the great thing about Ruby comes from Japan so Ruby is Watabi-Sabi I think JRuby is Watabi-Sabi Rails is Watabi-Sabi take these things and bring them into the enterprise but then also don't freak out when you see these legacy systems and just try to find a way to make it all live in harmony so of course your feedback is appreciated if you have any ideas or are inspired by anything that I talked about love to hear about them and that's all I have for today, thanks