 presentation of RubyConf 10. I hope you've all enjoyed the conference as much as I have. So to cap it off, my name is CJ Kielbom. I'm Jonas Niklas. And we work at Elab, a consulting company in Sweden. And we want to talk about the front-end testing frontier. I'm going to start off by asking all of you a little softball question here. Who here likes to test their code? Not very unexpected. We love testing in the Ruby community. I haven't seen anything like it in any other community. I guess maybe the small talk community, which I was never a part of. But we absolutely love testing. But I'm going to ask you another question. Who here tests their JavaScript code or front-end code? Somewhat, yeah. Alright, that's not too bad. That was maybe a higher number than I expected. But still, much, much fewer of you than the people who test the Ruby code. And I hope that's not because the rest of you don't write JavaScript code. Otherwise, I guess you wouldn't be in this talk. But the conclusion is that we love testing our Ruby code. But JavaScript, not so much. We rely on maybe some doing some manuals, some testing, clicking through, making sure things are working. But we don't really, in general, write that much JavaScript testing code. So why is that? I think that probably, at least for me, the biggest reason that I had a hard time getting into it was that front-end testing is very neglected in our community, especially in Rails. So the thing that really got me started with testing my Ruby was that as soon as I started a new Rails application, I had everything I needed to get started with testing it. I had all these scaffolding for my unit tests, for my functional tests. Soon as I created a new model or controller, it gave me everything I needed to get started. With JavaScript, there's nothing like that in Rails. So that means that we're on our own there. We have to figure out our own way forward. We have to pick a library to use. We have to figure out how to configure it and install everything like that and do it all on our own, which makes it harder. There's a speed bump on our way there to get started. Another reason, I think, is just that it seems hard. We think that front-end testing looks pretty hard. We're not quite sure how to do it. There's not as much documentation or examples or tutorials available as for normal units or functional testing that we do. So with this talk, our goal is to get rid of some of these notions and show you that it doesn't have to be hard and hopefully inspire you to get started with doing front-end testing in your own code. We're going to talk about two different ways to do testing. The first one is integration testing. Integration testing, of course, means that we're testing our entire application from the top to the bottom, from the user interface, all the way down to the database, and trying to use it in the way that the user would. Probably the most popular way to do that in Ruby right now, especially in Rails, is with Cucumber, where you write your features in plaintext format. I'm sure you all have heard of Cucumber, so I'm not going to go into that. But even if you're not fond of the plaintext aspects of Cucumber, you might use something like Stake to write your integration tests in pure Ruby instead. That's fine. But they all have something in common. They all need something to drive the application, to act as the user, basically, to flick around in it and fill out all of the forms and everything like that. And the way that we like to do that is with a tool called Keppibara. Oh, maybe you can't see that. Can you see the rodent in the background there? Awesome. That's a Keppibara. So, just put some context. Awesome. I'm from South America. So, we have this library in the Ruby community called WebRat, which is a fantastic library for interacting with the web application. Basically, it allows you to simulate how a user would interact with your application. Filling in forms, clicking links, et cetera. And it works really nicely, but the problem with it is that it doesn't actually execute any JavaScript. So, you can't actually test your JavaScript code using WebRat. It does have a sort of very limited Selenium mode, but it doesn't really work. And so, I looked at this and I wanted to improve it, and I couldn't really figure out how to do it. And if any of you guys were, Yehuda's presentation yesterday, you talked about how to incrementally improve the library by refactoring it slowly. Yeah, I didn't do any of that. So, basically, I threw the thing away and I wrote my replacement, which I called Capybara. And the reason I did that was because I thought it would be very hard to retrofit the main feature that I wanted for Capybara, which was, for instance, driver agnostic. And what I mean by that is that the backend feature that actually executes your specs, that executes whatever you tell the application to do, should be switchable. So, you should be able to switch seamlessly from one driver to another. We have a number of different drivers that we can use in Capybara for this. For example, we have Selenium, which most of you will know. It's basically a remote control for a browser. So, it opens up a browser, usually Firefox, and it runs against Chrome and IE as well. So, it will pop up in an instance of Firefox, and it will control this instance of Firefox, click around, fill in forms, et cetera, and so on. And the advantage of this is that it's actually using a real browser. So, it has a real JavaScript engine, a real CSS engine, and it's basically identical to what the user would experience. So, it's very, very compatible. Unfortunately, it's also very, very slow. It also requires a GUI, which is a problem. So, it can't actually run on a computer that doesn't have any desktop client installed. It's not the local Selenium version. So, you actually need some kind of window manager in order to run your tests, which obviously can be a problem on your continuous integration server, for example, which might not even have a user interface installed. So, there's also this simpler driver, which is called RackTest, and this basically works the same way that WebRack does. It runs your tests in a pure Ruby environment, and that also means it has no JavaScript support at all. So, it basically works the same way as if you switch off the JavaScript in your browser. And it can do these simple things like clicking links and filling in forms and so on, but when it comes to more advanced stuff, obviously, you have to switch to one of the other drivers. There's another class driver, Driver's Builder in HTML Unit, which is a Java library, which is basically a browser specifically written for testing. So, this library can be used for completely headless testing. So, it doesn't need a graphical user interface, and it has fairly good JavaScript support. So, actually, it can run most real-world JavaScript. It's also pretty slow, and depending on which driver you use it can be pretty tricky to set up. So, the way we actually have three different drivers at the moment which use this library. One is called Celerity, and this basically a JRuby library that wraps the Java HTML Unit API. We have Celerity, which can be used from any Ruby, and it spawns a JRuby process, and it talks to this process via socket. And that's pretty good, but it's pretty tricky to set up, and it's pretty unstable, so that's sort of the main problem. We have a new one, which I encourage you all to check out if you're using Catabara. It's called a Cataloz. It's pretty cool, but it's pretty new, so it still has some issues, but it's, I think, shaping up to become really probably the best Catabara driver. I think they can get it to work properly. And the last one we have is NBJS, which is super fast, completely headless, which sounds awesome on paper. Unfortunately, the JavaScript and CSS support isn't very good. For those that don't know, it's a DOM implementation written in JavaScript, which is kind of cool. So it's very fast, not very compatible. And the way we switch between drivers in Cucumber is that Catabara sets a Cucumber tags for you, which are tags that you can add to your scenario or feature files. And by specifying this at JavaScript tag, you can switch on the JavaScript-enabled driver. There's also specific tags for the drivers, for example, excellent and available and so on. So you can easily switch to a different driver in your Cucumber. All right, so that's integration testing, and of course the main benefit there is that we can continue writing all of our tests in Ruby, which is very, very nice. But as you see from the different drivers, it looks like if you want to have good compatibility, it's probably going to be pretty slow. So if you're doing a heavy TDD workflow where you're test driving your code to drive the design of your JavaScript, then you probably want something that's a bit quicker with a faster feedback loop. And of course that means unit testing. Now with unit testing, where we're trying to isolate as much as possible of the application, that means that we're probably not going to want to have all of the different integration points that you have when you're doing integration testing. And that means that we'll have to write our unit tests in JavaScript as well to run them in an environment that's as isolated as possible. And there's been a number of different JavaScript unit testing solutions out there for quite some time now, and none of them in our mind has really worked that well, primarily when it comes to sort of integrating with your application. So what we're using is that in our mind the best tool for the job is Evergreen. So Evergreen isn't actually a testing library in itself. It uses a JavaScript unit testing library called Jazzmine, which was written by Fibito Labs, which is an absolutely awesome testing library. It's, in my opinion, it's the best sort of general purpose JavaScript unit testing library out there. Pretty cool. But what we didn't really get from Jazzmine was this really nice out-of-the-box experience. We wanted to basically be able to just install a gem and have it work out of the box. No configuration necessary. Convention over configuration. And we wrote Evergreen to provide the best possible out-of-the-box experience. The trickiest problem when it comes to JavaScript unit testing libraries is templates. In order to test your JavaScript code you really need a DOM because most of your JavaScript code in your application is going to be modifying the DOM in some way. And in order to have a DOM you need an HTML page. And most of the older JavaScript unit testing libraries solved this by just giving you a static template. And this would reference your JavaScript testing library and your production code. And you would put in whatever HTML you need. And it would run your JavaScript test. But what we did instead in Evergreen is we have a dynamic template, which includes your template fragment that you wrote in just HTML in a dynamic template. And this allows us to package the JavaScript testing library and everything else we need in order to run your tests nicely in the gem. So you don't actually need to generate anything. You don't need to put it into your project. It just install the gem and it works. This also solves the other main problem that JavaScript unit tests have, which is that there's no way to do transactions. So if you have a static template, like most other unit testing libraries, you're going to end up in a situation where one test modifies the DOM and then the next test sees the modified DOM. And it's obviously going to be dependent on this new DOM. It's going to be coupled to the test that came before it. And that makes it a real pain in the ass to write JavaScript unit tests. I'm sure most people have tried to experience this problem at some point. So what we do instead is that we have a special div on the page that we basically empty after each test. And we fill it again with the template so that between each example in your test, the DOM on the page looks exactly the same. So we want to show you all of this and how it all fits together with a quick little demo. So we've built a simple little application here. Actually, I can zoom in a bit, maybe. That's a whiskey bronzer. So you select Scotch whiskey, I should say. So you select one of the Scottish whiskey regions here. I like Isla. So I'll select that and click show distilleries. And then I'll select Isla distilleries, such as Ardbeg, and click show whiskeys. And I get a list of some of the botlings that Ardbeg has produced. So that's a very simple application. There's no JavaScript in this one, but we have a cucumber feature to test this. So first there's some background. So there's some background setting up the regions and distilleries and whiskeys. And then the scenario simply selects a region and presses the show distilleries button. It selects a distillery and presses the show whiskeys button. And then it makes sure that we only see the whiskey that's from that specific distillery. So that's very simple. And we can run this. And now this is using Kefi Barra. Oh, sorry. So this is using Kefi Barra. And it's using the default Kefi Barra driver, which is rack test that Jonas was talking about before. So this does not do any JavaScript testing at all, which is fine because we don't have any yet. But this UI is not that great, apart from the design of it. We really like it so that the user wouldn't have to click these buttons. So we've got another version that looks like this. So you'll see that the buttons are no longer there. If I select a region, it updates the menu automatically with Ajax. And when I select the distillery, it updates the list automatically as well. So that's a bit nicer for a very important whiskey browser application. We're going to remove all the frustrations when we're using this to limit our drink, maybe. And I've added another feature to test this. Let me get this up on the screen a bit. There we go. So this is the exact same feature, except that I've removed the steps that presses the buttons. So if you compare it to the one above it, you'll see that I've removed this one and I've removed this one. Other than that, it does the exactly same thing. But the important thing to notice about this one is that I've added the JavaScript tag here. And this is, of course, using QCyber. You can use it with stake, as I mentioned before, and then you just use the Keppie Bar API to set the driver you want to use. But with QCyber, this is very easy. And you'll also notice that I've tagged this specific scenario. So that means that I can have this scenario which doesn't use any JavaScript and make sure that that still works so we can make sure that we have good accessibility, et cetera. But we don't have to run that one in Selenium or whatever JavaScript driver we're using. We can target the scenarios that we want to run really good. All right, so let's see how this looks when we run it. This will start just as the previous one did. It will start by running the first scenario which doesn't use JavaScript again. And that's all right. And then it will pause when it encounters the JavaScript tag and spin up Selenium and Firefox in this case and run the rest of the tests or the other feature in using Selenium instead. And that passes as well, which is nice. But of course, that takes a while. So if you're doing TDD and you really want to have that quick feedback loop. So I've got some unit tests as well. So if I look in the spec directory, there's a JavaScript folder here. And in that, I have two spec files. There's a region menu spec and a distillery menu spec. And these are pretty normal Jasmine tests. They have some additional things in here such as these required statements to load the JavaScript code that we're using and testing. This also means that we don't have to load all of our JavaScript code. We can load only the things that we really need. And a bit further down here, we have a template function that will load the template fragment that we use for this specific test. Other than that, this is Jasmine and it looks very similar to our spec in many ways. We've got our describe statements. We've got our it statements here. And of course, we're using JavaScript's anonymous functions instead of the Ruby blocks that we use in our spec. And the way we can run this is if we go to our application, every green includes an engine. So we can just go to slash every green in our application. And it shows us a list of the spec files that we have. And to run one of them, I just click that and all of our tests run. And I can show the passing tests here. I can show if there were any skip tests, which there weren't in this case. And this is very, very quick. So if we're doing TDD, this is great to just be able to change our test, search the browser, hit reload, and the test run in 0.024 seconds for these small tests. So that's great. But this is very simple tests. I'm not going to go into the specifics of it. We're going to make all of this demo code available on GitHub later so you can have a look at it yourself to see how it works. But this is a simple test. We're not really doing anything that tricky with JavaScript here. I think one of the things that people find the most difficult or imagine is the most difficult one they think about JavaScript testing is drag and drop. If you have a very interactive user interface with a lot of drag and drop in it, at least I had a hard time figuring out how to test that. But with Kaby Bar, it's actually very, very simple. So I'm going to switch to another branch here. And in this version, we've added a little shopping cart. When we run out, we can select our favorite distillery and then just drag the bottlings we would like to buy down to our shopping cart. So how do we test this? This is, at least before we started doing this, this was one of the cases where we just hope that it would work. Try it out in the browser and hope that the changes that we did later on wouldn't break this functionality. Which might be fine in a small app where you can manually test the entire workflow every time you do a change. But in a big app, that's not going to be feasible. So I switch back to the code here and there's, we have another feature called Purchase Whiskey. In order to quench my thirst with delicious whiskey, I want to purchase my whiskey of choice. And we have the same background here, setting up the regions and distilleries and bottlings. And then we have a scenario tagged with JavaScript that just says that when I select a region and distillery and I drag the whiskey Art Bank Provence to the cart, then I should see the whiskey Art Bank Provence in the cart. There's still some functionality missing here to actually get the whiskey delivered to you, but I'll leave that as an exercise for the audience. But of course this is just cucumber, it's just plain text, this doesn't actually mean anything or it doesn't actually do anything. So just by writing here and I drag the whiskey to the cart, it doesn't make that happen automatically. So the important part to look at here is the step definition. We have a whiskey steps file here that has our step definition that describes what the application should actually do or what we mean when we say that we drag the whiskey, whiskey name to the cart. So what this does is it uses Kappibara's API to find the DOM element that includes the text of the whiskey we named and then it finds the DOM element for the cart and then using Kappibara's API we just say whiskey dot drag to cart. So that's all there is to it. This actually will run in Selenium or in the HTML unit drivers and do that drag and drop for you. And if we test this, let me just cover features, that starts and it spins up everything we need which takes a while unfortunately. So that tested the drag and drop. That wasn't too difficult, was it? Excellent. Now there is one... that's basically it, that's all there is to it. So just using Kappibara will give you the APIs to do basically whatever you want to do with JavaScript. And that's really, really powerful. And Evergreen really helps us sort of get around the limitations of doing front-end testing or integration testing using Selenium or HTML unit by giving us a way to really run our tests really, really quickly. Actually there's one more thing that I forgot to show you. We don't have to run our Evergreen tests in the browser. We can run them from the command line as well. There's a rake task if you're in Rails or I guess in any rake environment. Oops, forget it. I always do that. So rake spec JavaScripts, it uses Kappibara as well. So it will use your default Kappibara JavaScript driver and run your unit tests from the command line. And if we configure it to use one of the headless drivers, then we can run our JavaScript unit tests in continuous integration as well, which is very nice. So that's basically it. But there's one more thing. I've got a CoffeeScript branch here as well. How many of you have heard of CoffeeScript? Right, that's decent. How many of you have actually used it in an application? I think CoffeeScript is probably my favorite thing that's happened in the JavaScript space in the last year or so. I really like JavaScript, but I'm not particularly fond of the syntax. So CoffeeScript is basically a nicer syntax for JavaScript. And its syntax is structured in a way that makes you write better JavaScript as well, which I need. It sort of gives you the structure, which makes it much easier to get a good JavaScript structure and not just throw a bunch of functions in a sort of top-level namespace, LAPHB. So Evergreen has built-in CoffeeScript support. That means that we can write our unit tests in CoffeeScript instead. What distinguishes this is we just give our spec files a .coffee extension instead of .js. So this is the same files, but written in CoffeeScript instead. And the biggest benefit is that we don't have to write function on every single line in the test. Which really helps with the flow. I love that. So instead we've got this stabby lambda syntax that we have in Ruby 1.9 as well. So this creates an anonymous function. You'll notice that it's independent, it's indentation-based as well. So there's not a lot of closing brackets or any. There's implicit returns and a lot of the good stuff that we like in Ruby as well. And this works exactly the same way. We don't need to configure anything really. We just have to make sure that we have CoffeeScript installed. And Evergreen will just pick these up, put them here as well, and we can click them to run. They run in exactly the same way that the JavaScript would do. So I think that's a really, really nice way to write unit tests in CoffeeScript. I forgot to mention, all CoffeeScript does basically is that it compiles down to JavaScript in the end. So you can very well write your JavaScript application code in CoffeeScript as well. But then you have to make sure that that can run, that that compiles and is uploaded into your, including your application in a good way. And of course Evergreen only, Evergreen is only concerned with your testing code, so it doesn't help you with that. But that's all I wanted to show you with the demo. Let's get back. So with that, I hope you have seen that front-end testing does not have to be hard. It's actually quite easy to get started. And we've provided some resources here. As I said, the code is available on GitHub, the demo code that I showed. So feel free to check that out and have a look, play with it and see what you think of it. And if you want to contribute something back, maybe more whiskeys. Feel free to do that as well. And check out Capybara and Evergreen on Jonas' GitHub account and of course Jasmine on Pivotal's account. And if you have any questions, feel free to send us a message on Twitter or come up to us here. And we'd love to talk to you about this and help you get started. We'll take a couple of questions and after that, if there's time, we can show you some real, what this looks like in a real application, and not in this maybe contrived example. I'll open up for a couple of questions first. It's in Selenium, at least. Oh, sorry. The question was how close the track to the I in Capybara is to the real world. And in Selenium, it's very close, actually. It pretty much simulates taking the element and dragging it on top. Unfortunately, I think that Selenium doesn't support all sort of mouse-native events in the operating system. So it's not 100% accurate, I think. There might be some very slight deviations. But I'm actually not entirely sure as to what those are. Yes? When do you use Capybara when you use Evergreen? The question was when you use Capybara and when you use Evergreen. And that's a really good question. That's actually really, really hard to sort of find a good balance between integration tests. I personally believe that integration tests are good for testing sort of the workflow through the application, but you don't want to use them to test every single possible case because it's just too slow and it's just, yeah, too difficult to maintain. So I'd use unit tests for driving the code in the same way you'd use Aspect and Ruby to sort of really test the internal API of the code in isolation or I'd use integration tests to test the test. So the question was if you can do integration level tests with Evergreen. Yes and no, it depends on how JavaScript heavy your application is. But what you can't do, for example, is AJAX requests because you don't have Rails application running. So when you're running Evergreen mounted in the Rails application, like on Slash Evergreen, obviously you'd have the Rails application running. But if you're running it from the command line with the array class, then it doesn't actually even use the Rails application. So you can't rely on Rails application being this. You can't test AJAX, for example. So there's limits to what you can do in JavaScript unit tests. And there's limits in general, like even testing drag and drop, for example, in JavaScript unit tests can be a little tricky. You actually do that as well in the application, you might show you that. Did you have the question? You actually answered it. All right, okay. Yes, so does Evergreen tap into the Rails autotests? The Rails what? Autotests, no. So you can run autotests and also the Evergreen? So the question is if Evergreen taps into Rails autotests then I think the answer is no. I at least haven't seen anyone do that. It might be possible. In the back. Can you speak up? Can you repeat that? All right. So the question was if this correctly, how are you keep the HTML templates up to date in your unit tests? And when I first started out with unit testing in JavaScript I actually wrote this really elaborate script for sort of taking a Rails template and dumping it into this JavaScript to testing template and it was pretty weird. But I've come to realize that you need to at least for me I find it works best to treat the templates for the unit tests sort of as fixtures. So you actually need to update them manually. And you need to let your integration tests make sure that the JavaScript still integrates with your application in the right way. That's what's worked for me. Do we have any further questions? Yes. Can you tell the audience why there is no letter E in whiskey? I can answer this. All right, so Yarko wanted to know why there's no letter E in whiskey and that's because this is Scotch whiskey that we're talking about which usually doesn't have an E in the name. Now American whiskey and Irish whiskey on the other hand usually spell it with an E. And if you want more information on that you can talk to Paul of course who's a whiskey fan as well and lives in Scotland. So that's your question? All right. So we have just a couple of minutes left. So if you want we can just give you a quick look at one of the applications we've been working on lately. That's a very very JavaScript heavy application. It's basically all most of it is JavaScript and then there's a simple rails back in. And this is where we really got into doing this a lot. The question was if we could show just Selenium running behind the terminal here. So if we run Rick, you come or no with the background music nicely provided so the first part is running with drag test which we obviously can't see. There we go. So the actual I meant on the big demo you were going to show it now. Oh well, yeah so let's switch over to that. How much time would do it? I guess we could just get the get the test started. I think you had the last session of the day so I think as long as anybody wants to stick around you have time. Yeah unless Hilton boots everybody. If we can take to like 8 in the evening or something. Take that long to run. On this computer this is a MacBook Air so it's not the most powerful computer. This entire suite takes about 10 minutes to run. So if you want to stick around and watch all these green dots fill up the screen it's very cool. With the MacBook Air it might be faster if it's iobab. It's not. iobab. Alright so I'm going to run one specific feature here. What is the name of it? Drag allocations. Let's get rid of the other stuff here. What is this? That'll get it running first. Just imagine some nice elevator music going on in the background here. Yeah this is all Selenium. Is this a Selenium web driver? Yes. Now you can use Selenium too with Chrome or with IE as well. You have to configure that. It uses Firefox by default. Many people seem to want to use Chrome but in my experience Chrome is much much slower via Selenium. So actually Chrome starts slightly faster and you usually end up with much slower tests if you run Chrome. So I actually recommend using Firefox because it works much better. Also Chrome doesn't support drag and roll. So obviously this is why one continues integration probably because you want to be able to make sure that your entire suite is run but if you have very heavy front-end integration testing then it's going to take a while and so the best thing that we usually do is that we run our specific feature that we've been working on and we rely on the unit tests for driving the development and then we let continue integration and take care of the rest. Alright. One more question. Yes. That's the one that sort of kind of cruise continuous integration tool for any JavaScript that's a whole bunch of different browsers. Alright. Yeah. So JS test drivers. I haven't used it myself and I remember sort of looking at it and thinking that well this is way too complicated. That was sort of my reaction to it. We're not really doing anything that's particularly new with Evergreen or we're just packaging it to make it simpler to actually get people to do it and to get ourselves to do it. So yeah. The dirty secret is that we're really terrible at actually setting up CI. We're waiting for Dr. Nick to solve that. So I have no answer to that. Actually can we just show the unit test from here? I think that's interesting. So let me just show you one of the one of the spec files in this application sort of a real world JavaScript unit test file written in CoffeeScript. As you can see it's pretty big. This application has a lot of JavaScript unit tests. I think that's it. I've been following Capybara on and off as client needs are rising. Some months ago there was a path to merge Capybara and WebRound. Is that still going on? Or what's the story there? I don't know actually. Really? Okay. Apparently Brian was here like Brian Hancock. And I haven't been able to find them. So I haven't really talked to them. And I haven't talked to them any. So I don't really know what's happening. From my perspective I sort of don't feel the need anymore. So if someone wants to do the work to make that happen then that might be interesting. But I don't really know what that merge would entail either. So what do we need to merge these libraries? I don't know. Okay. Do you have a question or do you just want to tell us that we're out of time? No, I have a question. Can you guys just provide a little bit of insight around the maintenance of a test suite like this? Just your personal experience? Maybe even on this project? So I think the biggest problem that we're facing now is if we can talk about the maintenance of a test suite in an application like this. So the biggest challenge right now is the execution speed of the test suite. Because it's just getting ridiculously slow. We have some ideas on what to do and how to fix that. We haven't really gone around it. So that's really our biggest problem in this application right now in JavaScript tests. Other than that I don't really think we've had a problem. Cucumber really allows you to sort of nicely organize your integration tests. And with Evergreen it's sort of writing your JavaScript test. It's basically like writing your Ruby test. I haven't really had a problem with maintenance ability in that. Right? I think we're done. Thank you so much.