 Welcome to Agile Roots 2010, sponsored by version one, Rally Software, Virio, Amirisys, Agile Alliance, and Xmission Internet. Before we begin, who here is a developer? Okay, who's here? Who's a tester here? Or fills in as a tester role? Um, product managers? Okay, UI guys? None. Not surprised. Um, more or less, you know, this makes up the makeup of a team. You have stakeholders, you have designed people, you have testers, and there's other roles, but more or less this is kind of the makeup of a team. And, uh, cucumber, well, acceptance testing I think in general, offers, um, offers a way to address challenges based by teams, um, challenges that are joint and also specific to a specific role. The versus requirements. Now, Judd Patton, um, brought up one of these, brought this recording as keynote, and that was at the chaos report, right? Um, this is an older report, but fifty, fifty-six percent of all bugs are introduced in requirements. And the flip side to that is also that 45% of functionality is never used. And so all that effort you get put into the software is, it's never used. It's like the weights that you buy and put in your home and you never really use it, right? It's, um, and when you do use it, you only use about 20% of it. Now, I'd only admit that, um, you know, Jeff kind of alluded to this in his talk, that this report is fine, it might be somewhat dubious, but I think it, um, the reason I bring it up is I think everyone who's been involved in software can relate to this. I think everyone who's been on a team has seen feature devotion within their projects. Um, the idea that we get so wrapped up in creating feature after feature would forget the actual outcome that we actually want and that's to make our users happy. So, in Agile, we're very familiar with this card, it's a user story, and I won't go into the details of it because this is an Agile Roots conference and I'm guessing everyone knows how you use these. What I'm going to say is how Cucumber picks up on this. You have a story like this and to make this into a Cucumber feature, you take that card and you drop it into a plain text file and you give it a name and that's the beginning of your Cucumber feature. Now, um, Cucumber won't actually execute any of this part. This is purely for documentation reasons. Um, what Cucumber will automate is the acceptance criteria that follows this and the acceptance criteria I view as essentially the conversation that comes from the user story. Um, so you have this user story, you have a conversation about it and what you do with Cucumber then is you record that conversation and you record that in a language called Gherkin. Gherkin is a type of Cucumber and it's a natural somewhat artificially structured language to record requirements. It looks something like this. You have a scenario, um, you have a given some context, when some action, then some expected outcome. And the nice thing about these scenarios, I think, is that they're just fairly sufficient for what you need to describe the needed functionality. Um, in addition to these, you can also have injunctions and disjunctions and they all do the same thing under the hood. Um, what's really important is the words that follow those keywords. Um, so if we take a scenario from our previous user story about, you know, as an inpatient buyer searching for stuff, we might get a scenario like this. And the thing I love about scenarios is you don't have to explain to someone what a scenario is, you just say, give me a scenario and they'll give you more or less something along these lines. Um, and so you could, you start with a scenario like this, you move on to another one where the context is different, because the context is different, you have a different outcome. So this is kind of 101, uh, gherkin language. Now there's some advanced stuff as well. Um, one thing that I think, the key coverage is really well is it encourages this workflow mentality. Um, this is how designers think, this is how most people think when they think about software. Um, and it, I mean James, you can make me dispute this, but for my understanding was it wasn't really geared towards that, it was geared toward more towards the, the domain model. Does that mean more or less the case? Versus, versus workflow? Workflow. It had support for both, but it was better at doing domain model. Okay. And one of the things that did really well though is, is tables. And so we've taken that idea and also rolled it into a cucumber. And so there's a number of different kinds of tables. The first kind is a step table where you essentially only be explicit upon the data that a step gets, for example. So here we're providing, instead of just saying we have moved it by Steven Spielberg, we're explicitly listing every data that's in the system at this time. And you can use these tables in a number of ways. Um, with this example, I'm saying I should see this table on the Ui, on the Ui. And you know, to prevent the curriculum that you can actually constrain this, so it only checks those cells and doesn't, you know, blow up if more cells are actually added. Um, the other kind of, um, table addresses the issue where you have a, you have a certain scenario and you have, um, essentially variables that you want to change. You know, what happens if you type in Steve? What if you type in Bird? What will happen? To address that, we have scenario outlines where you, um, replace those, uh, those strings with these variables. And so what it does is it loops over each one of these rows in the table and executes that scenario. And from the programmer standpoint, you don't have to do anything different. Um, as long as you write these original steps, then it works under the hood. Um, another notion is the background. Oftentimes you'll have scenarios that have the same context. And to get around that duplication, you can promote that up into a background clause. In general, you want to keep these to be context and not any action-specific steps. Um, the one final thing, kind of a side note is you can also do multi-line steps. So in the case of, like an email, for example, you want to make sure it's being personalized exactly. You can easily do that, um, with, um, multi-line steps. And we're using, like, the, the Python lock stream type notation there. Um, and also the side note, um, since it is supposed to be for, um, customers, these features can be written in over, like, 30 different languages. Um, so there's no, really, there's no filing there as far as communication. So congratulations, you're all now Perkin certified. Um, and I'll be selling these for $500 after the call. Um, that would probably sign a T for you. Um, so what now? I mean, what does that, doesn't really give you much, right? So I'd like to explain now the process that I've used, um, with my, with teams of using Kube Company. Now, this isn't a hypothesizing, copium world. This is actually what I've done. Um, let's start out with a user story. We're using, we have a little tracker, um, to store our, our backlog. And so we pull a, a story off the stack and we take it to the whiteboard. And that usually involves the designer and the, um, myself and the product owner. And we talk about what those requirements are. I then typically, um, transcribe that conversation to Perkin. And then I go back. And this is, going back is probably the most important part, because I say, this is why I understood what was the requirements. Am I right? And more often than not, I'm, I'm wrong. And more often than not, I've included functionality that doesn't need to exist. And so this back and forth is really example. And that's also where the scenario outlines are really helpful. Because you can just go line by line in the, by example, in each row and get a lot of mileage. After that, we begin the outside in development process that we follow with Cucumber. And so now I'm going to stop there and go into how do you really automate this Perkin? You have these plain text files, but how do you make them run? And it, um, first off, every project with Cucumber has a features directory. In that features directory are the plain text feature files also with the support directory. In the support directory, you have an environment file, which is akin to a test helper. It boosts your actual environment. There's everything you need to do to get that, your application up and running to be tested. Then you have the step definitions. This is where the rubber meets the road. And you have to actually make those plain text lines executable. And they look, so you have a step, a step definition would look like this. You'll see here we're using regular expressions. So we're essentially training regular expressions into method and vocation. And for the variables, you define them with regular expression capture groups. And those are then yielded to your method to be executable. And now inside the step definition, you can use whatever you want, whether it's web graph, you're in the Ruby land or web drive or water, HTML unit, J unit, anything. So Cucumber is not specific to any language or framework. Because as you see, this is obviously in Ruby, but we can address, we can actually test a lot more applications. So let me just curious, who here is a Java developer? Okay, we have a few. Okay, Cshar. Okay, wow. I signed in most of my talk to Java. I probably should have done this instead. So there's a number of ways you can use Cucumber to test applications. Now if you're using something like Selenium or water, you can obviously apply that to any application. Likewise, there's libraries for testing swing applications in Ruby. So that's one way. The other way is to use what's called Cucumber's Wire Pro call. And we actually have that for .NET and it allows you to write your step definitions in .NET. And Cshar, sorry, I hope it doesn't really offend anyone. I did use Cshar back in 1.0. Apparently we had lambdas now, I found out. So this is how you do it. Now this uses the Wire Pro call, which means it's sending over some JSON responses. I've never used it. My feeling is it might be a little, not as pleasant to work with as the native Ruby version when you're using Ruby code. So thinking around that, there's other tools. I don't know if it works with Iron Ruby. It does. It does, but I haven't heard great performance reports on that project. That's certainly inherent to it. Yeah. So gyrkin, the language itself, it's not tied to Cucumber. We have these parts that are written in a raggle. We define this as a finite state machine. It compiles down to a number of languages. And because of that, certain frameworks, specifically spec flow, have picked up on that. And this is a BDD framework written in C-sharp that uses the gyrkin language. So if you know gyrkin, you can just pick up this tool and it will integrate it nicely in Visual Studio. And I imagine it's probably a very nice experience as far as debugging and that sort of stuff goes. Kind of the third rally is through something like Iron Ruby, but JRuby. JRuby is actually very mature. It's kind of being the fastest Ruby implementation currently, if you exclude the one in mind versions. So it's very fast, and because of that you can use it to test any post-language on the JVM. I'm currently doing a closure project, and it's actually working quite well with it. And to give you an idea, this is how you translate those Ruby steps into the various languages. So it's more or less the same thing. It's your keywords and regular expressions. The job you have to use is allocations. What if the job is over rows? It's not that bad. There's only one more line, right? Double the lines. On a serious note, if you are doing, unless you're really investing in Java, using something like Ruby or Ruby, the integration is so tightly within Java there's not a whole lot of effort in having your acceptance testing compiled. So you might be thinking, this is a lot more work for me. I have to now write these step definitions instead of just getting down to work. That's true, but there is some overhead. I won't lie, but I think that overhead is paid dividends. And going back to the talk about requirements, I don't often take a story like this and show my business, state analyst or whatever, and say, does this look okay? And they'll say, no, you don't need to do this, you don't need to do this. And so off the bat, you save a lot of time not worrying about edge cases or functionality that doesn't really need to exist. And so you're only focused on writing the software that matters. Additionally, you also know where to begin and where to end. When that queue comes in turns green, you're done. Likewise, when it turns red, you broke something and you have to fix it. So it provides a safety net. And kind of going back to this whole theme of this conference of building great software, it turns out that thinking about software from a user's point of view turns out to be a good thing. And as developers, at least speaking for myself, it's really easy for me to think they get lost in the code and the inner workings and forget about the user experience. But when I'm focused on writing requirements in English at the higher level, and I'm really actually going to interact with it, I find that I produce much better software. And so to produce a software, we use something called the outside-in process. Now, traditionally, we'll often start with the database schema initially. Work up to the models, go to controllers, view what you're doing like a view of your web app, and eventually you'll do the UX. Well, outside-in puts this around and goes from the very outside and works inward. And so the feel of how this works is you start out, actually, I'll back up a bit. I like to describe this outside-in process as kind of bite gear. You have cube gears, the outer gear being cube cumbers. Because you start out with this scenario, and when you first run that scenario, what will happen is the steps will be pending. So cube cumbers will tell you you need to write step definitions. And when you do that, cube cumbers will go red and say, you're failing, you need to go write the code to make this pass. But instead of doing that, instead of just making the code, you actually go down in gear and do your traditional PDD cycle with your X unit. And the reason why you do that is, if you're operating on doing end-to-end tests with cube cumbers, you're going to get zero design benefit. In fact, you'll actually carry design if all you do is end-to-end testing. So you really need to fine-tune the innards with traditional PDD, regular factor. But once you're done with that and you're green on that level, you can actually take it up and see where you're at. And typically, you're at another pending step. And so then you repeat that process until you're green all around. And you're a factor of your repeat. So that's the kind of typical process. So I have an example of this. And these slides were taken from Azak Helisoy's slides, who was the creator of P-Cumber. And I saw these slides from him because he had some job examples. Because for some reason, I thought that everyone in the room was going to be a job programmer. I could skip them if you'd like. But it's still probably going to give you an idea of how it would work. So you start with this feature. This is a teach feature for the mastermind game. And I also should point out, when you're using Java or any compiled language, the step definitions can't live in the features directory. They have to live in the test directory. Now, typically, when you're installing Qtumber, you would just say Gemisoft Qtumber. But in Java land, you'll probably want to use Maven. Or if you're using Ant, you can do that as well. But Maven, I use Maven, and it's pretty easy. You use some XML pushups, and you're there. It uses Tico container for the dependency injection. You can also use Jiz or Spring if you're already using those. And what happens is, when you first run, it poked into the integration test task, when you run it the first time, you set the property to install gemstru, that will automatically install JRuby, Qtumber, and all the dependencies you need to run. And it will run it, and you'll see something like this. A couple of things to point out is, on the far right, you'll see that we have a stack trace. We know exactly where these steps are lining up to for this feature. And so what it's saying is, you need to go write these features. It actually does more than that. It doesn't say you need to write them. It actually writes them for you. It doesn't write everything, but actually it gives you some pending step definitions. And so if you're not too friendly with regular expressions, you'll be okay to get started because it fills in the gaps. So you copy and paste these into your step definitions, your step file, and when you run it, you're not told that the steps are pending. You have to implement the step. So you go down and, well, if you look at it, if you look here, there's a location. So you go to that location and you write the step. Again, this is the regular expression being yielded to this function, the secret code is function. So write it. We say our intention is we're going to create this new game object. So, of course, when you run that, it will have a compilation error. So you create the game object, and at that point that step is green, and you move on to the yellow step. And you keep doing this process until you're creating it all around for that particular scenario. At that point, you add more scenarios which will then break, but then you keep going this process until you have this entire feature delivered. Now, since you're all very concerned about it, you should see the problem of these scenarios is that they're doing the exact same thing. The only thing different is the names of the variables. So you can extract that out into something like this. So if you have a little more tricks about cucumber, that is the tag. I think tags are really one of the best features of cucumber, and I think every testing program could have one. Here's an example of how you might use them. We've tagged this feature as using Lucene. We're saying that we need this dependency to be there for this feature. Additionally, we're tagging them on a workflow basis. This scenario is a way to work in progress. This other one is proposed. It's pending UI or whatever. And we can also tag long-running ones as nightly ones because you're only going to run your acceptance tests all the time. If certain acceptance tests will give you enough confidence to deploy, then certain ones that are long-running, you can kind of delay those executions until night time. And so you can tag things like that in any arbitrary way that you see fit. And then to use it, you can say, don't run everything but the ones tagged, or you could say run everything with the ones tags. And one thing about the whip is cucumber kind of promotes a compound workflow mentality. And so you can limit the number of scenarios in progress or in flow. And you can also say, if this does pass, then blow up because this is a work in progress and I'm expecting failure and not success. And so that whip tag is actually helpful on CI builds so you can prevent false positives. Some other things you should do. Where about hooks? You can hook into the platform if you come before a scenario, after a scenario. The world is really to provide helper methods. Now the hooks, since the customers can't see them, they're really best to put in something like a database like if you're truncating a database, that's what you want to put in these hooks. Something really low level, not something that provides context or a feature. And you can also do tag hooks. So if you're using Selenium or something and you only want to use Selenium on a few features, you can tag it to Selenium and you could then use a hook to switch out the driver for whatever you're using to drive the browser. So one thing I want to touch on really quickly is the notion that acceptance tests have to be end-to-end tests. Now the way I've been talking, I've certainly given that impression and the way cucumbers mostly use is with them being end-to-end. That being said, I don't do that all the time. And there's, I like to get into this with a scale. On one side you have a slow to fast and on the other side you have nicely integrated. Now when you're writing these tests, and actually when you're even writing step definitions, you can pick what level you want to operate on. For example, when you're filling in the context steps, you can just stuff things into the database. Those are going to be under code level. You can go through UI data in. Likewise, there's some other intermediate levels. Before you go full-blown and do the water or whatever, you can use stuff like headless browsers like HTML unit or VA-driven tools that will actually manipulate the DOM that aren't quite as heavy. And so what I technically do is I'll write one scenario that's happy path that goes through the entire stack but then I'll have a whole series of edge cases that will just hit my domain models. Because I know that it's all wired up and hooked up correctly, there's no reason to incur additional cost of having every single edge case going through the whole stack. So that brings us back to testers. Because developers, at least in my mind, should be in the testing process intimately. So what do the testers do at this point? Well, they do what is really important. Well, I think it's ignoring that exploratory testing. And oftentimes, so when we're on path projects, we would always hand off to people to do exploratory testing and they would always find things I missed. And that gets just fed back into the cycle add a new scenario and then I would address that and then we keep doing that until we're done and we have the feature that we needed and hopefully, because we've thought about the user experience from the beginning, we'll get a happy afternoon step in addition to just output. The other event that I see of using a tool like Cucumber is you avoid this problem of dead documentation. Dead documentation I found is not only useful, but it felt to be harmful. And so instead of dead documentation, you have living documentation. You have executable documentation that you know describes how the system is actually working. And really the notion of it being executable ensures that as the system involves, this documentation will evolve as well. So to me that's really a key benefit of Cucumber. A few subtleties I'd like to point out. When you're writing scenarios, it's very easy to, it's more of an art than you would think. For example, we have this scenario that I listed above. This work, this is very high level, it's very declarative. I have these movies, when I search for this, I should see these movies. Now, you could rewrite this same scenario in a very imperative style. I mean, the intention of how the implementation is supposed to be is in line with that scenario. And there are certain trade-offs through these styles. I've worked with certain designers that really like this. They like being able to say, look here, I see this, because otherwise they don't really know how the UI is going to count out. But there's some trade-offs because what you've done is you've shifted all the maintenance to the plain text stories. Whereas before the declarative approach, you will be writing more declarative step definitions. But when you're writing that code, you'll be on the code level and you can have all these instructions to reduce the application. When you're on the plain text level, you're really setting yourself up for some pain in some cases. As an example of that, you take this login example. This login example of itself is fine because you're talking about logging in, and if you want to describe the process, that's fine. But if you add a new scenario, changing the process, and you simply copy and paste that, what you've done is you've really confused what this scenario is about. All of this above is incidental to what this main scenario of changing the password is about. And so what you can do with Cucumber is you can hide that noise and wrap those steps into another step. So the steps can actually call other steps within the step definitions. So here are a few resources. You can go, Coup's info is the main page. And you can also, on GitHub is where all the code is, especially the Coup for Nuke and Duke is up there. Anyways, that's it. Do you have any questions? I know I went through that really fast and after a following path presentation, it wasn't too hard to follow. I'm just a little bit not getting how you would take these Cucumber specs that you write and test like a Cocoa app. But at least you have like an iPhone app on there. That's really interesting to me. So like, I don't know, there's something I'm missing. I mean, how does it go all the way from the back to the front like that? So as I said, Cucumber is really a thin wrapper and it relies on third-party libraries to drive those things. There's actually a project called iCook, which will, what it does is I think it installs some sort of driver within your iPhone application and you send an HTTP request. Well, yeah, you don't have to worry about this. The library does that. But then, essentially, it will drive it. And you can, if you search for iCook, there's actually a screencast showing you. So these drivers tell you what's going on. Yeah. So it's i, c, u, k, e. I like Q. Yeah. And really, I mean, those things, I think those are nice, but to a certain extent they're somewhat of a novelty because you're, for the web, the web is slightly different because you're developing your whole UI. There's a lot more ways you can screw up unless you're using something like Wit, perhaps. But with the iPhone, code goes fairly set up, so it's harder to mess up, essentially. And so I think those, slot like iCook may be valuable for a couple scenarios, but I would go, I would do everything through iCook, for sure. Yeah. Mac Ruby runs it as well. Any other questions? Yeah. Has anybody made what seems to be the obvious correlation between link as far as the syntax is concerned? So, I never used the link. I know what it does, but I mean, that's really for data. I mean, you're talking about, I mean, the focus is completely different, but as far as the syntax is concerned, it seems obvious that it's very common. Yeah. Like I said, I don't know. Just having, because you said you can write these in C sharp or whatever, since it's obvious that it would be better to write it in more of a link style. Well, I mean, I don't know C sharp at all, or Mac community at all, so I've never thought about that. It's interesting. If you know who is running that project in C sharp. Yeah. So, well, if you go to specflow, think specflow.org. Yeah. I got a guy by the name of Richard Lawrence runs the Cooper Nuke stuff. But yeah, I don't know if anyone's really explored that at all. I know there is high end in specflow within Visual Studio, so maybe there's some sort of something happening there. Yeah. So, I have a question on when you should use it versus when you shouldn't. You talked a little bit about that. But, you know, a web application example will say, we have a bug, right? I click this form, and then I've seen cucumber specs where it says, you know, I fill out the form, I click go, and I should not see a 500 response in return. I mean, is that an appropriate use for this, or? Yeah. So, especially talking about rails, I mean, that's probably the best level to do it at, like the 500, to really make sure that we are seeing that. I mean, I would say that you have to balance the maintenance cost of these tests with the value you're going to get from them. There's certain times where, you know, adding a certain test on a new one just makes a lot more sense and it's a lot cheaper to do. So, it really is really valuable. It's really context specific. I just said, well, what you've done, what you should do is use cucumber 100% of time. That's what I do. You really need the lower level examples. I don't know if that really, that probably doesn't have to be it. It's one of those hard questions in software. I'm looking for you to say you should not test response codes in cucumber. Well, so I do test response codes, so I can't. Oh, of course. But also, I mean, you can do, like, I should see a successful response or something. But, so I'm actually, right now, I'm giving a, so I'm doing a project, I'm working on it now with a web service, and we decided to use cucumber to test that. I mean, it's in closure, but we're still using Ruby and cucumber to drive it. And we did certainly test response codes because as a consumer of that web service, you really care about whether or not that code is correct or not. And the other thing, I really like how the metaphor of starting with the story and going all the way down through your test driven development, coming all the way back up and kind of cycling up and down, I think that's really helped me understand, you know, where this fits. And I think it's terrific. Yeah, I've worked with some really smart, I'm going to call them hackers, that don't really subscribe to many processes, but when you send down to write one of these things, they really like the kind of the mood that puts them in, not the mood, but the mindset. Because when you sit down, have to write something in English and articulate something, it really changes how you think about things. And it kind of just, yeah. Can you talk again, I kind of missed out on why you're saying you don't use it for testing and what I like response codes. He was saying that. I was saying that. He was saying, go ahead. It's like, I don't know, you start testing for edge cases in it and it's slow and slow overhead. I think those are better tested somewhere else. Well, so we want to argument this. You typically want to keep, for normal scenarios, usually using the web case doesn't care they got 200 response. I mean, that's really incidental. And so you should be saying, you know, when I searched for this movie, I should be at 200 response and I should see this. By virtue that you're seeing the movies you're getting, kind of in furs you're getting at 200. And so, the idea is that you don't want to pollute a scenario with details that aren't really key on what it's trying to accomplish. Well, I guess it depends on what scenario you're actually testing. Exactly. And that's why the web service I'm doing is basically giving this JSON, I should see this JSON and I should get it, you know, 402 or whatever. Because that is something that is relevant. At that high level of detail. Because it is part of the API. Right. And there's actually been several people, like so there's a company, for example, called Chargeify, and they have all of their cucumber features are their documentation for their API. You go on and look at how they use their API and it says, given this, when I send this JSON, their XML, I should see this response. And because it's executable, you're guaranteed that that's how the system works and the burden of updating documentation is then, you know, removed or mitigated. Any other questions? No? Okay. Thank you. Thank you.