 So, as you're coming in and while people are getting settled, we want to ask a few questions about where people are at in terms of experience, just to help us know a little bit about pacing and tell you a little bit about the focus of this structure or the structure of this lab. So, we'd like to take advantage of the idea that this is supposed to be a hands-on opportunity as opposed to you're going to all sit there and watch things happen and not participate in that. So what we're planning to do is I'm going to give about a half hour, maybe less overview of just a little bit of B-hat because we're making an assumption that many or most of you do know a little bit about it. And then talking about the Drupal extension and some of the things that it does and a little bit about BDD philosophy, so we'll have that introduction. Then we're planning to take about half an hour at the most. And we've prepared thumb drives with a vagrant install that has all of the tools installed for you so that we're not trying to struggle with the conference wireless. So we're going to be doing some environment setup during that time, which is also a great opportunity if you're all set up and you're good to go to talk to the people around you about what they're doing, how they're using things, what they're interested in. And then Howard, so people are still coming and we'll introduce as soon as they're sort of settled, but then Howard's going to go through writing actual custom step definitions inside for a specific module as an example and he'll be doing hands-on things. Ideally we'll be able to do that and you may have an opportunity to even work on some of the things that are of interest to you if you're a module author and you're interested in the testing as well. And then in the last half hour, Sam and Graham, if he's here or when he gets here, we'll be talking about the future of B-HAT in Drupal 9 core because there's a lot of conversation about replacing simple tests. There's not a decision, but there's some momentum for that, which is not something that I'm particularly involved in. And then also talking about some of the conventions and what happens as we grow the idea of contributed modules having functional testing, functional B-HAT tests. So that's what we're looking at, a half hour at the most of presentation, a half hour of installation at least an hour of hands-on and then a half hour of more community style conversation about conventions. So what we were wondering is, and it's totally fine to be at anywhere on the spectrum, but how many people in here have not actually worked with feature files or B-HAT at all? Sweet. So this will be introduction for lots of folks, and that helps me know about the pacing. How many people are working with B-HAT on a fairly regular basis and feeling pretty comfortable with that? So those of you who raised your hands a minute ago, look at those people. Because this is a really big room, and there are a lot of folks here. I had envisioned like we'd be working kind of with 20 people or so. It's really great to have all of you here, but definitely if you know some things about B-HAT and can be a resource, that'll be really helpful for the hands-on portion. And then the specific focus for this presentation was actually using B-HAT with a contributed module. Who's here for that specific focus and is really wanting to know about that in particular, above all? Awesome. No, we're fine. We're flexible with however we want to take it. So I want to try to keep an eye on the people who are definitely interested in that, because we will be going pretty fast and pretty deep to get to the place where we can help people with actually the title focus of the lab. You are totally welcome. We're very excited. So don't sweat it. And how many people are here because they want to use B-HAT on their own project to test their own websites? So all of this will be applicable to that, absolutely. It's just that some of our example is going to be nudging you guys to share some of those step definitions as you create them back to the modules they're relevant to. But it's all the exact same stuff that you'd be doing in your own local one off for your site. And also, Show of Hands, Mac. Macintosh, OSX people. Yeah. Windows. Linux. We should have all of you covered, but we'll find out. We think so. I have the feeling that we have less Windows expertise among the three of us, but I did some practicing, so we'll hope on the installation part of it. So I'm going to go ahead and formally start now that we've done that bit of surveying. So my name's Melissa Anderson, Eliza411 online, and the three of us actually got started and got interested in B-HAT after the Git migration. So Sam was the technical lead. I was the PM. Howard was part of our team. And our interest in this automated testing came from months of actually having to click test the functionality for Git on Drupal.org. Sam and I really never, ever, ever want to do that again. The great secret of the migration was that after every build we locked everyone out, and she and I just spent now and a half on the phone. So like twice a day. Every time we integrated code. And so for us it's a very visceral desire to be able to automate things that should be automatable and to be able to do that in a way that when we actually open something up for our team or users even in that development area that they can be really confident about it. So I'm an independent consultant. I've been doing Drupal site building for years and I've been doing a lot more work with B-HAT since we started the project with Drupal.org and its functional testing. I'm Howard Tyson. I'm the VP of engineering for this company called ZipTech. And we've been using B-HAT on all of our bigger projects and sort of more web-appy things. Wherever we have custom sophisticated functionality where it's not just, I submit a note, it shows up in a view. Usually we don't test, personally we don't test that stuff really thoroughly, but where we have deep web apps we've been doing a lot of integration testing and it's been proving extremely valuable. It's time consuming it first, but you get faster and faster. And yeah, I'm Sam Boyer. I'm the administrator for D.O., Drupal A core person and contrived person. And in my working day, I am at NBC Universal. There's a large distro that is under active development and using a bunch of different, for a bunch of different NBC properties. And it's a very interesting case where the distro is being developed and there's a need for certain interaction experiences that are supposed to be guaranteed by the distro, which are the things that are being delivered to the brands that use it. And B-HAT is an excellent way to be able to capture that value at the center and provide them with tests that they can run as part of their own test suites and you can sort of back test on top of each other. It also aligns, as I think many will find if they've not already been using B-HAT, writing feature files will probably look an awful lot like what you already do in generating stories for agile. There's a very similar structure to them. It's just more rigorous requirements. And I think I'm supposed to remind both of you that since this session is being recorded when we are up here, we're probably supposed to be speaking into the microphones to be more helpful, but excellent. Sorry. So that's Sam. That's Howard. If you couldn't hear it on the recording, they're awesome. So I'm going to give you a little bit of foundation philosophy rationale to start with why it matters to us, why it matters to our clients and to our end users to do functional testing. So because Drupal has powerful value that's added by non-developers, that site building level of value, because it has fairly complex module interactivity where you can apply a security update or you can have a module that alters another module and you may not even have been aware of that, because we get frequent security updates that may contain functionality changes, you don't know if you can read the code great and lots of developers can if they have time. But that's something where you want to be secure and then suddenly find out actually that interface isn't even working the same way that it used to be. Because of things like that, we really need to know how users are expected to move through the application. And we need to know what the application is supposed to do. A little aside, as we started on the testing for Drupal.org, I promise you there was not a single human being who knew what Drupal.org was supposed to do. There were areas where it felt like no one knew what it was supposed to do. There's an application and the experience was, well, when I click it, it does this, so we're going to assume that's what it's supposed to do and we'll write a test for it. That way, we'll know that if it changes, it changed and we'll see if it broke or if maybe somebody changed something else. And so knowing how that application is supposed to respond helps your organization, when you have turnover, you get new people, and it helps become this thing that is a form of living documentation. So when your test fails and you update the test, even if it wasn't failed code and it was a change in process, you're rewarded for keeping your documentation up to date because you can automate those tests every time you put the security update out, rather than having this massive piece of documentation, if you even got that far sitting on a shelf that you knew you should update, but you never do because you're not going to reference it or it's already so far out of date. So that's about keeping our knowledge of how the application works in sync with the change orders that come through. And we want to be able to test that those user paths work, the things that users are supposed to be able to do with the app. Like I said, it's the reward. It's like, wow, so we don't have to click it. That's worth updating. And because our applications are generally dependent more and more now on third-party libraries, that is code we didn't write, code we don't write tests for, and integrated with third-party services where what's happening in our web interface doesn't belong to us. And because they're highly visible, like when I worked in education, I could get away with taking a site down for a day in the summer. It was fine. You could put some read-only things up. You could do all kinds of maintenance. But we don't have the luxury that we did in the earlier days of the web of having maintenance windows. And we don't always have the resources to have mirrors and other things. So this high visibility pressures us to be able to change quickly and to be constantly available. And because they're globally accessible, we don't get that window of, well, nobody's at work anyway. I can take care of this if we have folks who are coming from other time zones. And all of these things are encouraging us to develop more rapidly and to deploy more frequently. So we're highly visible, and that means that competitive services are highly visible as well. So there's a pace of being able to keep up and get functionality out there that's really important. So the goal of the testing portion of this, where most people enter, is to be able to describe our application behavior in a way that's going to let us automate the test. So this is actually the structure that you have for a typical B-hat file. You have a single feature at the top, and if you get into typing and you're feeling like lazy, this feature, stanza, is not actually executed. But you must have feature colon at the top of your file to parse, and it'll actually repeat this in a little bit. But that stanza is there as a part of the larger philosophy. So it's there to establish this pattern in order to do something valuable. As a user, I need to be able to do something. So in order to achieve a value, you have to do something. And then the scenarios that flush this out are actually the steps to do that. So each scenario may be something really straightforward, like here, that you want to see the forum posts on the front tab, and they have a pattern of given that the system is in a known place. So we have to tell the test, like, what's here? Once we do that, then we have a section of that where the user does something. And that is the user that we named in the top file, or in the top part of the given, excuse me. And then the last part is, well, when they do this thing, what's supposed to happen? And that's the basic balance. And there's some detail to how this comes together and ways of making things more simplified, or in some places actually a little bit more complex. But the basic idea is that we're gonna be able to describe it this way. And by describing it just like that, most of the time, without actually writing PHP, we should be able to execute that file itself and have it do something like this. So what we're doing is we wrote English language, and it's going through the process of actually clicking the browser on our behalf, doing the things that we described. So it posted a forum, and it's gonna sit here for a really long time because there's hardcore caching on the front page of Drupal.org. So we've created a forum post so that we know we have data that we need, and then it's gonna be checking to see that once we've cleared the cache, that forum post actually appears on the front page. So the cache got cleared with a menu callback that we made special for being able to do that, and we're looking for the same random title that was generated down there at the bottom. So that's what we're shooting for, and that is an example of instant automation. The behavior-driven development perspective, those are only a bonus. That is not actually the point. And I think it's really important to know that because it answers questions. Now, if that's all you're interested in is automated testing, that is okay. It's not like you have to feel bad because you're not doing some full process, but it explains some of the choices that you're being asked to make as you write those feature files and the scenarios. So they don't, they're not the primary purpose, and they do not replace unit testing. That's a question I get asked all the time, like, do we do this instead of unit testing? You absolutely do not. Now, you may, because you're not doing unit testing anyway. Like, that's reality, but that's not the point, okay? So what BDD is about is about discovering what we need to build. It starts when your business developers or salespeople or whatever you wanna call them if you have those, and your clients or your stakeholders are talking. It begins at that point in the process. You may notice that that feature statement looks an awful lot like an Agile user story. It actually pretty much is an Agile user story. And it came out of that tradition. So talk about that some more. The idea is to discover what we need to build to ensure that what we're building has value. And that when we test, we're focusing our testing on the right things. So the types of testing, there are lots of variations in how people define this. And I know that there's not agreement. So what I'm trying to do here is define the way that I'll be using those words when I talk about things. If you have a slightly different definition, that's okay. It's just that you'll know that we're not exactly on the same page. But there are types of testing, including unit testing, which tests a class or a method. And it's very, well, that's the point of the word unit. There's integration testing, and this is one that gets used differently, but that's where you have more than one class or method that are interacting. So you're looking at the integration of those units. You have functional testing, and that's what I use to describe what we're doing with Behat and Mink when we look at this. And so that's automated testing against acceptance criteria. So we've really changed audience at that point where we move from unit and integration testing to functional testing. And then there's GUI testing. This also has a couple different ways. This is not about pixel location, and some people do use it that way. But the bulk of the definitions that I'm running into as I'm working with people is just the idea that this testing is taking place in a graphical browser. So GUI testing means that if you're using CSS to hide something, to change the way the user experiences your site, that's being evaluated. And it means that if you have JavaScript functionality on the page, that's being evaluated. Whereas in automated testing against the acceptance criteria, that may or may not be the case. And finally, of course, you have click testing. Like the thing that most of us are pretty intimately familiar with at some point or another where you're walking through either a script or you just click it around. It looks like it pretty much works. We're all good with that. So for those types of testing at the core, the big important difference from my perspective is that the TDD tests, the test-driven development of unit testing and integration testing is absolutely intended to tell a developer that his or her code is working. As nothing to do with the business value, actually in the test itself, it doesn't have or capture anything about the user experience. It's that that method does what you think it does so that when you change something else, you're continuing to get back the things that you expect from the method. So BDD tests are about ensuring that your application is continuing to develop business value. And so that's a really important part as we move through what we test and why. So there are some links here and when the slides get posted, then you'll be able to look at some of the articles that I think discuss this fairly well. But I wanna talk a little bit about dealing with technical debt because that's where this kind of testing sometimes comes in. So the idea of technical debt is things got done in a hurry. They weren't necessarily best practice, right? Or they're part of the actual agile process of refactoring. Like you had to do it, you delivered, now you're gonna refactor. And if you don't get that refactoring done, you have this code sitting around. So this shape sometimes is called the ice cream cone, but it is an ice cream cone shape. When you have a lot of technical debt, that means that you may not have done that unit testing that you have old, crafty code and you're wanting to get reliability out of it. You may have procedural code that's really difficult to unit test because you can't get the isolated functionality that you need for unit testing. Typically in that case, you're gonna start with a bunch of manual testing. You're clicking around to make sure that stuff is there. You might have more GUI testing, which often introduces itself as, well, I don't have enough confidence to start refactoring the code to write unit tests because I'm afraid I'll break something and I don't know if I'll break it. So how about if I record some selenium tests? I'll just get the selenium browser plug-in and I'll click through it and then that'll record it and then I'll have a way of doing that. And those are fragile. If you've ever gone through that process, you can be incredibly frustrating and difficult to keep up to date. But you do that as you start to get a little more confidence and then at that beginning point, you obviously have fewer integration type tests and unit tests. What we're shooting for, actually, in application development is to have the pyramid a foundation of unit testing that makes your code solid. So you have good, well-architected, discrete methods and you're testing it. You have some integration testing, but not as many as unit tests. So you're gonna move up that pyramid to functional testing, GUI, and manual. And they actually, in terms of the speed of executing the tests, are like they take a lot longer to run the tip of the pyramid. So you're getting a little bit of balance there. So that's where we're headed. And it can take a very long time for a particular code base to get there. So when I say that we're headed to that direction, it don't mean that you should beat yourself up because you don't have the perfect pyramid shape or anything like that. But I don't wanna give the idea that it's going to be easy to refactor your code if you say, well, we'll just ditch the unit testing because it's really not an option. So the roots of it actually come from the Ruby community and Cucumber and Capybara are the two tools that the Ruby community used. And a gentleman named Dan North who did a lot of training for test-driven development discovered that as he was introducing the concept of you add a test, you run it, and see that you run your test, the new one fails. So you haven't done it, you write some code, you run the test, it passes. You refactor code as you need and you go. That's the test-driven development cycle that developers were often confused. The purpose of doing the unit testing is simple design, facilitate short-release cycles and give higher confidence in releases. Like that's the whole point of doing TDD. But the questions that came up were like, I don't even know where to start. I have this huge code base, what do I test? Where do I start? What do I test? How much do I test in any particular amount? Like what am I doing? And what do I call them? How do I understand why it fails? So as he tried to answer those questions, he realized that there was a disconnect in developer understanding sometimes of what the business purpose was of the application and where the value in testing was for them as a developer. So there are some fundamental principles as a result of that and some assumptions that BDD makes. And I just think they're fun and fascinating more than anything and this one in particular. So the idea of using these features and scenarios is that the largest constraint, the greatest constraint in software development is our ignorance. Almost always going into a project, what we don't know and what we don't know, we don't know are the things that typically turn around and bite us. And if you ask someone, think if you could do the same project over again. With all of the same people who went on vacation and got sick, would you be able to do this project faster? And most people who look at most projects will look at it and say, well yeah, because now we know how to do this thing, we didn't know how to do it and we know the client actually wanted that. And so the goal of finding out what you're supposed to be building is underneath all of the philosophy and it will come up as you think about how you write the features and the scenarios. If you're using them in a purely automated testing sense, you may not get that value. And again, it's okay, but it's worth knowing that that's why you're being asked to do certain things, that that's the intention. So it's not just like annoying. I mean, it might still be annoying, but you understand why. So the next premise is that every feature on a website should have a measurable business value. And that seems sort of obvious, but I know from writing those Drupal.org tests that it's not always clear what it is. Like it may have business value, but it's hard to communicate or to understand just by looking at the application, who it's for or how you make changes to it. And the final is that everyone on the team should have a common language to describe the site. And that brings in, there's a business domain language often where there are specific words that they use to understand what they're doing. There are developer database domains and the idea is to be able to all talk about the same thing and be sure that you're talking about the same thing. And that's what that English language feature file facilitates when you're talking with folks who are stakeholders and you say, I'd like to talk to you about the test of the thing. They often will check out. But if you say, I wanna talk to you about the scenario where somebody enters their credit card and it's not valid, but we actually go ahead and finish the sale, they're more willing to talk to you about like, oh, that's a scenario I care about. That's a situation that I'm interested in. And so that's part of why that English language layer exists there or other languages. If your development team is an English speaking, I'm gonna say that, but that's an assumption because that's what we're working in. So it does have multilingual capacity. Anyway, that you're all talking about it in the same way. And the final thing that for me is important is the idea that having conversations is one of the mechanisms that we use to actually discover what's expected. So having conversations is actually more important than capturing the conversations, which is more important than automating them. You may come at it from a different way, but philosophically, behavior-driven development wants to increase the value of software and believes that you're gonna do that through conversation. So if you're circumventing the conversation of an agile team, the stakeholder and those people, then you're going to see that some of the things that you're encouraged to do as you're doing your test automation may not make sense. And you can choose to do that anyway, but if the emphasis is on value, then that's the emphasis that they philosophically take. So you can use the tools without agreeing, but I think it's important to have that foundation. And I split up the slides so that I wouldn't necessarily have to go through all of them in order, so I'm just gonna switch to a few of them. Are there any questions about that at this point before I move into the structure of the feature files just a little while? Okay. All right, so this is the template and you'll have examples of this on the virtual boxes that we're gonna get set up here in a little bit. The feature file template is this straightforward. It has feature colon, it has a title, and it's followed by a narrative that has the pattern here. So in order to achieve a benefit as a roller user, I need to be able to do something. That's the pattern. That's the idea of what you're supposed to put in there. And then the Gherkin scenario format and the case matters here. The given, I am on the home page. When I press login, then I should see the username field is required and I should see password field is required. So there's a very simple example. And the structure is you have a given, a when, and a then. And you could have ands and buts in there, so it's not that you have to have one line, but you conceptually, the idea is that you have one given, that is where you put the system into a known state. So you have to know what page you're on before you know what you see on a page. So you put the system where it's supposed to be. The when is the part where the user is interacting and actually changing the state. And the then is the outcome that you expect to see. So that pattern could be different. You can say and nothing technically will stop you from saying given it's like this, when I do this, then I see that. And when I do this, then I see that. You can do it, but it makes your tests in the long run harder to maintain and harder to read. There will be overwhelming cases where it's easier to do it that way and you may have a reason for it and nothing will stop you. But if you try to keep your B-hat tests with a given, a when, and a then, you get the same kind of isolation when you run them of this is testing exactly the one thing I know. Instead of when, then, when, then, because the first time one of these steps fails nothing else runs. So the system is no longer in a known state, it's done. So if you have a bunch of when, then, when, then, when, then and it fails the first time, then you're not going to see the rest of that test run. That makes sense. So you wanna keep that, just to hold that as a rule of thumb and have a good reason if you're gonna do something different than that. It is occasionally common that you will want to just check something out and say, if somebody's on the homepage they should see something and they didn't even really do anything. So that state might be missing from time to time and it feels really artificial to make something up and your tests will run that way. So normally a given and a then is how I phrase them and I know that some of the folks that do B-hat training from the B-hat community would say you have a when and a then. When I'm on the homepage I see the slideshow or given I'm on the homepage I see the slideshow. You'll have some examples like that when you're writing them. So that's a lot of the question as I work with people is like oh can I do this? Why should I do that? That sort of thing. And that's when you're working hands on and we're circulating through the room part of what we wanna be able to answer and it's hard to do that question for all of you at the same time. So we'll be coming around and answering those. So that you end up with a whole feature that has one feature, stands at the top, and multiple scenarios. So those scenarios are typically going to be in the style of you need them to run independently. One scenario should never depend on another scenario. And there are a lot of reasons for that. Fundamentally you can run scenarios out of order. You will probably want to run scenarios out of order and test just this one part that you were working on before you run a bigger set of tests. So thinking about them out of order is important and it keeps your tests maintainable. As soon as they become dependent they become very very difficult to maintain because one failure here and you're troubleshooting more than one test. Your scenario is always going to be from the perspective of the role identified in the feature so you don't typically switch between as an administrator I see this and as a user I see that. So if you're setting up your data you're gonna set up whatever you had to do as an admin. The state changes from the perspective of the person identified in the feature and again this is a thing about facilitating conversation but also about maintainability. As a rule of thumb you typically end up with somewhere maybe only one or two but kind of six to eight scenarios in a feature tells you that you're on track. When you start getting a lot more than that you're probably defining your feature too big. So it's a helpful rule of thumb about the size. One given set the scene, one when change the scene. One then observe the outcome with as many ands or buts as you need in each one of those. So an example here is once you've written the PHP the first time for a step then it can be reused. So anything you have there in quotes can be subbed in for what's on your site. So given I am on you have some quotation marks. Regular expressions are matching that up with PHP so that it gets substituted in and you can use that all over and there's no additional code required. And that's part of why we're talking about things the way we are with the Drupal extension is that we have patterns in Drupal that we all face and if every single one of you is having to write that custom code each time then that's a lot of effort that we're putting in to test like media module for example. Like it has some really weird things that you have to deal with when you automate testing for it. If I've done that I don't even like the idea that all of you also have to do that if you're using the module. So it is about that kind of sharing. So the reusability of steps is an important idea and they come from a lot of places. Behat and I had not prepared to talk about this in depth but there's definitely documentation out there. Behat comes with, it's what handles all of the framework for the testing as well as the parsing of the Gherkin which is the name of the syntax for that feature and scenario. Mink is an extension to that that actually allows you to interact with browsers. So what it does is it abstracts the driver so that you can use Selenium or Sahi, you can use Zombie or Phantom although no one I know has had really good luck with that so if people wanna talk to you about that at toward the end we could do that. Selenium is definitely my choice in terms and I don't know if you guys use anything differently but we tend to use Selenium. But we can switch that English language with just a tag and say at JavaScript and that uses Selenium and if we leave it off it uses a headless browser. There are wildly different APIs for Goot which is the built in text browser and Selenium or Sahi and so that abstraction layer allows us to just choose in favor of speed. So anytime you're not loading the browser your test is gonna be faster but sometimes you need to and you can change back and forth in a single test suite. So that's what Mink does. The Mink extension then adds on to that and gives you a bunch of prebuilt definitions so you can actually do navigation forms, you can see outcomes and it has some debugging steps. The jupe? Yeah. I was just gonna, is this on? Yeah. So just to jump in on the driver question. Zombie is pain. It's a Node.js implementation where they tried to recreate the entire DOM structure and all of those events and it has all kinds of bugs that you just, the things that should work in real life don't work there and vice versa. Phantom now has a web driver API so you can use, basically use the Selenium driver against Phantom and it's a little bit faster and runs WebKit. Selenium by default will fire up Firefox but you can actually set it up to use all kinds of browsers including Internet Explorer. The other thing just worth reiterating is so Behat is the part that maps the English language, Gherkin or the natural, the feature files and scenarios. It's not quite natural language. Doesn't have to be English but that's what maps that to code that can actually be executed as tests and executes the tests. It doesn't know anything about Mink which is an API for talking to different back ends including just to reiterate Gout is, when she says headless, that means it's basically a browser emulator. What it's basically doing is running curl and then regex is to parse the HTML. That's way faster than round tripping going back and forth to talk to a real browser over a REST API. That's just way slower than PHP code believe it or not in memory parsing HTML. So then the thing that ties the two together is the Mink extension which knows about Behat and Mink and brings a whole bunch of other sugar too. And when we first got into Behat and Mink, I failed to discover the Mink extension which wasn't super well documented at the time and we wrote a bunch of code that we didn't need to tie those together. So it's really important. The code that we're gonna give you guys in a minute has that wired up the right way but just understanding what those projects are and how they relate to each other is really, really useful and I thought it very repeating. Yeah and I would add that when I teach people how to set it up themselves and we'll show you but we're not going to have you actually do that installation because of conference wireless and system variation whatever so you'll have working examples. It all happens via Composer and if you start with the directions for the Drupal extension and the Composer file for the Drupal extension it will go backward for dependencies and it will get the Mink extension, Mink and Behat. If you start with Behat and you keep adding on layers it is a much, much more difficult installation process. So I do highly recommend starting with that and I'll show you a little bit about the documentation where you can see what that Composer file looks like. So there are other locations that you can get pre-built step definitions from one of which is the Drupal extension. So this is becoming Drupal aware of things that we do. We have regions, we have this idea of regions and certainly other things do as well but it allows you to configure the regions of your site pretty easily so that you have sidebar first, sidebar second, tie it with its CSS selector and it's good to go really quickly. It knows about node types, it knows about users and roles and taxonomy, both vocabularies and terms. So that we have that already available. You can write your own custom scenarios. So this is when I say where do they come from? Well, if you're the developer you may have implemented custom steps but if you're getting support from like your Scrum Master or Project Manager they're going to receive those pre-built steps from you. So it's another place that they come from as well as additional sub-contexts and I think Howard can talk about sub-contexts later. You don't typically need them until we get into things like very different API interfaces or if we're working with the contributed modules and then the thing that is not very much implemented but which we're really excited about getting support for is that any Drupal module could give you step definitions that help you test it and the Drupal extension allows you to set a couple of configuration settings and it will auto discover those and put them together on your behalf. So you say, go look in Sites All, you define the path to that and it will auto discover anything that anyone has given you so that it becomes something that's just there and available for you to use and that's what we'll be looking at shortly. A couple of strategies on the language side of it because that is part where I spend a lot of time. So when I talk about two ways you can enter, one, you're using it for discovery, you wanna have great conversations, you wanna reduce ignorance and find out what you're supposed to do, you're gonna focus on excellent conversation with your product owners that's gonna help you make decisions. You're gonna try to record them in natural feeling language and the pre-built steps don't always feel so natural. You're gonna worry about automating later in the process so you just wanna capture the ideas and you're gonna retain that natural feeling as long as you can to continue to talk about these things with your product owners. But if your focus is on testing, you're gonna go a different way most likely. You're gonna work with pre-existing steps whenever possible. You're gonna bring in non-developers to assist because you have people probably in your shopping to exactly what that's supposed to do and can totally learn how to describe it and be almost there for automation and you don't even necessarily have to take that part forward as a developer. Those can be super quickly useful. They're sometimes easier to understand failures because you can see from the language file exactly what needed to be clicked. So if you have QA people who are gonna look at a test failure who aren't necessarily a PHP savvy, they can easily see what was going on and what was expected and what didn't happen. But they're often really eye-rollingly horrible. I click and I see and I push and I press and I fill in and when you try to talk to a product owner about that they can get in the way of that conversation. And this is an example of what I mean. So here in the top, better conversations. I'm on the home page when I click pricing and sign up I should see these plans. Just it's a lot shorter, it's pretty clear, but not immediately automatable. There's not any built-in step that has any idea what you're talking about when it says yearly, bi-annual and month, like what is that? What is it looking for on the page? So you can talk about it but you don't see it as well. The immediately automatable reusable version of that is given them on the home page when I click and then you look at the words that you see on the page and I should see the heading and it's gonna be anything in H1 through 6, right, if it's in H1 through 6 tags. Then I should see and then it's just looking for words. So not always as easy to read and talk about and this is a fairly short example, they can get really long. And there are two choices that you have. So when you write a step that it doesn't recognize, BHAT's pretty helpful in that it gives you the regular expression that's going to make it automatable and then expects you to fill in the PHP code that you need in order to make it reusable. So if you look at that comment there, that's an annotation which is also common, all sorts of parts of symphony. So those are not just comments, that's actually what makes it work. So you can't just go in there and mess with it because it will not line up anymore. What you wrote in the feature file is being matched with the regex for the given and then you fill in the code and it automatically gives you the function name. The other thing is you see the at given and then there's the slash. So the regex starts matching at the beginning of line marker, that carrot. So it's not going to match, it's not going to ever enforce a step has to be used with given or when or then. You can use it wherever you need it, although you will notice that it sounds really weird when you use it in the wrong places, sometimes depending on the syntax, but there's no actual syntactical enforcement of what goes with which. So to list the available steps and when we get your images installed, this is something you'll get to do. Oh, also you can do, I think it's a dash dash snippet append when you're working in your main feature context file and actually append it to the end of the file so you don't have to cut and paste it, which is another nice thing to know. But anyway, if you want to know what's available to you that already works, you type binbehat-dl and it will just give you a list of all of those steps. The regex can be kind of helpful as far as knowing sort of what you can sub in, but you can also experiment with it. You can see all of those with comments and if you're writing your own steps, please, please, please use the comments for the people who are going to maybe use this who aren't developers and to remind yourself six months later that it either goes with a module or in a particular use case or anything you need to know, using the comments space for that is great because you know that it's what it's intended to work with and that's another thing that I found is that when you first write the test, your head is in one place. Six months from now when that test fails, your head is in a very different place and you want to take as good a care as you possibly can of your future self. So if you're looking for a step that you can't remember because you have like a hundred of them, if you're grep happy, you know, you can just do the DL and grep for it. If you're not comfortable with grep, you can do the bin B hat dash D and then either one word or in quotes, part of a step and it will actually locate the steps and narrow them down for you. So the kind of things that you'll be able to do when you start experimenting are things like navigation. So I'm on the homepage or I go somewhere, I reload, forward, backward, so they're all about getting somewhere and navigating. You can fill in forms. So these are the, I press and this is one thing that will catch people. I click a button, I press, or I click a link, I press a button, press. It's for buttons, click is for links. It's arbitrary, but it's helpful to know that. The outcomes, things you should see or not see. And some debugging steps. One of your best friends will probably show last response. So that's a way of having it actually open a static picture of the thing that happened right before the next step. So if it's failing, you can see what the test runner saw. And it needs to be configured in your YAML file. And so when Howard gets there, he's gonna walk you through a lot of the configuration stuff for that. So before we go over there and I'm gonna go a little bit faster with these, it's more conceptual, but there are many reasons why those prebuilt steps won't work for you. Almost always, it's actually that you when you're writing the test or making some kind of assumption or mistake, it's not that the test isn't working. It's that what you think it should do and what it's doing don't match up and their mistakes. Sometimes, and the next most common, is application markup. So it assumes, for example, on a form element that your label is properly attached to a field. And a lot of times in Drupal, that doesn't happen. And so it doesn't have what it's expecting in terms of standard markup. Sometimes, much more rarely, it will be the driver that has an issue, but the pages loading either too slowly or too quickly are a thing that will happen. The application might actually have something wrong with it, like there might be an actual regression. It happens when you're doing the maintenance, often it's one of these other things and environment issues. So you may have network connectivity issues that are getting in the way or other things like that. So to be clear and honest, the fourth most likely reason for your tests to fail is that you found a bug. It's just kind of the nature of having a computer click through and look for things. They're picky. One of the reasons I give you the list though is there can be a beginning tendency to be like, oh, it doesn't work, like it doesn't work. Not, I didn't tell it the right thing to do, but that there's something wrong with the framework. And it's like, no, usually it's like, that's capital and yours is lowercase. Sorry. So know that and be persistent and investigate these areas is kind of what that's about. So test writers fail because they cut and paste an example that has different assumptions than their website. It's like, well, I cut and pasted this from the example set. I'm like, yeah, but your system is set at your site and that example thought you were on Wikipedia. So those things, not there. Confusion between what the page looks like text only and in a browser. So if you're following along in your browser and telling the test what to do in running it headlessly, there's often a difference in like, plenty often a difference at that point. You may not notice that you have duplicate items on the page. So you're like, there totally is a search button. Why is it not clicking it? And it's like, it's clicking something else that's going to a different place, but you're not watching it because it's text only and it's gonna have a different experience. Case sensitivity typos, they happen. Me, my most common mistake, I describe all of the form and I forget to click the, or to press the save button and then it doesn't work. So don't forget that after you fill it out, you have to save it or do the next thing in the wizard. And then on a maintenance level, you choose unstable test elements. That is your testing content that disappeared and should have been allowed to disappear. So you depended on it instead of setting up your own data. And that's a huge like architectural issue that you may run into is how do I know? And that's what that given is for. You inject data so you know you have what you need to test that the application is putting it back out. The exception to that is if the value is all in your database, right? And you need the database to be there, you may be testing those elements on purpose and somebody comes along and changes them and you get into those communication issues where it's like, was it legit or not? And you may have element testing that isn't granular enough or that bad data setup or deletion. Like some tests may have deleted the thing and the next test thought would be there. So application markup, if an element is not unique or the labels are not plain text or they're not properly linked, those are all reasons that your test won't work. So for example, in this here, the mink is gonna locate a button by ID value or alt, but not by name. So the only thing different in these examples is the name and it's not something that we had a choice of using. So select, select, select means that we can't actually narrow down where we need it to be. The solution to that is to either fix your output so it's unique. If you have that power, it's a great idea. If you can't do that for any reason, then you can narrow it down. So in this case, we're gonna look for a CSS selector that is a list item and that has the plan name in it and then we're gonna look inside that. And so Howard's gonna talk lots about how you work with actually writing the custom steps, but the idea is use something else to narrow it down and check the thing that you're looking for. Also with Drupal, we'll run into things where it's the thing that you're trying to match that's visible to the user, isn't plain text. Like we have all of this HTML wrapped in there and so it doesn't want to match on the basis of that. So another thing is inappropriately linked labels here so there's no, it wants it to be linked, but it isn't, there's no label for relationship between those two fields and what it's really expecting is that the label has the four and then you have the ID tag and that's how it knows those two things go together. Again, you can fix the label connection or you can use a different selector. Other reasons that they fail, some methods just aren't supported. The driver's too fast. There's a bug in the driver. The application actually has a regression or the configuration has changed or the environment varied things that changed network connectivity. You're missing a dependent process which happens to us all the time when like the Git Beanstalk isn't running so the stuff doesn't go through. It's not about the application, it's about the connecting things or you're missing a library. So we have to, in order to do Git pushes, we use the expect library to actually enter passwords on the command line and push that stuff through for Drupal.org if you don't have it installed on your test runner, your test fails. So these are all other reasons. And if you're using something external like that, that's where that comment really comes in handy to remind you that this is actually dependent on something and you can also put it in the error messages. Let's see. So the Drupal extension, and this is the last part of presentation here, is that before we get to installation, what the Drupal extension does for you. So it allows you to override default behavior. So for example, those snippets that you cut and paste normally have the symphony indentation standard of four spaces, it actually puts them out in two spaces so that it fits with Drupal coding standards and the way that things are in the file. And it also has some rewrites of the mink extension for better readability. It gives you a driver to connect with Drupal to set up data. There's a black box driver, which doesn't actually really do anything. It assumes that you don't have any privilege and that's what you get by default. You can use a drush to have drush create a user. Drush does not know how to create nodes, so it does not have much more functionality than that and it's not used very often. And you can use the Drupal API to set up the data. It's important to remember that the idea of setting up your data, you can do some magic there and put the system in a known state, but once you do that, you shouldn't be doing anything that the target user couldn't reasonably be expected to understand about your application, which means that if you say then I should see and you start typing CSS into your steps, you're getting out of that conversational mode. It'll technically work, but it's not a good idea in terms of those conversations. So when you focus on that, the API is for setting up data and you can do anything that the Drupal API can do. So Howard's gonna handle that when we get hands-on. It also adds support for regions, node types, users and roles, taxonomy, and the subcontext for contributed modules. So the drivers work with six, seven and eight. Oh, sorry about that. The black box is connecting with no privileges. That can be low. Your tests can run locally or remote. And that's a thing that tests can be anywhere. For black box testing, they don't have to be on the same server. You can run them across the network. For Drush, it can be locally remote. There's some Drush alias configuration that you need to do and whatnot. And then there you can connect with the Drupal API. The limitation of the API is that you actually have to be on the same machine in order to bootstrap Drupal and it's gonna bootstrap it and use the API. So about your data setup, every scenario should be able to run independent of other scenarios. That's why you wanna set it up. You'll often grab a fresh copy of the production database before you do a test run. So it's not an empty database. It's your configuration setup so that you can test that. You can't always assume that there's specific contents in the database. So you inject it yourself to be sure functionality is working. You wanna separate your architectural elements from your content elements. So if the content is supposed to change, don't write tests based on it cause they're gonna break next time you pull prod over. Like it's pretty straightforward but it can be edgy and challenging. So you insert the data to get reliable checks. There's documentation to go back to but basically you're gonna end up setting for Drush, a Drush alias in the normal Drush alias way and then in the behat.yaml file you'll declare Drush as the driver and you'll put the alias in there. We don't spend a lot of time with it. When you work with the API, then you're going to define where your Drupal root is and you're gonna have to have that API driver and that it's going to be Drupal instead of Drush. So that's the syntax. There's documentation online and included on the thumb drive so that you can look at that. Then you get steps that have to do with regions. So it's basically I should see something in the region. Those regions correspond to anything you want them to but are intended to correspond to the CSS selectors for your theme. So the things that you see on the demonstrate block regions. And you map those in the yaml file. So you can totally write this in code too but it's an easy way to reuse that and redefine it. It has the idea of nodes. So you can say given I'm viewing a node with the title and when you do that it will actually create a node with that title. So you don't have to write code to create the node. It will do that. It also has the idea of users and roles. So if you say given I'm logged in as a user with the administrator role it will create a random user with the administrator role. The role has to exist because there's no point in testing a role that you set up because the point of testing the role is to make sure that you got the permissions right. So it does just that user setup and it will create taxonomy vocabularies and or terms. So those are all things where you don't have to click through the interface. It will just insert them which means it's faster and it's reliable. And then it has a few more ideas about success messages. So if you really need error checking it will look for those. It will let you run cron, things like that. And even more there's just some of the other things you should not be able to see which isn't part of the Mink extension. So sometimes it's really good that something is not visible on the page. And it will also load module specific step definitions which is how it's really gonna focus on today. So you define it in the behat.yaml, dribble extension, searches recursively for anything called something.behat.ink. This is not yet widely adopted so you're not gonna receive a lot of those and you define it in the yaml file where you want it to do the searching. And that lets you inherit from the step definitions. So what we're gonna do right now is probably starting from the center of the room we're gonna get thumb drives and installation directions out so we can get your environment up and we can start being hands on and you can actually touch it and start applying some of this. Then Howard will be giving you more information and whatnot. So the thumb drives that we prepared if you'll use them and pass them on don't feel obligated to set up an environment if you think that you won't get to it or it's gonna be confusing for you it's totally fine to follow along. He's great. So the first thing we'll do is we'll split these in slightly weighted over here because there's multiple over here. If you know in advance that you're somebody next to you that you can already work with that you don't feel the need to install then go ahead and just ask them back but basically we'll take the thumb drives and the instruction sheets take one, pass it back. We're gonna run out in the back we only have 50 and there are like 65 people in here. So as soon as you finish with the thumb drive then keep on passing them back. And please do take the opportunity during the time to talk to each other and look at what you're doing and talk about things that you're curious about how you're gonna use it and the front sheet side with the image on it that actually has the vagrant directions. One more thing, just a second. If you don't have virtual box or you don't have vagrant they are on the key. So you do not need to try to download them with the conference network. So right, we should talk a little bit more about the stack. So the question was if you have the hat installed what's on the thumb drive? How many people here are familiar with vagrant and virtual box? Okay, half and half. So virtual box allows you to run a VM, virtual machine kind of like parallels that lets you run windows on your Mac. The idea is you can have a little application that has a full Linux install running in a window. Vagrant is a thing that manages those. So you can have one command that says vagrant up and it builds you an environment. So what we've done is we've pre-built a project. So we've set up an Ubuntu machine. You can see here, this is exactly what you'll get. It already has a copy of Drupal 7 and Drupal 8. It already has the hat installed. Yeah, we're not gonna work with the Drupal 8 copy. Like don't even go there. Not yet, we'll let Graham talk about that some. Pretend I didn't say there's a Drupal 8 on there. Drupal 8's a moving target and they broke our stuff. So, take it up with WebChick. So yeah, you can see here that we've got this thing all ready to run so that we can get you guys running the vagrant virtual box and you'll be able to run a test right off the bat and jump in and we'll start adding stuff. So those sheets have basic instructions. There's vagrant and virtual box installers on the thumb drives. If you don't have vagrant and virtual box, you'll want to install those and then you can install the virtual machine image. If you've already got the hat, you could sort of follow along and make up your own thing but you won't probably be able to follow along with the project we've got set up. Because one of the things I did was set up a git repository with checkpoints so that if you start falling behind or having something break, you could just check out the next checkpoint and get caught up. So that's the idea of how we can keep the next part running smoothly when we actually start writing code. Yeah, so visually if you're using a Mac and you're looking to install vagrant and virtual box, please use the vagrant 1.3.3 and there's only one virtual box image on here and there are regular DMG files which I see Mac people know what to do with. I know what to do with the Linux programs. So they're in the folders there and a super quick process. When you open the browser, if you are fast in getting it installed and you're interested, all the Drupal extension documentation is downloaded locally as well as the Mink API and documentation and the Behat documentation. You'll probably hit a few links in there that don't work because they're downloaded to your local file. You're certainly welcome if the internet is up which it is right now, right? To browse documentation online. But we wanted to be sure that if we ran into a situation like yesterday that we wouldn't be stuck where you didn't have access to the things that you needed. And then the Drupal 7 site is there also. We got to read a question over here too. So the question was, does he have any experience using Behat with Selenium Grid? I set it up as a proof of concept but haven't needed it for my real testing stuff. So in theory, it should work fairly easily because instead of pointing at a local instance of Selenium, you would just configure Selenium to point out at some remote grid server hub to which the client's connected. So when I set it up as a demo, it seemed to work pretty fluidly but I haven't used it in an actual continuous environment, like continuous integration environment. So hard. Give about 20 minutes to be doing this to make sure that we can pass things along 20 to 30. I'll be taking a look at it. If we get to that point where you maybe weren't able to install, you can perhaps follow along with someone else or just watch Howard and I'll wander around and see if there's anything that you need to come with or places that people get stuck in trying to help you get to the SAM. And we're gonna get to introduce yourself, right? Hey everyone, I'm Graham. If we get everybody online faster than 20 or 30 minutes, we could keep moving. So keep us informed on how you guys are making out and definitely throw up a hand if you have any questions. We've done this a lot of times. But hopefully it should be as simple as running those two installers and then running the commands on those sheets. Yeah. That's the key. Okay. Is that a word to you? Okay. If you set up the whole thing for the sake of it, is it working? Yeah. Is that actually what's up? I'm not sure. I don't know. Yeah, that's a good assumption too. I think you're better. I'm sorry about that. I'm sorry. We put 127, 123, that's a good start to say 127. I think at the time, once 123 had been totally tested, but once we tested it, we realized that it wasn't working. And it had another bug fixes in it? Yes. So, what? Yeah. When we imaged the USB keys, we hadn't tested 133 yet. We knew 127 worked. So we included 127. 133 totally works and fixes other bugs. So you'll be happiest if you're on a Mac with 133 for Vagrant. Just install the more recent one. I know the sheet says 127, but 133. Show of hands. How many people have a working Vagrant virtual box Linux machine? Awesome. Most of the audience. Drupal extension? Let me see if this. So someone asked a question that they asked me to share with you that it's a strategic question. And I normally cover it in training, but because this focus was different, I didn't have it in my slide deck. But one of the things about running these tests is that if you have lots of them, it's important to be able to run what you need when you need it. So if you are having trouble, like what they were describing is, they never get around to updating their tests because their tests take a while to run so they kind of stop running them. And you can do a lot of powerful and flexible things with a tagging system. So part of the reason to have your scenarios independent is that you can pick and choose them above that scenario block. You can put tags in your tests and only run those three arbitrarily and they can be in different files. However you want to use that. And I just wanted to put that out there since someone asked me about it. And since it's not part of what we were really intending to cover, it is in the BHAT documentation. And I think you'll talk about the API and the JavaScript tags. So there's some magic tags too. But it's just something to keep in mind that there are some very flexible ways about running them. And if you have more questions, feel free to ask me later on. You can put multiple tags on one scenario, but do note that they go on the same line separated by spaces. So if you had at JavaScript space at Smoke, then you would have each of those tags on it. But if you put them on separate lines, it will look like they don't work because they go across, not up and down. Dash tags tilde, at slow, and you'd strip out. And even fancier, you can do everything tagged this or this or this, but not that. Like there's a syntax, so it's very, very comprehensive and it really does help in the adoption when you can pinpoint what you need. So who doesn't yet have a working instance? Again, for people that are blocked on something that you think we could help with? He was having a problem with Ubuntu wouldn't allow him to install the Ubuntu installer because there was a different version. Yeah. Okay. So, a couple of the questions that I heard kind of in pattern form as I was walking around is how this stuff fits into your workflow. Melissa just kind of talked about some of the running the tests before you push stuff. That's a really important component. You should slap the developers that don't run the feature or scenario that's relevant to the thing they're working on for sure. And if you have smoke tests, you should make sure that people run those, especially before they push code. But then you probably at some point if you're gonna get serious about this, you want those tests to be running regularly and you wanna be getting feedback from them. So what we do and what we recommend is to hook those into a Jenkins job. If you're not familiar with Jenkins, it's a continuous integration server. So you set up this thing and it can watch your repository or there's some great GitHub integration. So when you push to GitHub, a web hook fires and tells Jenkins, hey, there's new code. Jenkins checks that out. You can have some build steps that pull your database from production, which is often a good idea with integration testing because in Drupal someone can change a view that changes what's available to the user or changes what information gets selected for a rule and the important bit here is configuration changes in the database can change critical functionality in your app. So a lot of the times in Drupal you don't want to do your integration tests in a clean room environment. You want all the filth of production. So you update your testing environment, your CI environment from production. You run your integration tests against that and this part's important. There are different formatters in Behat so that you can say, hey, Behat, when you run tests, I want you to format the output in XML and put it in this folder. Then you can tell Jenkins, hey, I've got JUnit formatted XML. You have a plugin that understands that. After the build run, go pull that information out and parse it and then Jenkins can give you graphs of how long your scenarios are taking over time so that you can see that your tests are getting slower and it can tell you three tests failed out of 50 in an email that gets sent to you, all that kind of cool stuff. You can also have it email the person that pushed the code that broke it too. Saying, hey, Dev, you broke the test, man, fix it. So hopefully you guys see something kind of like this. Hopefully your resolution's a little bit higher and it looks a little bit nicer. So this is our Ubuntu machine that has everything set up. So that includes Selenium. Selenium, again, we talked about it a little bit before. It's running kind of behind the scenes here. It's demonized in the background so that when we go to run tests, we can talk to Selenium, which is a Java server, which will start a Firefox process and run our tests in a real browser. I think you should be in the BeHat folder. And so you should see the anatomy of a BeHat project. So I'm just gonna mention a couple of the things here before we move into writing custom step definitions. You'll see BeHat.yaml. Melissa was showing this a little bit before, but you'll see there's a bunch of important stuff here. So there's the base URL. That is where it should go to look for the actual site. You'll see a bunch of configuration for the Drupal extension. If you leave off the region map, your region selectors, your region step definitions won't work. You have to tell it when I say I should see this in the left column, that means use the CSS selector. Otherwise, it doesn't know where your region really lives. And sub contexts, that says scan these folders for contrib defined step definitions, which is what we're gonna be looking at in a second. You'll also see this bin folder that has our executables in the project. Oh, come on, there you go. Since we've got BeHat and web unit, BeHat is the only one that we're concerned with here. BinBHat-h will show us all of our different available options. BinBHat-dl will show us all of the predefined steps. DI will show those predefined steps with comments. The comments aren't gonna be super clear to you, or sorry, the comments you'll be able to see. There's also in dark blue, in royal navy on top of black. You can see where that definition's actually defined. So again, there's a whole bunch of different step definitions that will pull from a whole bunch of different places, and that's telling you where you can go to look to find that item. And you can also do dash d and do a search. So you can say, I think I have a, I should see maybe, I think that's what it was called. And it'll search and find all of the things that say I should, because that's what I put, dash d in quotes, I should. And you'll see that there are a ton of definitions that come with the mink extension and drupal extension when you have both of those. You can write a lot of tests without writing any code. So it's a really good idea to review these because you have a definition for I'm on the homepage. I follow the name of a link, and the text of that link can be the textual content, the alt text, any number of different selectors. And mink does a bunch of magic to turn those into an xpath query that looks through your DOM element and extracts the thing that you're supposed to be clicking on. You follow links, you press buttons. That's confusing to people at first, but just know that. So we want to run our tests. Where do our tests live? By default, they live in the features folder. And inside of the features folder, we have home.feature and slides.feature. If I go into home.feature, we'll see the feature file with the kind of syntax that Melissa was going over before. Visit home page in order to access site features as any user. This part is expressing the business value. So this is the part that's supposed to be justifying to your boss why this is worth having on the website at all. Then you've got the part that you guys really care about the most, the actual scenarios. So you've got a scenario description, text connection. Given I am on the homepage, that's gonna load up the homepage. Then I should see welcome. And I should see user login in the left sidebar region. Note that this first scenario does not have any tags. So the way we have things configured, gout, which again is curl plus some regexes. It's basically a PHP in memory thing. It's kind of like Drupal's web test case. It's got a virtual browser. Way faster. We know we don't need any JavaScript or CSS interpretation for that. So we can just run it. But we tag the next one at JavaScript. That's a magic thing where we've told it if we need JavaScript, use this other plug-in as the engine that's gonna run a different driver that's the engine that's gonna run the test. So by default, that's Selenium. If you look through the documentation, you can find examples. You could swap it out again for zombie.js. Don't do that. Or Selenium or Phantom or a number of other providers. Sahi is one of the other really well-supported ones. So if we run bimby.hat without any arguments, we're gonna run our entire test suite. So you saw how fast that first one went by. You'll see this one's going a little slower, but you'll also see it fired up Firefox for us. Then it really quickly jumped to I visit the slides page. When I click Slides, then I should see a slide with the title inflatable. It doesn't know how to look for a slideshow. So we need to teach it. It gave us an example. Without the Drupal B-HAT extension, this would be formatted using symphonies coding standards, which obviously are wrong objectively. And so we put the correct and universal Drupal coding standards. Those are injected by the B-HAT extension so that when it generates code for you, it's formatted the way Drupal people like it. Space tabs, Doxygen style commenting. Curly bracket on the same line as the function like civilized people. And then you'll see that it throws a new pending exception. This is a special B-HAT exception that says we have a step definition for this, but no one's written any code yet. And B-HAT can kind of give you reporting on the fact that oh, we have these definitions, they're in code, but they're not implemented yet. So we're gonna skip those steps and tell you that those tests didn't pass really, but they didn't fail really. You're just not done. Question? So, right, so the question is when do you add the JavaScript tag? Yeah, in my opinion, you default to not using it. You can also do some clever stuff with the B-HAT.yaml. So in addition to the default context, down here you could set up like the slow context where you could go in here and replicate the parts of the configuration up here that you want to override. And just the parts that you specify will be overridden. So I think it's you put something like JavaScript driver or something, and then you can specify that you want gout for that, or sorry, that you want the non-JavaScript, you can default it to Selenium. I forget the exact syntax if you look it up. So what we've done in the past actually is had our continuous integration server actually have a mode that would just run all the stuff in Selenium. You have to be a little bit careful because there are some differences when you get down into the nitty-gritty about what gout can do and what Selenium can do. So it's possible to write step definitions that would work only with one. So that's good to be aware of and just watch out for the API because some will have a capability that the other doesn't. So some Selenium implementations, I'm not sure if this has been fixed, but some Selenium implementations for a long time couldn't actually give you the response code. So if you wanted to check that something was a 201 because strictly in HTTP, if you create a resource, you should return 201, not 200. So if you wanted to check for that, you had to use gout, but it all depends. So yeah, default to using gout, I would recommend, but sometimes you know that you have two different test cases because sometimes you know that you have graceful degradation without JavaScript and you have a different code path with JavaScript. Well there you might wanna have pretty much the same scenario with an at JavaScript and without one. So yeah, so we've got this step definition. Now there's two ways we could go about putting that in place. Inside of features, we have a folder called bootstrap and bootstrap has a file in there called feature context. All of this location of stuff can be overridden in the behat.yaml file. I recommend you leave it as the defaults because it's consistent with all the docs. But this is where you go in and start setting up your step definitions for your own custom stuff. In our case, we're going to show you how you would contribute this new step definition back to Drupal.org, because this is Drupalcon and we all want to benefit from your work. But if you're doing something for your site specifically, this is where that would go. So what you would do, I'm just going to show this, don't follow along, just watch me here because I'm going to reimplement this differently. If I open this, oh Linux with the copy and paste. It's okay, I was prepared for this eventuality. So if I go to bin behat again, another key tip here, I see new people not knowing to do this. So check it out. You can give the path to the feature you want to execute. So we know that ours was from the slides feature. So we'll just execute that one, no reason to run the homepage one again. We know this is the one that's our problem. So you can see that it is complaining as expected. So going into behat, features, bootstrap, feature context again. I'll just show you that I can paste that in. Is this legible back there? Can you guys see it? Should I make it bigger? Okay, good. So you'll see we've got our regex. It generated it for us. We put the thing in quotes and that automatically said, oh, you've got something in quotes. That's our convention for this is an argument that we're going to take dynamically. And so it automatically is saying with the title, regex that will extract your title and make it the first parameter. It's suggesting we call the first parameter arc one because it's creative. And then throw a new pending exception, which again was that thing that just announces, we haven't actually implemented this yet. So now if I run this again, you're going to see, oh, like I said, it kind of gives us a to do. It actually says to do write the pending exception. Now here's a key about behat. You want your exceptions to have useful and informative messages because they bubble up. So the default one for the pending exception, the default for that was to just kind of give us this to do that's not implemented yet. But if we give a more descriptive message, then that's what's going to bubble up. And we can say to do, you know what, we really don't even want this one here. Actually, we want to put this in its contrib module. So you can surface information to the user when you throw exceptions by passing in useful information. When you're looking through a Jenkins job at 1 a.m., trying to figure out why your build is failing, you're going to be mad if somebody didn't put a message in there about what broke and why. So make these as descriptive as possible. So again, you weren't supposed to follow me into copying that definition there because we're going to be cooler than that and we're going to put this thing where it should really belong. So our slideshow, if we want to see it, we can open our browser and click Drupal 7. Through the magic of vagrant, you can also go to localhost 7777 and you can see it on your local machine as well if working in the VM is annoying to you. And you can see that we have a little slideshow here. First slide says inflatable and has a beautiful big inflatable slide. In a second it should fade, excuse me, to a modern slide. There we go, so that's what we're going to be testing for is whether those slides are doing what they're supposed to be doing. A new world classic slide. It's beautiful. Okay, so the module that provides that, we're not going to talk too much about the module, but the module that provides that is beanslide. So if you go to var www7, I should talk about this too, there were a couple of questions about this. There is some controversy about where you should put your B-hat stuff. We don't have a really clear answer. This is something that Melissa and I like, but different people put them in different places. So here we've got in our Git repository, we have drupal.org in a sub folder called Drupal. And then next to that we have our tests. So that really nicely just moves them out of the web route but we've commit this at that top level so that the tests are revisioned with the site. I think that's important. You want your tests to be in the same repository as your site, so commit for commit. As you change stuff on the test side, you change stuff on the drupal side because obviously having those two get out of sync is going to cause fails. Another thing that people sometimes at ZivTech, sometimes we'll check things in to the inside the drupal web route. If the drupal web route is the route of our repository and we just have Apache configurations that ignore that folder. Another step that's popular is if you have a build step that's going to compile your SAS and CoffeeScript and do whatever other cleanup it needs to do to tear any test artifacts or any of that stuff back out before it gets sent to your production server. So these are some different ways to handle where the tests should be but I highly recommend getting them into version control with your site. Yes, you can put your tests in with the module and you can contribute them upstream which is what we should all be doing. Getting our step definitions into that module. I'm glad you asked. So how do we do that? Bean slide already had tests for Behat. So we wanted to start from ground zero. So I created a little sandbox project and if you were to type git branch dash a in here you would see that we have snapshots for the different steps. Step zero is delete all of the delightful tests so that we can start fresh. So what we're going to do is we're gonna go into Bean slide and we're going to create a new file called Bean slide.behat.inc. We're gonna open up a PHP tag and one thing that I want to call out here is this piece of configuration. So I really need to close my other copy of this wherever it lived. So your feature context, this is basically what would be generated for you if you ran Behat init when you're setting up a new Behat project. Behat init will stub out your folder structure and create your feature context. You'll see that we import a bunch of PSR, zero E name, space E classes and then we create a class called feature context. So that's by default your sort of main entry point for pulling in your step definitions. But in Behat.inc, or sorry, Behat.yaml, here's my Behat.yaml, in Behat.yaml this is your configuration for your whole Behat project. One of the things that we have in our extensions are Drupal extension. So pulling in the Mink extension pulls in a subcontext which is another one of those kinds of classes that defines a bunch of steps for us and those get imported into our one big step name space. You can't have two regexes that would match that are the same in two different steps. We need one answer to when I say when I visit thing in quotes that should only resolve to one function. So make sure that it'll complain and you'll have to fix it if you end up having two step definitions that step on each other's toes. But you end up pulling step definitions in from a bunch of subcontexts. That feature context file in your Bootstrap folder that's your main one. Extensions can pull in additional ones and then you can use the API to tack on additional ones as well if you wanna organize your code by splitting things up but we're not gonna go through that here. But what I do wanna call out here is that in Drupal extension, we have this subcontext key what I'm talking about subcontexts and we can give a list of paths that should be searched for files that match thing.bhat.ink. So we're gonna recursively crawl through var www seven Drupal sites and we probably should have made this a relative link but we're gonna crawl through var www seven Drupal sites and any www.bhat.ink file in the modules or themes or profiles folders in there they're gonna get picked up. Yeah, you could just drop it right in the sites folder if you wanted to but the idea is to be able to extract them from the projects where they belong which could also be from your custom modules. If you wanted to organize things by the module that they're testing you could drop the www.bhat.ink into the custom module so those are packaged together. Um, yes. Yeah, um, I'm pretty sure, I'm pretty sure we could do this and it would work. It uses symphonies finder component and I'm pretty sure symphonies finder component will resolve relative paths. We did that with the hope that we would make it more reliable and we should have thoroughly tested the relative path thing there. My bad. Oh, base URL. Base URL you sort of need to have set. I don't think there's an easy way to pass that in at the command line but again you can have different contexts. So if you need to run this against different contexts what you could do is create tizolocal as a context and then have you know, extensions behat mink extension and I can override with the environment that I'm setting. Melissa, did you have a better answer for that? Hey, sorry. So yeah, you can see we've got behat.local.yaml example, this is from the doobie project. Sorry. Can I just say something on that? And in terms of the profiles and the yaml file it's just a configuration on the command line so you can pass like a parameter instead of just running bin slash behat you can run bin slash behat minus p and then the profile name and that will trigger the profile. Similarly, you can have another yaml file and if you do bin behat minus c and c for like config file and pass the path to your second yaml file so you can have as many yaml files as you want if you want to have that. It's up to you. Yeah, so the short answer for the hard coded paths is my bad. The longer answer is sort of exactly right. This is, and I misspoke when I said context. This is a profile. So you can see here I just made a TISO profile. That when I introduced myself I didn't say I'm TISO from Drupal.org on Twitter which is probably if you know me, how you know me. And where you can bother me if none of this makes sense. I'm on IRC and I love talking about this stuff so if you have any trouble please ping me TISO in IRC or Twitter and I'll try to help you out. So again, this is an example of another profile so now I have a TISO profile where I'm gonna try to run this against google.com on port 777. Probably not gonna work very well. Hey Pat. So if I do bin slash behat dash p TISO now we're going to try to run this against Google and we're gonna fail miserably. So that's just gonna sit here in time out trying to talk to 777 probably. But that's how you would override it for different environments and I've seen people have their own name. The way my company solves that is we use a standard vagrant development environment so everyone's path for their local development is the same. So the default profile works with your local vagrant environment like the ones you guys have here so everyone types a vagrant up. It builds them a site, they grab the site, it's in the same place with all the same configuration and then we don't have to worry about people be having heterogeneous environments and we don't have to worry about people building the client or the company 20 hours for setting up solar because it's just on the box. If you're interested in vagrant stuff I have a session tomorrow, shameless self plug on a project called Previsa with Pat Kahn back there. We're trying to make a standardized Drupal vagrant environment that everybody can use. So if that sounds cool, please come and please join us on GitHub in the issue queue. Okay, so there you go, you see it totally filled. So how do we get started? We're going to again, we're gonna create this beanslide.behat.inc and then we're going to need to import a few things. Now I recommend using a reference implementation like beanslide to figure out exactly what you need to import rather than trying to hunt it down yourself. And I'm going to cheat for the sake of time. And actually, yeah, I'm gonna do that. Yeah, so if you do git branch dash a on your local you'll see that you're on step zero zero be hat clean, which means we haven't done the work yet. And if you wanted to just check out a different version to reference, you totally can. So the first thing that I'm going to do is, what's that? Oh, so you need to be in, I'm sorry, I switched. I do that sometimes, call me out on it please. I switched directories quick. So that's in var, dub, dub, dub, seven Drupal and then it sites all modules beanslide. Cause that's the beanslide project. And I do have a sandbox of this on Drupal.org. So if you search Tizzo Drupal.org beanslide, come on Google, there you go. You can also find my project. And if you wanted to look around, you could see that we've got those branches that I was talking about right here. So if you kind of want to copy and paste from something you can see the different steps here. Okay, so let's forge ahead. So the first thing that we need to do is pull in a couple of classes so that we can use them. So if you're not familiar with PHP namespaces, the idea is that we'll say use Drupal extension context, Drupal subcontext interface, and that's going to auto load that subcontext. Actually it wouldn't auto load till we instantiate I think. But what it's going to do is say we can refer to this class just by saying Drupal subcontext interface. And it's going to get it from the Drupal extension because we've got behat doing an auto loader thing for us. So that's all we need to do is say we want to use that and then we can just call that out and it'll figure out what it refers to. We don't have to use that giant long string. Then we're also going to say use behat, behat context, behat context. So again, this is PSR0 structure. This is inherited from symphony. Behats a symphony thing originally. So it follows sort of their conventions for how things get nested and put together. I get that part's weird, copy and paste it. Next thing we're going to do is create our class for our subcontext. So again, I'm going to copy and paste that. You'll remember it better if you type it, but you can copy and paste it too. So we've got a few things going on on this line. You'll see we've got the class bean slide subcontext. I just made that up. It seemed reasonable. It extends behat context. So behat context is a class that we're inheriting a bunch of functionality from. So we can say get me the main context and that's going to get me that feature context file that we looked at before from our bootstrap file. That's kind of the parent. So the child can say get me the main context and then the main context has methods like get me the session. And the session is the current running driver session, the current running browser session. So that's actually going to be depending on how it works. You're going to have an object that might be gout underneath or might be the Selenium driver underneath, but it's sort of wrapped. So we need to extend that so that we don't have to reproduce all that work. We just get it for free. And now this is the piece that's special for the Drupal extension. We need to implement Drupal subcontext interface. Why? Because when you import a subcontext, you need to give it an alias, which is a friendly human readable name more or less, by which you can refer to it in code. We want to just import this magically. So what we've done is we've created a new method that you have to implement, which basically just returns the name of your module so that we can use that as the custom key. So your module basically, you just need to announce what your unique name is. In Drupal, the module name is usually the unique name space. So that's what we use. So the function we need to implement is this static method get alias. So I'm going to return just bean slide, because that's unique. And now if I run bnb hat, we should still be working, still be functioning. I'm gonna cancel out of that because I only meant to run features slide.feature. While I'm here, another pro tip, I'm gonna open up slide.feature. Another pro tip, you can actually also name the scenario. So when you start having features that have 50 lines and 20 scenarios in one big feature file, if you can fit 50 scenarios in 20 lines, you're probably doing it wrong. I'm just gonna say that too. But you can have, you'll start to get big feature files that have a whole bunch of scenarios in them and some of them might be kind of slow. You can run one individually by putting a colon and then the line number. So this should be the line number of where that feature starts, where the word scenario appears. That'd be line seven. So I can say feature slides, feature seven, and it'll run just that one, which is really helpful and saves you a lot of time. And so you'll see, again, we still don't have this defined. So I'm now going to copy and paste this. This is the part where you should follow along copying and pasting this. I'm gonna run set paste so that I can paste it. And now you can see I've got a pending exception that I'm gonna be throwing. So now we should see what we saw before, that little to do. So now we have created fatal class pending exception not found. Ah, I need to add a use statement for the pending exception. And I can steal that from my main feature context. That was that file that was in features bootstrapped feature context. So if I steal that use statement, I need a comma or a line break. Anybody, I forget. All right. Yeah, you can either put a semicolon and then the word use again, or you can just separate the uses by commas. So there we go. Now we're throwing our feature exception, pending exception, and it says to do write a pending definition. So now we get to really dig in. So what do we wanna do? Again, I'm gonna go back to sandbox slash tizzo slash 2088877, you all got that. And I'm gonna hop up to my other branch, step two, and I'm gonna copy and paste from this for the sake of time. Sure, tree and bean slide dot behead dot ink. And I'm gonna start taking pieces from this individually. So first, so first this is a little helper that I like to throw in. So I thought I'd include it for you guys since I think it'll save time for you long term. I like to create a little function called get session. Why? Because again, a lot of the time if you're going to need to do something, you're going to need to ask the browser to do it. And the browser is your session. But you can't get the session directly from a subcontext, you need to get it from the main context. So some subcontexts come with a method called get main context, or actually all behat contexts do. So you can say get main context and then you can call the get session method on that. This just shortens that because we're gonna be calling it a lot. So for those following along at home, if anyone's watching this recording that had a big dead space in the middle of it for setup, that the useful tip there is adding get session here. That's the method that you use from the main context a lot. So whenever you're developing your own custom steps in your main context and you realize, oh, this is generalizable, I should contribute it because I'm a good person. You don't have to refactor if your subcontext has a get session method because you'll be able to call this get session and it'll exist. So next thing we're going to do is remember to enter insert mode before we paste stuff into Vim is we're gonna get the page object. So the page object is part of the mink API. So that's the thing we can talk to your browser. So that's the thing that can search the page for CSS selectors. It's a transverse, it's a traversable element, traversable node element. I'd have to check the mink API but there's sort of a whole bunch of functionality that different pieces get. Page is one of those items and it can do things like searching. So there we've got a page, right? I've got a new variable called page. Next thing I'm going to want to do is dig into the HTML to see if I see the slide, right? So again, I should have started off probably by saying our definition is I should see a slide with the title, give a title. So what we want to do is we want to find the slide. So we just go into Firebug or Chrome DevTools as the case may be and hunt around in here and see that the ID beam slide identifies our slideshow. It then has a UL class of slides and then list items for each individual slide and you'll see that it does a bunch of inline style-y stuff to do the animations. Great, that means that we know everything we need to know to start hunting around in here. And the first thing we're going to need to do is find the headline. So first off, I'm just going to call page find and now I'm going to use the CSS plugin. There are other plugins for doing find and for looking within the page. CSS is the one that uses a CSS selector. Another popular one is the XPath. There's also one called named that a bunch of the minc stuff uses internally. But CSS is probably the one you're going to default to. So here we're going to look inside for a div with pound beam slide, with a ul.slides, with an li.first, field name, field slide headline. From hunting through the HTML, we saw that that was the selector to get a headline. Savvy? Now, what if we don't get headline? This is going to be a key because a lot of people you're going to forget this step. So if we call find and we don't get anything back, it doesn't find a match, it returns null. If it doesn't return null, it returns an object that again is a traversable element. So if we get back an object from the Minka API, we're good. If we get null, we're bad. So I'm just checking is object. And if anything is wrong, what I do is I throw an exception. So throwing exceptions is key in behat. It's how you say something went wrong. You throw an exception. Mink, sorry, behat catches it and behat reports on what the text of that exception was. So again, throw exceptions when anything goes wrong and be descriptive. So here slide show title wasn't found. Now we have something that's at least going to test that we found a slide show title. So this should pass. But if I wanted to break it, I could just comment out, I click on slides and try to run this from the home page and we should see a fail. Boop, slide shows slide title not found. Wasn't on the page and I know exactly what went wrong. So now I'm going to copy and paste a little more out of my pre-prepared example here. Save us awesome typing. And at this point, if you guys just want to check out the branch starting with 01, you'll have the same code that I do and you can tinker with it, break it in creative and new ways. 15 minutes, awesome. We started with this section a little later than... Where what? Is 03, perfect, 03. Thank you. So if you check out step 03, you'll be, you should see what I see here pretty much. So we're going to be checking to see whether that headline, you'll see that headline has a get text method. That strips out all the HTML and just gives you the actual text. So then we're just doing an STR position to see if it contains that. And then we're printing out for clarity what the actual title was. Now the fail that I had was because my code, I retitled that from arg to title just to be more descriptive. I like to keep those variables really descriptive about exactly what their contents is. And I think now we should have a pass. Yep, so now we're passing. If we were to go and change that so that it wasn't looking for inflatable, it was looking for inflatables, then we should have a fail. So we've implemented a test that can translate natural language into looking into the page, into doing a CSS query, into looking at the context of the text. Now one pro tip here, I recommend using this in association with PHP unit for clarity and simplicity. We didn't pull that in because it's kind of a whole other library with its whole other set of stuff, but that has functions that just do single asserts so that you can say things like assert contains the thing you're looking for and the thing to search within with the message to report if something goes wrong. So you can take those four lines and boil them into one just by using an assertion and it'll throw the exception. It also has great things like checking to see what's in an array and the exception that it'll throw will give your message and then we'll also say thing you were looking for wasn't found within imploded list of things that you were looking within. So that's really helpful. So I highly recommend importing something like like PHP unit to get assertion so that you can just boil that down and simplify your code. So I wanna look, I'm gonna guide you guys through a couple more of these examples. Why don't I just, we prepared this just for you guys, just for you. So we've never done it before and there was a little more content to cover than I think we realized in this amount of time. Because we did it just for you. Is it working? It's sucking up. So I just wanna go through and I'm gonna show you some of the other definitions that we created here. I'm just gonna reset hard and check out 06 to jump through the last step. All right, fine. That's not to say that you're any less special though. Maybe a little less special. Okay, so there we go. So we've got a few other helpers here. So, what's that? So since we have very limited time left, if you wanna give us a little bit of time for the talk then we'll be great. Can I take two more minutes? Yeah, sure. Sorry, Graham. That's good. So a couple of notes here about other implementation details. The name of the function doesn't matter. It's all about that mapping of the regex. Also, you can map multiple regexes. So if I had another thing that was like a slightly different syntax that I also wanted to be able to search, I could do that too. If you can also, because this is just PHP code, you can call one step of definition from another. So for example, if we had something that said assert a specific slide contains and assert a specific slide has a title and assert one has an image, we could call both of those from one that checks both so that you can create kind of consolidated slides so that you can chain together other implementations. There's a couple of ways to do this. This is a simple one. You can also use named placeholders so that if you wanna specify the parameters in different orders, you can do that using the sort of regexes that are there. The examples from Mink extension here are excellent for that. Another thing worth noting is there is a find all method that will return an array of those node objects to get all of the matches. So if I wanted to do that, assert that one of the slides contains, I should have been looking for all of the titles and looping over and checking each one. So our initial definition was incomplete and we were gonna loop back and look at that. So keep in mind whether you're finding one which will find the first one of multiple matches or whether you're finding multiple where you'll get an array and you can do whatever you want with each of the ones in the series. Also just the last, my last thing is comments. So this is specifying, this is for beanslides7.x-1.x. So if you have two slideshows and you see this definition, you'll know which one it's for. And last but not least, I like to do these helper methods like offset, get offset from English. So this will find the array position that you should have by looking it up among the English words first, second, third, fourth, fifth, sixth. So that you can say in the first slide, I have this thing. Just to keep it really fluid and natural. And I also say that to underscore the point, Melissa made earlier, you don't wanna be putting CSS selectors and things into your feature definitions because of two things. One, it makes business users turn their minds right off. And two, it makes your tests really brittle. Cause the idea is if your application logic doesn't change, if the user should have the same experience, you should be able to use the same feature project when you port your project from Drupal 6 to Drupal 7 or Rails because it shouldn't be picking out CSS selectors. It should be the implementation details of what PHP code that's running is changing. Okay, sorry for eating up a lot of your time, Graham. I'll turn it over to you. So as we're transitioning, for those at home, there's a Be Hat group on groups.drupal.org, and that's where we're gonna be discussing this stuff and solving your problems. So yeah, I'll probably just skip through this real quick since we're kind of pushed for time. But I just wanted to briefly talk about some considerations about how we might use this, we as in the Drupal project, we're going forward and kickstart some discussions about how we might take this forward within the community. So as most people know, obviously at the moment we use simple tests and PHP unit as well now in Drupal 8. Inside simple tests, we have over 300 web tests. I don't know the exact number, that was kind of roughly from doing a search, this morning. What these web tests do is basically they run the simple tests, has its own browser, and it tests functionality on the site. Similar to Be Hat, so what we could do is, here's an example of the color module test. At the moment, it extends the web test base, which is a class in simple tests. It's got setup functionality, it's all code, and then carry on, carry on, set up some stuff for the color module, and then we have a test here, which tests that when you go to this page and change the color palette, then it actually changes the color of something on your site. If you look at the same test, in Be Hat it would potentially look something like this. I hope everybody can see that, it's kind of small. But yeah, there's some potential benefits to this, aside from the obvious, the tests are in English. It opens up for the possibilities of cross-browser testing, so testing in different configurations of operating systems and browsers, via, maybe even via saucelab.com, they do free accounts for open source projects. Tests are in a real browser, so you can run in Firefox version 14, or Chrome latest, or whatever, not in the simple test browser, which nobody uses, by the way, in the world, apart from us. Also tests don't actually change when API changes, if we move through Drupal versions, and the functionality's not changing, you wouldn't have to change the test, because the test actually tests what you can see. Whereas with simple tests, if the API does change a lot of, well, not all the time, but sometimes you do have to actually change the simple test as well. There's probably more benefits I haven't thought about. But there's also a lot of things to consider, so it's not just a simple thing, like, okay, yeah, let's use Behat, just decide that one day and it's in the next. There's obviously a lot of things that needs to be discussed before we even get down to writing any code or implementing this in Drupal. It's like how do we educate the community and train them about Behat and Mink and all these new things, because there's no use putting in a new test framework that nobody knows how to use it, because nobody will actually, we won't get any tests. There's also a massive consideration around the current Drupal.org infrastructure and the test spot for Drupal.org. If we were to do this, then that would completely break test spot. We don't have a test spot for Behat, for example, multi-lingual or something else I threw in there. I'm not really sure what the issues would be there, but I'm sure there may be some issues. So what I'd really like to do is get in talked about a lot Behat in the community. There's been various sessions on it this week and I just like to kind of plant these ideas in people's minds about how we might use it as a project and kind of kickstart more discussion. So as Seth said, to continue those discussions, let's talk on groups.drupal.org. If anybody wants to have a look at the second link is a sandbox where I have, it's slightly out of date with the current, the latest head of Drupal 8, but the sandbox basically is a core sandbox where I have put Behat into Drupal core and I've converted a few simple tests. So the color module ones in there, there's also a couple other modules. I've done that and they run against the latest Drupal 8. As I said, it's slightly out of date. I haven't been given it much love, but I just put it there as kind of a proof of concept to show people what's kind of possible and hopefully to again kickstart some more discussion. So that's all I wanted to say. Sam, do you want to talk about anything? Yeah, I was gonna take just a minute, sure. Yeah, if anybody's got any questions, then feel free to shout. So I'm gonna be the least snazzy of all the people here. I'm just gonna talk for two minutes. I just want to make the general point that we're not looking towards core so much, but looking towards contrib. Then I think the thing to focus on as the reshareable element, this has been said in one way or another here before, but it's the step definitions. Like writing step definitions, that's something that you can offer up, that can be reused by other people that can be brought in. As a, I mean, I can tell you that like at NBC, as we're moving into using this, one of the primary things we've been investing effort in is just figuring out where we're going to map out the step definitions that we have, like when they're supposed to be used, who's spreading them, just there's a lot of maintenance that goes into creating a whole vocabulary out of these step definitions, and in the same way that you know, you should write drush commands for your modules if you write modules in order to ease access to their functionality from the command line. Community starts moving towards adopting more of a hat, should be looked at as I would say a similar type of task to write step definitions for things that are unique to your module, assuming that it's appropriate. And you know, over time, standards will emerge, we'll figure out nice ways to phrase these things, go base and share that information, there'll be nice drush tools that collect all the step definitions that are in your site. Although I suppose there's that for, the hat can do most of this, most of that right out of the box. So, but yeah, think about your, craft your step definitions carefully. You know, start messy with what works for you, but if you think that there's some value there for some reuse or something like that, or if you even see a contributed module that you use a lot, you think, hey, we wrote a bunch of tests for this on our site that we met ourselves, then gradually maybe we can move towards contributing back step definitions, that's probably the most reachable thing. So, yep. Any questions for any of us? Oh, cool. Yeah, you know. Sorry, I just wanted to add around the core thing, it may be decided that like after some discussions that yeah, we don't actually want it in core, that's cool too. I'm not saying like, I'm not pushing the idea of like, yeah, we definitely need to get this in core, like I just want to have like discussions around idea. And then maybe decided that yeah, we don't want it, that's cool too, but it's always cool to float new ideas. Sorry, so you said at the very beginning that he wasn't meant to replace unit testing. So, what are you using for unit testing? Are you still carrying on using the same thing for unit testing on Drupal.org? Drupal.org doesn't have unit testing itself, it relies on the simple tests. Are you asking about Drupal.org, the website, or are you asking about for the development of Drupal the software? I was asking Drupal.org, but I guess. Yeah, we don't have tests. Well, we have been here, we have had tests, but the fact that we don't have unit tests is a source of ongoing embarrassment. You guys said about technical debt? Yeah. Okay, yeah. Yeah, technical debt on Drupal.org. But we do have good testing for the version control module and a bunch of individual components. Right, that's fair. And then we have integration tests for the thing as a whole. But Drupal Core in Drupal 8 is going to be using PHP unit for unit tests. Mark Sannabam had a great presentation on unit testing in Drupal 8. I highly recommend watching the video, it was yesterday. But yeah, and we use unit, we use PHP unit or simple test depending on the case for unit testing. So that's for testing that your code in isolation works. And then this is to make sure that once somebody clicky clicked a bunch of stuff together with views and rules and panels, that it actually does what it's supposed to. So at some point we may be able to adopt PHP unit on Drupal.org, that'd be great whenever we get there. Any other questions, comments, concerns? Any amazing insights? Sure. So what's the state of the documentation on Behat because there's lots of stuff to learn? I'd say it's a little bit mixed. The documentation is fairly good, but some of it just involves reading the code. So if you go to behat.org, there is a link to the Behat docs, which is a pretty nice overflow of a quick intro, installations, cookbooks, best practices, links to more theory. And then it also has a generated PHP doc thing. And if you're down with object-oriented programming and that kind of stuff makes sense to you, this stuff is named really intuitively and pretty well architected and it starts making sense fairly quickly. Even though the giant list of interdependencies and interrelations can be a little daunting at first. So can we find your pro tips you gave us today? Yes. I will work on a blog post to wrap up the pro tips and I will post it to the groups.drupal.org, Behat group. Okay, thanks. Yep. So this was a great presentation and it really builds what I see as the momentum. We had some really good discussion at Portland about the value of Behat and it became clear the community was interested. I was absolutely thrilled when on the last day of submissions for this conference, I went to put in a Behat talk and discovered there were already three of them. So I went and did something else because it was clear the community had started to say, hey, Behat is something we want to get into. This has been a great intro I think and what I really wanna make sure that everyone who's here understands is the next step really is people starting to see the value of writing tests for contrib modules because if you do it for yourself, you're gonna start saying, gee, I actually am testing a piece of contrib here. It's not really anything custom that's only about my site. I'm actually testing this module, this module and this module. If you give that stuff back, you save everyone else in this room and in the community from having to write those same tests. If you do that and everybody else does that, six months from now, we're gonna look back and we're gonna go, wow, the Behat tests are awesome because there's all of this pre-written stuff and instead of having to code, you're gonna be like, well, I just wrote some English and boom, my new custom module is all tested. I didn't have to write a lick of code for it, which is great and that's really where we're going and I just want people to see that vision because six months ago, we had people go, what's Behat? Now we've got people going, I'm interested in Behat. Six months from now, we could have people going, wow, can you imagine what Drupal was like before we did SimpleTest and how long it took to get Drupal on board with SimpleTest? Behat, we're seeing it really take off really quick. It's not gonna be a slow process. We just have to have the community realize how valuable it'll be. And I have one other thing here. Howard, are you on IRC? I believe I am on IRC. Excellent, I have an important thing to share with you. Does everybody get the joke on this, by the way? The Behat? He's wearing a Behat. So here's another joke coming, except it's not a joke. So cucumber, well, gherkins are multilingual. So you can write them in any number of different languages, including an official translation for LOL, where Ohai is given, or sorry, no, that's feature, before scenario, mission, mission scenario, before his background, then, yeah, there you go. So there are standards for this. There's also pirate and a number of other humorous translations. So once you master Behat, I encourage you to translate everything that you do to LOL. This is clearly, clearly, you would do this. There's also support for a lot of extra actual languages that would be useful to folks just saying. Yeah, yeah, if you don't want to write it in LOL, you can use French or German or whatever. That too, that too. So how many people in here are gonna be using Behat on projects? Do we sell you? Nice. If even half of that is true, that is an awesome win. So one more thing to add. Howard pulled up the Drupal extension site. So Jonathan Headstrom from OpenSorcery, who's also a Drush maintainer, is one of the main contributors, and I'm one of the other co-maintainers for the Drupal extension. One of the reasons that our docs are sort of buried is that they are actually supposed to be deployed when version 3.0 of Behat comes out, which is done and in beta. So it's supposed to come out any time, and the docs will actually live on Behat.org rather than Drupal.org, since this is not a module, it's an extension to Behat. So that's where they're gonna live, and I just want people to know that one, this is a friendly issue queue. So we are both really busy, it's not always super fast, but it is nice. So don't be afraid to ask questions, don't be afraid to ask silly questions in it, and also don't be afraid to help each other there. So it's great to have people participating and asking and talking about what is generalizable. And if you're writing steps for a contrib module and you're like, like media has been the thorn in my side, because it takes a lot of special steps. And so as I work on that, like if you're curious, but you maybe don't wanna approach a module maintainer because you're not sure they wanna commit it and you're not sure it's done, it's okay to ask questions, not just about tests that would apply to Drupal in general, but also contrib modules in the Drupal extension queue. Like those of us who hang out there and have talked about it and it's great to kick it around for feedback, do make sure you tell us what module it's for and you know, do those nice things that you do when you open issues. But the conversation and cooperating on this stuff is something that we're all very committed to. Are there questions folks? Awesome. Right, so I hang out in Drupal and Drupal contribute a lot as do most of the other folks that were presenting here I think. What's that? Sure. Yeah, and then there's also a BeHat channel on FreeNode, pound capital B BeHat and Jonathan Headstrom and I both also hang out in there. So do quite a few Drupal people, it's a quiet channel. You don't, I mean if we happen to notice, you'll get an answer sometimes, a lot of times it will just sit, but you're welcome to ask it and if people are around they can try there too. And the more of us that are there, the chatter it'll get about our Drupal steps. And of course the issue queue and groups at Drupal.org is really good because it's asynchronous and we don't have to be there at the same time you are. Especially if we're in another time zone. But thanks a lot guys. Thanks everybody. Thank you. Thanks.