 Hello, everyone. Hello. Welcome to Bending Be Hats Benefits. My name is Steve Perche. I'm an agency and community engineer at Pantheon. We're a sponsor here at DrupalCon. You can see us down on the main convention floor. Again, I'm an agency and community engineer. That means I spend a lot of my time working with agencies and web development teams, adopting Pantheon, working on Drupal best practices. A lot of what I've done lately is related to our continuous integration tooling, often when working with continuous integration. You may want to use an automated testing tool like Be Hat, and that's what we'll be talking about today. You can find me online as Steve Vector on githubdrupal.org, wordpress.org, Twitter. I'll be tweeting out a link to these slides after the presentation, and you can, of course, email me. So today where we're talking about Be Hat, we'll be talking about the what, the why, the who, the where, the when, and the how of Be Hat. Those wonderful five W's and, of course, a how. We're starting with what? What is Be Hat? I'm curious, in the audience here today, who here has used Be Hat? Oh, wow, more people than I was expecting. Not everyone in the room, but for the recording we had, I think a little over half the audience raised their hands. For those of you who haven't used Be Hat, that's fine, we'll go over some introductory things. I'm curious, are the people in the room who have used Be Hat? How many of you have used it as a client communication tool? How many of you who are using Be Hat have clients who are actually looking at your Be Hat test and it's far fewer people in the audience? I think I saw two hands there. Two people have clients who are looking at their Be Hat tests. Okay, good to know. So using Be Hat and Drupal together, again, we're talking about just what is Be Hat? Be Hat is a PHP command line tool that you can use for behavior-driven testing. When you're using Be Hat, there are three main elements that I want you to be aware of. We have Be Hat, the PHP command line tool that we'll take a look at in just a second. We have an actual Drupal site somewhere. I'm about to do a live demo where I have a site running locally. You might have a site running up on the internet, maybe on a Calibox locally. You may have a site somewhere. And then you have documented what you expect the behavior of that site to be. These expectations are written in plain English. They're written in .feature files. These are simply plain text files that document what you expect the behavior of your Drupal site to be. So Be Hat looks at those expectations, looks at the actual Drupal site, and compares the two. So it reads the .feature files, and it interacts with the Drupal site. Now, often when I heard Be Hat describe to me, when I was first getting familiar with it about two years ago, I just had a very hard time wrapping my mind around, wait, what are we actually talking about? What does this actually look like to use Be Hat? So it's a little risky to start a presentation with a live demo. I feel like I'm living on the edge here, living dangerously, but that is what I'm going to do. I am going to start this presentation with a live demo. Jumping over to my command line here. I have Be Hat, the command line utility. I happen to be inside of a Drupal 8 site right now. I'm gonna jump over to PHP Storm, or we can see here is all of my Drupal 8 code. I've got my vendor directory where dependencies like Be Hat live. And I have a Be Hat-local.yaml file. .local is compared to Pantheon. If I wanted to run these Be Hat tests against Pantheon, I could, but I don't want to be extra risky in my live demo here. I'll run this against a local site. So we can see here, the main bit of configuration I want you to see here is that I am pointing at a base URL that is local here. Again, jumping back to the command line. So I've told Be Hat on the command line, haven't yet run this command yet, but I'm about to. I've told it that the configuration is here. Be Hat local.yaml is the configuration, the config that it's about to take, and the .feature file that I want Be Hat to process, the set of expectations that I want Be Hat to process is a file named who's new .feature. And that lives in the features directory. Jumping back over to PHP Storm, just to briefly look at this, what we'll look at it in greater details, the presentation goes on. But I do just want you to see that this is a plain text file. I've got two Be Hat scenarios that I'm about to run here. All right, so now, I hadn't showed this part to you yet. I do have the actual Drupal site. This is a totally vanilla Drupal core site. Not much to it. The part that we're testing with these Be Hat scenarios is the who's new block. All that does is just list the recently created site, or recently created accounts on this site. It's just a normal part of Drupal core. I've got a very simple Be Hat feature file that's going to test my expectations for the who's new block. All right, live demo time. Going to, oh no, oh no, what did I do as I was, oh wow, that's, I added a comma as I was interacting with PHP Storm. All right, so I feel like that was probably my lesson for attempting a live demo. I hope it goes smoothly from here on out. I've added break points here, a nice feature in the Drupal extension for Be Hat, which I'll talk about later, is this ability to add break points. I'm just gonna be hitting the enter key as I go along here to let me pause and talk about these things. So Be Hat is processing my dot feature file up at the top here. It's telling me the feature that's being tested. It's the who's new block. And here we get really to the meat of things. Be Hat is logging in as an administrator. It has a used drush to create a brand new administrative account. And with that administrative account, it's now going to visit the URL admin slash people slash create. Looking at that in my real browser, this is just the normal Drupal core form for adding a new user account. And now Be Hat is going to fill out all of these form elements. So it's adding the email address, adding the name, adding the password on and on, hitting the create new account button. And then going back to the homepage and hooray, we have two new accounts. The one that was just created so that we could log in as a throwaway administrative account. And then that test account that was actually added to be detected by this Be Hat scenario. And we can see here that the way it's checking for this test account is by just checking for the string test account one in the CSS ID block views block whose new block one. So this is a test that I think developers can understand. We can understand what Be Hat is doing. It's just simulating some browser interaction. And this gets the job done for automation. This isn't a great tool for communicating with your client. I wouldn't want to show someone unfamiliar with the implementation details of HTML. This as a way of understanding the whose new block. There's a whole lot of implementation details here around the creating account form. So this second scenario is going to test basically the same thing. And I think a way that is more understandable by perhaps your client. It will again sign in first as an administrator. And then using custom written Be Hat steps is going to do basically the same thing. It's going to make a new account rather than making that new account by manually visiting the create account form. I've just written a custom step that does that same thing. Creates a new user account and then goes to the homepage specifying it as when I go to the homepage rather than the more developer oriented when I visit just slash. And then it checks for basically the same thing. Then I should see the new username for that new account in the whose new block. I think this is a more readable way of doing it than referencing the exact CSS ID in your feature file. So for those of you new to Be Hat, that may have been a whole lot of information all at once but I did want to just give you a sense of what does it look like to run a Be Hat test. There we go. We have two scenarios that passed. We could plug this into a continuous integration environment and automatically execute our tests. And we saw two types of tests. Now why would you actually do this? Jumping back to the slides here. Why would you use Be Hat? Why would you use any automated testing tool? Well, continuous integration is a really popular topic these days. If you go down to the Pantheon booth, you'll see a demonstration of a relatively complex continuous integration process. One of the great parts about continuous integration is that if you have automated tests, you can be much more confident that the changes you're making, that the new feature you're adding, if you're adding the who's new block to your site, none of your other features have broken. That's a great part of testing, just the ability to automatically run those tests over and over again in perpetuity, verifying that you haven't broken anything. But there are other reasons to test, quoting my colleague, Michelle Creechie here. There are other reasons to test, among them defining done with a client and translating the definition of done into code so that we can know when a feature is complete. This adds focus, it adds documentation, and yes, it adds regression testing. And this really is where Behat shines, the ability to have a better conversation with your client about what do they actually expect from that who's new block. This allows us to get both of these benefits. So the primary benefits of Behat that I'll be talking about today, automation. This is a benefit that you get really from any automated testing tool. You get this benefit if you're using Selenium directly. You get this benefit if you're using PHP unit. Behat shines with the benefit of clarity. If you're using Behat well, this should give you a clearer picture as a whole team. The developers, the project managers, your client stakeholders, it should give everyone on the project a clearer understanding of what the definition of done is. Quality is a benefit we often expect to get from testing tools. I don't think this is a benefit you get directly from Behat, and I'll explain why. So a simplified view of the different types of testing. This comes from a blog post on the economy of testing, a blog post from the author of Behat itself. He simplifies types of testing into unit testing, integration testing, and system testing. When we're talking about unit testing, we're talking about the smallest units that we have as developers, testing individual functions, testing individual classes, testing individual methods. We know we're doing unit testing when we can reason about our tests in isolation, when we can look at a test and see that the only thing that this test touches is a method and it's a method that someone on the team wrote, it's a method that we have totally under our control. So this is a great way of answering the question, is our code any good? If you're doing test driven development and writing these very granular unit tests, these unit tests will guide you towards writing good code, code that is decoupled, code that is clean, code that's readable. Integration tests are meant to give you a slightly different benefit. Integration tests, well they do what it sounds like, they test the integration of different units. Sometimes those may be units totally under our control, sometimes that may be units that interact with a third party API that's outside of our control. I think we're less likely to see integration testing in the Drupal community because often we're dealing with the smallest units or we're dealing with the complete picture. So integration tests are meant to answer this question of do our pieces fit together, but often when we're working with Drupal, it's hard to subdivide our pieces and say I'm only testing the backend and not testing the theme layer. Drupal is still relatively monolithic. Often when we're working with Drupal, we're doing system testing and this is really what Behat is designed for, system testing, testing a complete website very often. Behat can be used for testing things other than websites, but mainly what we're talking about today is testing a complete website running in a production-like environment or just running on my local machine here. This is verifying the system as a whole, answering the question, do all of our pieces fit together? You can think of these different types of testing as answering, again, different questions. Did we build the thing right? This is a question unit testing can help you address versus did we build the right thing? Did we build what our client actually expected? You could have the old users block and have really clean code that gives you the old users block, but that's not what your client asked for. Building the right thing is what Behat is designed for. So who is going to use Behat? Well, who cares about our website at all? Often when we're making websites, we have at least two groups of people. We have often the people building the website, often these are people working at an agency, working with a client, your developers, your designers, project managers, everyone working at the agency, and that they can be somewhat separated from the people using the website. Often you have a main point of contact with your client. You have stakeholders who are working at the same company as your client, and they have their own end customers. And one way to conceptualize how all of these different people work together is by separating each group from one another. You have the stakeholders and the end customers, and they have expectations for what the website is supposed to do, and they may communicate that to the one person at your client who really owns the website as a whole. You can call that person a product owner. The product owner at your client company describes what they expect from the website, maybe to a tech lead or a business analyst at the agency or within the web development team. They then write down detailed requirements, often just in bullet list form of what are the requirements for this site. Those can then be formed into tickets that get handed off to a project manager. The project manager then hands off those tickets to a developer. The developer then may take the tickets, look at the bulleted list, write tests that are really only readable by developers due to the work, maybe hand it off to a QA person, and then the QA person may send it along to the final approval and deployment process. And each step is separated out from the next. The developer maybe is never talking to the product owner, maybe never talking to an actual person who really relies on this website. So if this is the way you're conceptualizing of a development process, then BeHat is going to be a difficult tool to use because BeHat encourages everyone in the process to at least at one point in time have a conversation about what does everyone actually expect from the website. At least one conversation that is going to set expectations for what we're doing here. A leader in the behavior-driven development community says, this is really the point of writing our expectations in these plain English dot feature files. They allow more people to get involved. If you're not writing down your expectations in plain English, then you're going to have clients who think, well, I don't really know how the site really works. That's for the developers to know only the developers can read the tests. I'm further removed from that. By writing out your expectations in plain English is a chance for everyone in the process to get involved and have a common understanding of what we're doing here. So these dot feature files should really be a representation of a conversation that was had at some point in time by everyone involved in the process. This should really be the anatomy of a conversation. So what are we talking about at all? That's that first line in the dot feature file. What are we even talking about? We're talking about the who's new block. That should be easy to scan right there at the top of the file. That is the feature that we're talking about. And who even needs that and why do they need that? These top lines are not the computer-processable lines. As we get lower, we'll get to the machine-executable steps. These are really just for people to read and get some understanding of what are we even talking about, what feature are we building, if your expectations are just written out in bulleted lists, it can be difficult to know what are we even talking about. Who needs this feature? Why do they need it? Getting into these scenarios. Each scenario is a description of what can this feature even do? Well, in this case, this feature can display new users in a list. You might have a negative scenario saying old users don't appear in the list. In this case, we've just got one saying new users appear in the list. Here's where we get into the really machine-executable parts, the strict given when, then syntax. The expectations here are that the given portion, this is where you're starting from. Before we even talk about the who's new block specifically, where are we even starting from? Well, we're starting from the point of being logged in as an administrator. And we're assuming that recently, at some point in time, a new user account has been created. The when portion, this is our logged in user actually doing something. Each of these .feature files, each of these scenarios, should be describing a person doing something, a person actually interacting with your website. And that's what the when clause is for, that person actually taking an action. In this case, it's a pretty small action. All they're doing is going to the homepage, but they may be taking a more complex action. The then portion is the observable result. And observable is really the key word here. An actual administrator for this site should be able to read this scenario and understand what's happening. They should be able to say, oh yeah, I really can see new user accounts in that block. You could write these then clauses against like reading database tables directly. You could write this scenario to say, okay, then I print out all of the rows in the user table and I see that there's a new user there. Well, that's not really observable to the end users of your website. The then clause should really be about observable things. And it's important to remember that these .feature files alone are not our tests. This is a bit of a pedantic point. Again, from Constantine, the author of BeHat, these are really just our expectations. The feature files alone are not the tests. The feature files are the expectations. In order to have tests, in order to be doing testing, we need a couple other pieces. We need something like BeHat that can actually evaluate those expectations and compare them to an actual thing. So if we're using BeHat with Drupal, then the .feature files again are our expectations. The Drupal site is the actual thing and the BeHat command line program is the means of comparison. If you're doing PHP unit testing directly, then you'll probably see those functions that have an expected, an actual, and an assertion. In between, you need those three elements. An expected, an actual, and an assertion of comparison. So where are we actually running BeHat? Where do these pieces actually go? So Pantheon uses BeHat with all of our WordPress plugins. We use BeHat in a bunch of different ways. So with our WordPress plugins, we use BeHat to answer the question, does this work on Pantheon at all? We're looking here at just a screenshot of an internal dashboard we have. We have CircleCI badges and Travis badges lined up with a couple of our WordPress plugins. So we're answering two different questions here. Does this plugin work at all? And does it work on Pantheon? Before I started at Pantheon, we were already answering the question, does this plugin work at all? We, to answer that question, we have a WordPress site. It installs the given plugin, WP Redis, native session handling. We've got a bunch of different plugins. So we have WordPress running just in a Ubuntu container. Again, all we're trying to answer is does this plugin work at all? PHP unit is the normal way the WordPress community does automated tests. So we use that, that's the test that we've had written for years with these plugins, and these happen to be running inside of Travis CI. So this is how we answer abstractly, does WP Redis work at all? Does WP native sessions work at all? That's separate from the question, does this work on Pantheon? So when I started at Pantheon a year and a half ago, I didn't have a ton of familiarity with WordPress, but I was still occasionally reviewing some pull requests here, and I wanted some level of confidence. Not only does this pull request allow the plugin to continue working, but I really wanna make sure that it's not going to break anything Pantheon specific. That's kind of a different question to answer. So to answer that question, we again have a WordPress site. It's running on Pantheon. All these plugins have WordPress sites running on Pantheon. Every single time we get a pull request, Behat runs a set of tests against that WordPress site running on Pantheon and Circle CI happens to be the tool we're using to execute those tests. I wanna look more closely at those YAML configuration files. We saw that in the live demo, a YAML configuration file that really sets up how Behat executes. The main thing I wanna point out here is that in your Behat.yaml file, you're going to specify the path of the .feature files. Where are these expectations actually living? With all of these WordPress plugins, we have two sets of expectations. The expectations specific to the plugin, we wanna be able to install the Redis plugin, go to the administrative page, verify that it's actually connected to Redis. And I also wanna make sure that I'm not breaking anything in WordPress core itself. So we have a separate suite of tests that we add in with Composer. So those are living in a different path. Those are living in the vendor directory. These are expectations just general to WordPress core, the fact that we can add posts, we can add users, we can add taxonomy terms, make comments. And anytime we add one of our plugins, Redis, native session handling, we shouldn't break any of those. So we should be able to execute all of those same tests, those same tests that just verify WordPress core behavior. We should be able to execute those tests with any of our plugins installed. So there are a few terms that you'll hear often in BeHat, and even for the first year or two that I was working with BeHat, I didn't have these terms fully differentiated. So I just wanna talk about three terms you'll hear a lot with BeHat. Contexts, drivers, and extensions. First, let's take contexts. So contexts are how BeHat understands these .feature files. These .feature files again have the given, when, then syntax. How do we actually map between the phrases in those .feature files and actual PHP execution? So we do that with context files. They're just PHP classes and they use annotations here to line up between these plain text phrases like give and I am on the homepage. There's some regular expressions here so that you can vary them slightly, but basically give and I am the homepage maps really to a one line function here. Only one line of PHP is executing here, visit path. That's all that happens when you use the phrase give and I am on the homepage. The when clause, a lot of the examples in minkcontext.php are from the world of Batman, so when I select bats from user fears then this relatively simple PHP method executes. It's really just these three lines. Moving on to the then clause, then I should see who is the Batman. All we're doing here is checking for text in our HTML page. Again, it's just a one line function. These contexts don't have to be all that complicated. Now you may be thinking, Steve, there's no way it's really that simple. There's no way we're really just visiting the homepage with a single line of PHP. So contexts are one level of abstraction. They abstract between those plain text feature files and the world of PHP. Drivers are a layer deeper of abstraction. So two places you'll hear about drivers a lot are mink and Drupal drivers. First, taking the concept of mink drivers. Mink is a way to actually do browser execution. And this is an abstraction layer that allows you to use a common interface, a common set of phrases like give and I am on the homepage, and then execute those with different browser simulation tools. So if you wanna actually execute JavaScript, you're going to need something like Selenium. It can do more things, but it runs a little bit slower. Guten and BrowserKit run much faster, but they don't process JavaScript. So you can switch between these without rewriting everything. Drivers are an abstraction layer. As long as you're implementing a PHP interface, the driver is going to work. Drupal drivers, another thing you'll hear about a lot when using Behat with Drupal. These are an abstraction layer that determine how Behat actually interacts with your Drupal site. So if you're using the black box driver, then HTTP is really your only means of communicating with the site. You could probably use the black box driver against Drupal.org. I don't have any administrative access to Drupal.org, but I could probably use the black box driver and just interact over HTTP. The Drush driver moves the Drupal site a little bit closer to Behat. So we're getting a little bit closer. We can access our site over SSH. We can execute some Drush commands. This is gonna let us do some fancy things like write steps saying, give it I'm logged in as an administrator. The way that actually works is we have a Drush command that creates a new user and sets a certain password. And then with that set, you can sign in as that user. Finally, the Drupal API driver moves the Drupal site very, very close to Behat. They're literally just running on the same server. The same PHP functions can be called in either. So with that, you get a whole lot of power. You can call node save from your Behat steps. You can do anything directly on the Drupal site with the Drupal API driver. Extensions, the third term in our glossary here. When you hear about Behat extensions, you can really just think about modules to use the phrase from the Drupal community. We say there's a module for that and Behat basically there's an extension for that. So if you wanna do something like add some Behat tests specific to SEO to check meta tags or metadata on your HTML responses, well, there's an SEO extension for Behat. If you wanna do things like analyzing your tests themselves, find out how fast or slow they are, find out how many times you're calling individual steps, there are extensions for that, taking screenshots of failures, a bunch of different extensions for testing REST APIs. And of course, integration to other systems like Symphony, WordPress, and we've been talking a bit about the Drupal extension. If you have a deeper familiarity with Behat, then you can start to bend it a little more. Just as you become more familiar with Drupal, you get access to deeper and deeper layers. You get more and more comfortable doing things with Drupal that aren't available out of the box. The same dynamic is present in Behat. So I wanna talk about one way we use Behat a little more deeply. So again, taking those WordPress tests that just validate normal WordPress core behavior. In order to validate normal WordPress core behavior, we have to sign in as an administrator. And when we wrote these tests, there wasn't a robust WordPress extension the way there is with Drupal. So I had to write my own step in order to log in as an administrator with WordPress. And it's not all that fancy. All it does is use some environmental variables to set the WordPress administrative username and password. And then I can just go to the WordPress login page and login because I know what the HTML form elements are. And that works for basically all of our plugins. There's one plugin though where this doesn't work. We have a plugin for SAML, single sign-on. The whole purpose of this plugin is to override the login form. So when I tried to hook up our pre-existing Behat tests to this plugin, it didn't work because those tests were dependent on logging in to WordPress in a very specific way. To get around that problem, all I have to do is override that one context that supplies the implementation details for administrative login with a different class, specific to SAML, this SAML plugin that we have, and then slightly different form fields, basically doing the same thing. Again, using global variables here for the username and password, but accessing slightly different administrative forms. And then those same Behat.feature files can execute. Those same feature files that have expectations around adding posts, adding users, adding comments, can all execute against the SAML plugin the same way they execute against all the other ones because we've overridden just this one part. So when do you actually start using Behat? Again, to quote Michelle Crachy, you should write your tests early and often. Ideally, at the beginning of a project, you have a great conversation with all your stakeholders. Everyone gets in the same room, everyone smiles, everyone has a common understanding of what we're doing, and it's all wonderful. I don't think you can actually expect it to go like that. How do you really start to use Behat? Like, there is a lot of complexity here. I have shown you a lot of complexity in Behat. I've been using it off and on for two years, and I still feel like I'm learning it. I don't think you can jump from zero to 60 with Behat. To quote myself from Drupalcon Dublin, the agile your agile presentation. There's a lot of overlap between the agile methodologies community and the behavior driven development community that you'll see in Behat. I think it's kind of weird that these methodologies that really encourage us to work incrementally, take baby steps in the right direction. I find it odd when there's also an expectation that we're going to jump to perfect scrum all in one new project, or we're gonna jump to perfect Behat usage all in one project. I don't think that's reasonable. So again, to go back to those two main benefits that I think you can get using Behat, a clear definition of done, a clear understanding with your client of what are we even building, and automatic execution. I think when you're starting to use Behat, you have to prioritize one and bend on the other. So which do you prioritize? Dan North, the guy who coined the term behavior driven development, says the definition of done might actually be more important than the automatic testing because what good is automated testing if you built the wrong thing? What good is it if I get green checkboxes that tell me yes, my old users block is still functioning the same way it functioned yesterday? The client never asked for old users, they wanted new users. So how do you get a clear definition of done without automatic execution? Demo driven development. I don't think this term will catch on because DDD has already been claimed by domain driven development, but this is a term that we threw around informally at my last job at Palantir.net. We would talk about how every sprint we end with a demo for our clients and we show them these are the stories we built and we would just have like an informal human readable script saying first you should demo the blog archive feature and then you should demo the new stuff on the homepage and then show them the new user role and then show them the new field. So we just had an informal human readable script that said here's how you demonstrate that the sprint is done. So you can do that. I could have a human, really human friendly script saying the way I'm gonna demo that this new user block is done is well I'm just gonna sign as an admin user, gonna make a new testing account, gonna click on the homepage and then I'm gonna say to my client, look, the new account that I just made is in this block. This isn't really complex but I do think it's a good idea before you start a ticket, before you start a user story, just think about and actually write down in a place that you and your other coworkers can see how you're going to demonstrate that you're actually done with the thing. I think you can get a clear definition of done without automatic execution. If you do wanna focus on automatic execution, you may run into that pain point of wait, is testing going to make me slower? BeHat is this new thing that I have to use. It may take me an extra hour to write a BeHat test for this thing. I don't have time to write a BeHat test for every single thing I'm doing on my project. Well, I think you can start with the attitude of we're going to have a BeHat test or some kind of automated test for every single bug fix. Because by definition, a bug fix is something that has already broken. If you're concerned about writing unnecessary tests for things that aren't going to break, a bug is something that already broke. So you might as well write a test to verify that it doesn't break again. I think this is really helpful for data migrations. At my last job, I was doing a ton of Drupal 6 to Drupal 7 migration. Even in my free time, I do seven to eight migrations. As part of my job at Pantheon, I wrote a blog post in some documentation just showing that, yes, you can use migrate module on Pantheon migrating from seven to eight. So this is a pretty messy feature file, but this is a feature file that runs after a series of Drush commands do the correct steps in the correct order to migrate from seven to eight. Then all I do here is validate that by going to node slash one on my Drupal 8 site, then I see the text test node made on D7. That's the name of the node that I made on Drupal 7. I even just checked directly the exact HTML I expect for the image that I added on Drupal 7, and I even go to the exact URL for that image, and I validate that it's a 200 instead of a 404. So it's not a pretty test. I wouldn't wanna show this to an end client, but this does automatically execute every single night. This test runs. It makes a brand new Pantheon multi-dev environment. Does the correct Drush commands in the correct order migrating from Drupal 7 to Drupal 8? This behat test runs, and then I know, great. That blog post that I wrote a year ago is still relatively accurate. Great. I even caught a bug this way. On a personal project, I was migrating from Drupal 7 to Drupal 8, and often when I'm migrating, I just have the two windows open. I have my Drupal 8 site, I have my Drupal 7 site, and all I wanna do is validate that the taxonomy terms, the nodes come from one to the other. So I can just script that with behat. I can just say on my Drupal 8 site, if I go to the taxonomy listing page, I should see blog tags, I should see iTunes categories. This is a test that actually failed, because a couple months ago, there was a genuine bug in Drupal Core that broke taxonomy term migration. I probably wouldn't have caught that without this test, but because this test failed, I was able to find the appropriate issue on Drupal.org, help review a patch, and get an actual bug fixed in Drupal Core. So if you're looking to experiment with behat, in a way that's not going to require you to be working too closely with your client, if you just wanna do a developer-focused introduction to behat, I recommend using it with migrations. Behat is not a performance testing tool, but I kind of used it as one. Last summer, when we were comparing PHP 5 against PHP 7 on Pantheon, we just wanted to show clearly, especially for authenticated traffic, PHP 7 is going to be genuinely faster. So I took a real Drupal 8 site being built by a major agency on Pantheon, and I asked them, hey, can I just script up some basic administrative behavior with behat? Basically, I'm gonna make a node, I'm gonna go to a workbench moderation listing page, and just click around a bunch, and I'm gonna do that in PHP 5 and PHP 7, and the result was pretty clear. It showed me that the exact same pattern of traffic in PHP 7 is faster than the exact same pattern of traffic in PHP 5. Now, again, behat really isn't a performance testing tool. If you're comfortable with other performance testing tools, you can script up this kind of thing more cleanly. But if you're just looking to experiment with behat, this might be helpful. We even use behat for testing command line tools. Here's a .feature file for evaluating the behavior of the terminus command that enables new relic on a site. So it's a pretty simple command. I just wanna point out this portion, the command is terminus new relic colon enable, and then you give it the site name, and we just wanna validate that when you run that command, you're going to get back from terminus that new relic has been enabled and that our environments have been reconfigured. So again, if you're looking for a more developer-oriented way of experimenting with behat, you can use it on command line tools that you might be writing. Another detail I wanna point out here is this at VCR annotation. This is telling us which predetermined HTTP interaction is going to be referenced. It would be pretty slow to run our full suite of terminus behat tests using real HTTP against the Pantheon API. So we have those pre-baked with this VCR plugin that's just going to replay the same HTTP interaction. I'm also excited because there's a new extension for WordPress. For a couple years there have been a few WordPress extensions that got started and then didn't really go anywhere. We've got a new one in the community that's really modeled after the extension that's present in the Drupal community. I think in the coming months or a year or so, we'll be looking at refactoring our WordPress tests around a much more robust behat tool in WordPress. So what have I learned with all my behat work? Bending behats benefits, again, be willing to bend. I think if you're trying to get both of those main benefits of a clear definition of done and automatic execution, you're probably going to have to sacrifice on one in order to get the other at first. Behat, behavior driven development in general, really is about the communication that you have with your team, the communication that you have with your client. So, find teammates to work with. Prioritize tests that will fail. It sounds odd, of course, we want our tests to pass, but if they never fail, I sometimes wonder why did I write that test at all? And ideally you want them to fail meaningfully. You want them to fail in a way that gives you meaningful information. Some of our WordPress tests just fail because WordPress has changed HTML implementation details, and it's not so much a meaningful fail, it's just that somewhere we've assumed a certain HTML structure, and then sometimes that breaks. I think when you're working with a new tool like behat, you're going to have to practice. It can be time consuming to learn these new tools. You may not have time on a billable client project to learn something brand new if you're new to behat. I also think it's fun to get creative. So, I do want to take this opportunity to introduce a new behat extension. It's a behat extension that I have released under my joke GitHub account for Foe Al Gore. This is the behat say extension. This is a extension that uses the say command that's present on most Unix systems to say out loud your behat tests as they run. So, I'm going to switch Git branches here. I'll do the behat. Oh, I need a T. It's another risky live demo. And this. Given I fill in edit mail with testing at example.com. Given I fill in edit name with test account one. Given I fill in edit pass pass one with some random pass. Given I press create new account. When I visit, then I should see test account one in the block views block who has new block one element. And I should see the user name for the new account in the who's who block. So, why would you actually use this? This again is a joke or novelty behat extension. I kind of miss the culture of novelty or joke modules that we had in Drupal, in Drupal six, and Drupal seven, the bad judgment module, the misery module. I want to introduce some joke behat extensions. And yes, this isn't meant for normal usage, but I think it does make two helpful points. It makes the point that you can write ridiculous behat steps that really aren't friendly to your clients. If you hear these steps spoken aloud, if you hear spoken aloud, block views block, who's new block one, that's a sign that this is a ridiculous test. This is not something that I would want to show to my end client. I would much rather show them this scenario. Another thing I think this helps highlight is the concept of user roles. I found that one problematic pattern I get into with writing behat tests is it's really tempting to just write everything around the administrative user because the administrative user can do anything. You're not gonna run into a permissions error, but the whole point here is thinking about who is actually doing these things. So if I set different voices for these different user accounts, if I set Fiona, oh wait, that's the local, if I set Fiona to be the administrator in cellos to be the moderator or any of the voices that you can get from the say command, I can reinforce myself that these are different people. These different people have different expectations. So again, this is actually out on the internet. If you do want to add this to your project, you can do so. It is registered on Packagist, just composer require foal gore slash behat say extension and you'll be good to go. Thanks everyone. Of course, later this week we've got contributor sprints and I think we've got a few more minutes here for questions. Yes, please use the microphone at the center. After behat fills out a form, it just, everything disappears, just swipes everything. Is there any way to save that form so you can go in and look and see all your fields filled out? Interesting. I think, I wanna make sure I understand the question. Are you saying is it possible to see visually the form filled out before it's submitted? Would that be like a break point before it hits save and then you can go and look? Well, it's important to remember that the browser simulation is not necessarily using your, probably not using your own browser. It's using a simulated browser, but I'm not certain what the screenshotting tools would use, like you can take screenshots at the end result. I'm not sure if you took a screenshot midway if it would show the form fields being filled up, but it might. Anyone in the room know if you took a screenshot before pressing save what would happen? I haven't gotten that far. Great, yeah, so we got an answer from the room here. Browser stack will let you do that. Okay, one more, one more. Yeah. In that one slide you had where you mentioned it looks at the taxonomy vocabularies available and there's four listed. Is there any way to say, and I should not see any other taxonomy terms in case somebody created one and you didn't want another? That's all you wanted. Yeah, you would probably need to write a custom step there to say something like, and there might be a pre-written step for something as abstract as like, and there are X rows in the table, that might be a pre-written step, but I would guess you would probably have to write your own step to count the exact number of rows in the table. Okay, thank you. Hi, first, thanks, I really enjoyed your presentation. Oh, thank you. I'm having essentially the Drupal 5 problem, which is to say Drupal 5 does not work until somebody tells you when you're a newbie you must download CCK and views. Sure. So it sounds like context drivers and extensions are crucial to having a kind experience with Behat. Is there a resource for where to go to get the crucial specific ones that we're gonna use for 80% of our Drupal things? Sure, I'll take this. And what they mean? I'll take this opportunity to plug a Pantheon repository. This is a repository that's an example of how you can use Composer Drupal 8 Pantheon all together with GitHub. And because this is a continuous integration example, we wanna show some tests. So we have a Behat-Pantheon.yml file here that is using, in this case just two contexts, the Drupal context and the Mink context. So that will get you pretty far, pretty fast if you can start from this example. And to just further the plug, if you stop behind the Pantheon booth demo, you'll see it in action. It's a different example repository but it shows the Behat step actually running. Yeah, thanks. Hi, thanks for the presentation. Sure. So the question is, is it worth doing schedule monitoring, writing Behat scripts? Interesting, so you're asking would it be worthwhile to execute a Behat test on a schedule to verify that your site is continuing to run? Right. Maybe. And the reason I hesitate is because I'm hesitant about executing Behat tests against a live site because some of your Behat scenarios might do destructive things. They might delete real content or they might add content that you don't want to. It can affect on my analytics as well. Yeah, so if your Behat tests, if your Behat tests don't do anything observable, but the point of Behat tests or part of the point is to do things that are observable then it may be safe to run them against a live site. So technically, yes, you absolutely can execute Behat tests against a live site but it may not be a good idea to do so depending on the specific tests that you have. Well, you can do it on different environments like at stage, pretty probably. Absolutely, yeah. And so on, yeah. Yeah. Just curious, thank you. Hi, I'm writing Behat tests for a site, a Drupal 7 site that has a lot of content sort as entities and the Drupal extension seems to mainly help you with nodes. Yes. So where's a good place to go for templates for customizing, creating that custom logic to support creating a Drupal extension? So there's the concept of these contexts, the things that actually define your steps. So yeah, it would be great if the Drupal extension just had generic entity handling rather than node specific or user specific handling. So I guess the Drupal community correct answer is open an issue on GitHub, engage with the maintainer and see if you can get that added to the whole extension and then everyone can benefit from it if you're just looking to add custom functionality to your own project. The way I've done so for this demonstration specifically is I've used the concept of sub contexts. Let me make this bigger here. Basically it's again a simple PHP class that's extending this Drupal sub context base and then I've got some, well, here is where I did all the messy work of adding those breakpoints and a few other things that made the demo more demo friendly. But basically you might find it's easiest to use a sub context like this to add additional Drupal functionality specific to your site. Sure. Hi. How would you separate the tasks between the custom modules and your Drupal site? It may be difficult to draw a clear line between what behavior is expected only of the contrived modules and what behavior is expected of your site only and the reason I say that is to go back to those different types of testing, unit testing, integration testing and system testing. From the perspective of system testing, you don't know exactly where the boundary is between workbench moderation module and your add-on customization of workbench moderation module. With system testing you only have the system in front of you. It may not be clear exactly where the boundaries are. If what we're talking about is an individual client site and it could be helpful conceptually to have a directory of feature files that are just validating what you think is the core behavior or the contrived module behavior separate from behavior that you've really customized. That's something you could do for your own organization but on a technical level, Behead is just seeing a Drupal site. It doesn't necessarily know where the boundaries are between what the behavior was and when the contrived module was enabled and what the behavior is because you've done some customization. Thank you, that makes sense. Sure. So you mentioned Behead running on a production-like environment. Yes. But also doing things where it's like adding users and content and things. Ideally you would want a staging environment where you have your latest live content. Maybe you run that, if you have a large database, gigs and gigs, that could be a heavy operation. Absolutely. You run it every time testing. Is there an easy way short of restoring a backup and doing a reset to kind of get rid of cleanup Behead's mess so you don't have to do a full import? So there are a number of different strategies you can implement there. I've talked to some agencies that using Pantheon have an automated script that on a nightly basis copies the database from live down to a multi-dev that exists only to have a smaller database. Like they copy the full database from live to the multi-dev and then they run a script that just like drops a ton of nodes, thereby greatly reducing the size of the database. And then they, and their automated tests can grab that database. Not the full live database, the trimmed down database from a multi-dev. So you could do that. Another way you could think about it is the given step is where you should do any setup that you need. So if you're writing a scenario that depends on the fact that there are 10 nodes on the home page, then you probably have a step like given there are 10 nodes on the home page. And then you can fulfill that expectation however you want in PHP. You might use like DrushDevel generate to generate 10 new nodes. You might just go through all the forms directly in HTML and make 10 new nodes that way. So I guess like the correct answer from the BeHat perspective is the given step is where you do whatever setup you want in whatever way you want. I think in practice to make it fast, if you want a production like database that's still smaller than your gigs and gigs real production database, it's probably best to just have like a .sql file sitting somewhere that you can grab and quickly throw into your production like site. Yeah, so I have a question about UI testing. So if it makes sense to use BeHat with UI testing over Selenium or something like that, how would you set up a specific test for different breakpoints? So let's just say you wanted to test on Chrome but at two different screen sizes, how would you organize that? Yeah, I think that may be a line item in this table for different mink contexts of the understanding of breakpoints. Window resizing here. So the very, very simple Goot doesn't have the concept of window resizing but Selenium does. So I think that would be the means through which you conceptualize of different device breakpoints. All right, well, thank you everyone. All right.