 Welcome to Testing Business Critical Features with Behat. Thanks for coming back after lunch. I appreciate you being here and in on time. My name is Andrew Taylor. I'm a Developer Programs Engineer at Pantheon. So I consult with our agency partners on complex projects and workflows and a lot of automated testing. So happy to be able to share some of that with you here today. I'm A. Taylor Emmy on GitHub and Twitter. I tweeted out a link to the slides. So if you want to follow along or download them for later, there's some resources and stuff and things linked throughout. You can go ahead and grab those. I'm from Portland, Oregon. I love being outdoors and just away from the computer after being in front of it all day long for work. In traveling, good food, good beer, I'm going to check out, I think, Elvez tonight. I heard that's pretty good in Philly. If people know other spots, let me know. I'm here until Monday, so I'd love to check some stuff out. But more than all of these things, I am a maintainer of websites. And really, that's kind of what all of us are. That's why we're here. Whether it's personal or professional projects, I actually have my dad needed a website for his business and he goes, can you refer me to an agency knowing full well that I'm going to be like, no, I'm just going to build it for you, right? That's what happens. So we all maintain websites. And sometimes they're to help friends and family out. And mostly, hopefully, sometimes, we also get paid for that work, right? And they're for clients. And so we're maintaining websites that maybe if my dad's site for his company goes down, it's not the end of the world. I built it for free. If I get it back up within a day, it's not that big of a deal. But when we're doing client work and we're maintaining websites that we're getting paid to do, it really is more important that we do a good job and we really maintain those to a high standard. And when we're working on these projects, especially client projects, changes happen frequently. So we're going to get client requests. We have to run updates all the time, right? Like WordPress updates, plugin updates. We're fixing bugs, developing new features. All this work is happening. But how do you decide kind of what's business critical and what do we really need to make sure we're taking care of and maintaining well and what is maybe okay that if a bug gets out there? And for me, business critical is whatever you're gonna get that phone call at 2 a.m. if it goes down. So that could be a donation form for a nonprofit. If they stop taking donations, they're gonna call you. If there's a little bug in the menu, they might send you an email and say like, hey, can you fix this like broken icon or something? That's not a big deal. That's not business critical. A shopping cart for e-commerce, most definitely business critical. If people can add stuff from their cart and they've read reviews and everything's great and then they go to check out and it doesn't work, bummer. So we all know what these business critical things are for our projects. You need to define them. And they're really an important part of your QA. They're things that need to be checked, right? We probably have a QA checklist or something that don't forget to check this donation form. This is what I'm gonna use as an example throughout the talk here, right? Don't forget to check the donation form during our QA because that's really important. They're things that just can't break in production. That's not acceptable. That's the type of thing you're gonna get that phone call and you know what? It's not gonna be a pleasant conversation and they might even fire you, right? Or a lawsuit or who knows what? So business critical testing is more than I see people when they do automated testing, and they do kind of some cool stuff with visual testing or I'm gonna run a performance test and I'm gonna make sure we bake that into our QA process. Well, a visual test and a performance test aren't gonna tell you if the card's broken or if someone can't submit the donation form, right? Or if the payment gateway's down, whatever it is. You need to actually test against a full copy of the site and ideally a full copy that's like a clone of production. So we're not just testing on, oh, I have something locally that has a few dummy posts. No, you need to test the real site and make sure these things are working. So who uses a staging server, right? It's part of your QA, hopefully everyone. We know we shouldn't be cowboy coding and running plug-in updates, right, on the live site. When I'm working with agencies, a lot of what I see is this kind of staging server workflow where maybe we get a bunch of client requests, we have a new feature, we gotta update some plug-ins, somebody else is working on a bug fix, we made all of these changes and they're up on the staging server and now we're gonna have this round of review where our team does internal QA, we're gonna send the staging server to the client, they're gonna hopefully sign off on everything and after that we're gonna roll it out to production. And so this is, hopefully everything works, like all your internal you go through, make sure the checkout works, the donation form, whatever it is, you're doing all your QA, you're doing your browser checks, the client says things look good and it goes out to production. And so we might have a list, right? If you're running through QA, you have written this stuff down, we need to go to the donation form, I need to fill out the info, I need to submit the form, I need to verify that I get a confirmation, I need to log in to WordPress and verify that a new donation post is created and that the status is complete and that it actually processed. Maybe I need to verify that the correct user role can log into the back end and manage this form in the admin that they can update the donation amounts and that's gonna show up on the front end. But QA doesn't always go well, right? Like sometimes it's a big hot mess and you get to that point where you have all these bug fixes or plugin updates and all of this work and now you have to find and fix the issue when maybe you have changes from multiple developers or multiple pieces of work all on the staging server and you have to kind of untangle this mess. You have to find the issue, you have to fix the issue and it's kind of holding up all of this other stuff from launching. If there's like an issue with the donation form, I can't roll out the security update. Well, what if the site gets hacked in that time, right? So it's really not an ideal situation and when I see a lot of people working with a staging server, they kind of have that we have this every two weeks sprint or we're gonna have everyone push up their changes and go through this process and it really is stressful for not just you but for the client, right? Because you're probably out of time. By time you get to that cycle where you have plugin updates, bug fixes, new templates, everything's up there and you're doing this QA, it's kind of like, we hope it goes right, right? You're crossing your fingers and you're at the end of the deadline, like you need to get those security updates out or this feature needs to launch, you don't have time to go back and untangle this mess. There's gotta be a better way, right? So ideally, you wanna test whatever these critical features are every single time you make a change. So I run plugin updates, I test the donation form. We saw before, like actually writing down those steps and making sure you do that. I make a fix to the menu that the client reported. I check the donation form. I develop a new template. I go in and check the donation form and you do that every single time because if you check as you go, I kind of think it's like cleaning your house, right? You let the dirt go for a month and then you're gonna be on your hands and knees scrubbing the floors but if you kind of keep up on things, it's not that bad. You can kind of make sure that things are working as they should as you go. And so you get to more of this consistent testing workflow where you're building that new feature and then you're testing it and you're running through your list of QA and then you go in and you test the next thing and maybe you have multiple staging servers where you're testing these things and then you're bringing them all together, doing a final run of QA and getting clients sign off but that process is much smoother and less stressful because you've been testing as you go. So when you do that final QA and you're waiting for clients sign off, you're not like crossing your fingers that nothing's gonna blow up because you've been doing all this QA and testing along the way. But the problem is that's kind of boring to do, right? If I'm doing a menu fix or developing a new template, I wanna have fun with like some animations, some new CSS3 stuff. I'm like developing this new template, this landing page looks great or whatever. I don't wanna go check the donation form, right? I just did this little bug fix that took me five minutes. Why am I gonna spend 10 or 15 minutes running through this QA checklist? I have other stuff to move on to. I need to go do the next task. It's really time consuming to sit there and do that every single time you make a change. And if we're being honest with ourselves, you could have this big QA checklist and give it to developers on your team and say, you need to run through this every time you commit a line of code. Are they really gonna do it? Like if we're being honest, it probably doesn't happen because of these reasons, right? It's not something we wanna do. I really like the difference between checking and testing. So checking is like, I'm gonna check the work that I do and make sure that's okay. And then testing like your full QA, nobody wants to sit there and just test all the time.