 Okay, thank you for joining us. If you're in Tahoma too, you are in the right place. You are here to see Andrew Taylor with the benefits of testing and automation. Andrew works as a developer programs engineer at Pantheon, consulting with their agency partners on complex workflows of automation. In his presentation today, he was able to share the benefits you can see as a result of teams adopting automation and encouraging you to automate parts of your workflow. Thank you for coming. So, yeah, 10.30, right on time, I love it. So yeah, talking about benefits of testing and automation, A. Taylor and me on GitHub and Twitter. I tweeted out a link to the slides, so if you want to follow along, grab the resources at the end, please do that. I came up here in Portland, so I love the Northwest, being outdoors, hiking, mountain biking, and all these sorts of things. But more than working at Pantheon and being a human who likes to be outside, I am a maintainer of websites, as most of us are here at WorkCamp Seattle. And so that means that, you know, I'm building websites in my day job, but also my dad's company needs a website. So can you refer to an agency, knowing darn well that I'm in the building myself, and not going to go find an agency, right? So these are sorts of things that happen. We pick up these projects, and we have to maintain all of them over time, whether they're personal projects or client projects. So I want to talk today about testing and automation and really look at ways that we can improve our maintaining websites and improve our workflows. And changes, especially if it's a client-active project, right? Happen frequently. We get requests from the client. That might be changes template. We have security updates that have to be read all the time. Big WordPress 50 update coming. I bet every single plugin you have installed is going to have an update that goes along with that. Bug fixes, new features, we're developing, all these sorts of things are coming in. And so who uses a staging server? So yeah, hopefully we all do. We shouldn't be cowboy coding in production anymore. Because we don't want to break the life-site, especially if it's a client-site where getting into that work can't go down. And usually the workflow I see when I interact with folks who are using a staging server is that they have this new feature, this bug fix, all of these things, and people are maybe doing it locally, maybe they're working right on the staging server, but everyone pushes their changes up, and then they do around the QA, right? So you have this bug fix, this new feature, all these plugin updates, and they're all on the staging server at the same time, and now we have to do this QA. And, you know, people are a little bit more advanced, maybe using version control. We do that work on different get branches. So maybe developer A and developer B, our new feature and our plugin updates are on different branches. But again, they both kind of look good on my local machine or wherever I'm doing the work, and then I push it up to the staging server. And so we have this spot where we're doing this large amount of QA with multiple changes coming into play. And so just thinking about the scenario where we're updating plugins and things, right? So this is one sort of change that maybe that QA looks something like this that I'm going to go take a look at the live site. I'm going to go take a look at the staging site. If I'm feeling fancy, I'll load them up side by side. Maybe I'll go check mobile, because mobile is really important, right? And we want to make sure that that's looking good. So who noticed what was different? There actually was a change there when I ran the updates between the staging site and the live site. And it's the search box. More specifically, we can see the live sites on the right, the staging sites on the left. And we can see that the label search got added in addition to the icon, which is not a big deal. But you might not be able to notice that the actual search button is a little bit off. It's kind of running into the border on the bottom there. And as a human, just looking at these things side by side, this is something I missed. I didn't notice that. Maybe you have an OCD client who does notice that it's running on the border, right? So, and that's just the homepage of one site. And now we have to repeat this for all the different templates we need to check. Maybe a single post, maybe an archive page, maybe the contact page, whatever. And we're doing this over and over. Every time we run updates. So this is just one type of QA. And if we have the staging site, we're not just plugging updates, but the new template, the bug fix, the client request, all these things are stuffed on the staging server. We're doing this QA, it doesn't always go well, right? We catch things. And then we have to find and fix these things. And we're probably out of time because at this point, the client wants their request to go live and they want the bug fix. We're at the end of a sprint where we need to get the stuff out there. But now we have to untangle and figure out where was the issue. Maybe it's holding up all of these other changes. There's got to be a better way, right? So this is what I usually see. People have a staging site. It's great that we're using that. We didn't used to and that's sort of a standard of best practice there. But there's a better way than shoving everything on the staging site doing this big stressful QA that's stressful for you. It's also stressful for your client because they have those deadlines. And so you can test after every change. And this is the idea of continuous testing. So we looked at, you know, staging server that maybe if you're doing a more advanced work quality working and get branches or things for each change. But you actually need to test at each point after every change. So I run plugin updates. I test things. I fix a bug. I test things. I, you know, develop a new feature or new templates or whatever. I test things. And then when we get to the point where we're doing the big QA before we go to production it's a lot less stressful. It's a lot less time consuming because if there were issues like with that search bar after plugin updates I would have tied it back here and then I can go in and actually, alright, maybe I created a backup before I ran all those updates. I tested. There were issues. I can restore the backup. I can update the plugins one at a time. I can find where the issue was rather than getting all the way to the end and trying to untangle that less later. And so this continuous testing sounds great in theory but checking things is boring. I actually talking with other developers. There's a difference between checking and testing. And if I'm fixing a bug with the menu, I don't mind testing that the menu works correctly. But going and checking everything else on the website making sure that every template looks okay, that you have to go check Internet Explorer or going in and, you know looking at all different templates or filling out contact form adding products to the cart. So if I have this quick fix that takes me like five minutes for the menu going in to check every other part of the website is too much to ask. But we should be doing if we want to catch issues early and fix them before it gets to be a tangled mess. And it's also really time consuming. A lot of people I talk to don't do this in practice. They know they should be doing the more theory QA more often but it's so time consuming that it's just not practical. Because you need to actually be doing the work. Not just checking the work. Your clients it's hard to send them a line item for, you know, five hours of QA when you also have five hours of they're like, wait, what? I'm paying you to work. So, can we test more often without doing more work? So I started asking this question. And the answer is to make the robots do the work. If they can actually serve lunch. I think this is in Japan. Then, you know, they're going to be driving our cars and all of these things. Surely they can help us with our QA. And there are lots of great tools out there. Tons of tools that exist to help with this. Maybe some of you have used WPCLI, right? To automate parts of your workflow. Go in and I want to just check if updates are available or I want to install a plugin. Well if there's tools like this where we can do these things in the command line, then we can start writing scripts and we can start building up towards this automation. It is an up-front time investment to learn these tools and write these scripts and get this process going. But it results in this consistent testing practice that is going to have long-term benefits and we're going to get to those benefits. And this is actually one of my favorite comics, right? It kind of helps me get in my mindset of what developers want to do. Pass the salt 20 minutes later. I'm building something that will make it so I never have to pass salt to you again, right? And that's what we're going to be doing with our QA is looking at how can we put the time in up-front to make sure we never have to worry about this. And it is the road plus travel. So most people are still doing things manually. We know that this automation exists and you might go and look at some sites that are really large-scale like Amazon, right? They're deploying changes so often with such a big team that they can't do it manually. They have to have automation. But for us, where the projects we're working on may be on that scale, but we can still take some of those best practices and adopt it. So it's kind of hit at that enterprise tier. And so it's a bit of the road less travel because it's kind of this upper echelon but there's no reason that the rest of us can't take advantage of these things. So looking at our plugin updates again I want to talk about visual regression testing. And this is what I recommend most people use to kind of cut their teeth on automated testing. So visual regression testing is really neat. It will go in, use a headless browser. If you didn't know this, Chrome actually has a headless mode. So I'm running Chrome right now, but it can actually view a web page and not display it on the screen. It's just parsing the HTML, right? So it can go in, do that and rather than display it on the screen, it can save a screen shot. And then it can go in and check the live site, take screen shots of that, take screen shots of your staging site. And this is the things we're doing manually pulling these up and comparing them side by side and figuring out the difference. There's no reason that machine can't do this. And so there's a tool called Backstop.js that helps you automate this process. And so actually uses Backstop.js and we can just run it. So I put it in a list of a couple of sites. I picked the one I want to run test for. And I already have the URL of my live site and my staging site saved. And it is actually opening instances of Chrome. Looking at the pages I've defined, different viewports I've defined for desktop mobile. So this process that we used to do manually and even if it only took me 10 minutes 10 minutes of work after every single change I made is still a big verdict. But now if I can just pop open my terminal and run the script and it took less than a minute, generates this nice report, actually go in and click it's showing me the difference and here's where it found I'm not spending even the mental energy to do that anymore. All I'm doing is reviewing the report and going look okay before, after looks like that might be a little bit off. I need to go treat some CSS. I know that it's on this conflict in this spot. So rather than wasting my time and energy trying to find the needle in the A-stat that may or may not be there let the tool do that. I just do the analysis and make the decision of is this acceptable or not. And maybe it is. Maybe it finds a difference but it's an acceptable one and then you deploy the update anyway. And so that does make it feasible for us to get to that point where we're testing after every change because I can run updates or I can fix the menu and if I don't have to do all that other stuff manually it's going to take a good chunk of time. If I can run a script and kind of automate that process then we can do it on every change. Catch the bugs early and fix them before we get this giant staging server with 20 things in it and we don't know where the issue is. So let's kind of keep going down this path. We use version control, hopefully using version control on your projects. GitHub, GitHub, Bitbucket tons of them out there if you're submitting items to WordPress.org or working with WordPress core unfortunately you're still using SVN but whatever it is hopefully you're using version control. There are actually continuous integration providers that will help you map these scripts and this automation to your version control which is really great. So Bitbucket has their pipeline and GitLab has their own. You can use circles here. It doesn't matter which one you use. Go check them out and figure out which one's a good fit for you. But they're all pretty similar in what they do. So here was my ideal workload that I thought of in my head and I was actually able to use these tools they have a configuration file that you define these are the things I want to do this is the order I want to do them in and you write those scripts and it takes care of actually writing them. So we can do things like build production assets if you're writing SAS and that needs to be compiled to CSS you might be using Rector, Gulp or something locally instead of doing that on your own now that can be offloaded. You just send your SAS files to GitHub CI server runs you know give it the command compile your SAS files and it runs it automatically it can handle deployments so you can send your files to your staging server with Git, SSH SFTP whatever that is you can automate that and this is actually a screenshot here from one of my GitHub repos where then I call back to the GitHub API and post a link to where it's been shipped so now when I open a pull request I don't have to even log in to the Pantheon dashboard and spin up a new environment it's happening automatically so taking these things that we're spending time on and automating them so that every PR comes in this pull request I request to change the color of the sidebar so it detected that's where there was a visual difference that's fine visual aggression does have its limits though so if you're working with dynamic content maybe banner that rotates and add that changes on every page there are ways to ignore those but then you're not testing against them and if your ads aren't working that probably is an issue so it still saves you time if you have to go template to template it will catch I think of it as like a big net it's going to catch most things but some stuff's going to slip through but you can add more tests on top of that so there's tons of other tests you can run browser tests, unit tests performance tests, tons of things basically you want to add enough automated tests that you're comfortable when they all pass we're not going to get that phone call at 2am whatever think of those mission critical things so for a non-profit that might be their donation form what is that thing that's most important or we talked about ads so have visual aggression as a nice baseline but then figure out what's business critical that site and test those things for an e-commerce it's probably that checkout works and the cool test is not going to tell you if the shopping cart's broken so you do have to add a few more tests on top of that but visual aggression gives us a nice baseline that if something is fundamentally wrong with the site it's probably going to show up in a visual way and then we can test for other things that might not show up in that way and then I looked at other things I was doing manually like examining performance and I would only do this when there were performance issues and then I looked at the bug well there's a tool called Lighthouse you may or may not be familiar with it from Google it's built into Chrome but now also have a standalone package you can run and it will audit your site and give you scores for things like accessibility SEO it also does one for performance so instead of me manually opening the URL going into my dev console looking at this network trace that's not feasible for me to do after every change maybe it's part of that QA we're doing like before we go live but again if we catch issues there it's too late so now I can just run this Lighthouse tool and it will tell me a performance score so start thinking of ways you can wrap that in your workflow in my case it's when I open a pull request I'm going to run the performance report against the live site and the code that's in the pull request and look at the difference and if the performance score decreases more than 5 I fail the test because I want to make sure that any code I'm introducing doesn't make the site less performant and this is what the continuous integration kind of UI looks like then we'll show you a roll up view you can go and click on individual jobs but I have things like I'm going to build the site I'm going to run code sniffing I'm going to do my visual tests and Lighthouse tests I can define all these things you still are going to have to put into work to write those tests and write the scripts but once you set it up then it's always there and tests that pass allow you to merge the PR and tests that fail will actually block a pull request and I don't know about you but it might be testing that maybe you have some internal QA checklist that says go check these templates go run a performance test go do whatever what happens when you're really busy and you bring in a contractor is the code they're submitting going to go through that same QA process we all work in open source software I work on a lot of open source projects as a contributor to somebody else's project for the first time I probably don't know why I should be testing on that project but I love when I open a pull request on a project and I see something like this and all the check marks are green I know as a contributor that the code I'm sending is not introducing errors they know as a project maintainer that the code they're accepting is not introducing errors and nobody had to go do the work to figure it out because we've made the robots do that work and it happens on every single PR so you get that consistency and so it makes it feasible to get to that point where we're doing continuous testing with every change because now if we open a pull request and that pull suite of tests run automatically we don't have to spend dev time on it and we're going to catch those bugs and those issues earlier we're going to be able to resolve that before we get to that point of hey we're going to do final QA on a staging server before launch it takes all the stress and all the worry out of that because every piece of code that's gone it has already passed this list of tests one final run to make sure things are good before you go live and so I've helped a lot of agencies and developers kind of improve their workflow with automation and I want to talk about some of those benefits so definitely reduce overhead is the biggest one if you're spending human hours doing something and you can not spend human hours doing that you can reallocate those two things that are more valuable our time is better spent actually writing the code than running browser tests or performance tests or whatever you get consistency it might be that you have a senior developer on your team and they're really proficient with opening up dev tools and running a network stack trace and figuring out where the performance issues are we might have a junior dev who's not as comfortable doing that and your QA list that everyone runs through says performance test theirs might not be as thorough but if you actually automate that test and make it happen on every time regardless of who makes the full request then your QA process is consistent and it's not oh I forgot to check this template or I didn't test this as thoroughly as I should have because I'm running out of time everything is happening on every single change and that mitigates a lot of risk if we're testing early and we're testing often and we're doing this more full stack QA on every single code change then there is less risk that that final QA process is going to turn up issues and that gives confidence to you to your clients to everyone working on the project and involved in the project and you get reliable communication if we think back to that full request that has stats on this is the performance not just the person if you're doing QA manually maybe I document that somewhere maybe I don't if we have an internal checklist and someone is doing a review they go in it passes they merge the thing did anyone write down what that score was whether you know so have that reliable communication and things are documented because we're actually having that happen in an automated fashion so now I want to go back to our update process and think a little deeper about that and how can we take some of this automation because doing it on full request is great we're going to catch bugs earlier we're going to catch up more often and a lot of people stop there and I wanted to push it a little further and think about how else can we utilize this and solve like WordPress updates is something we all spend a lot of time doing and honestly I have demo sites and other sites if I'm here speaking at WordCamp I'm not logging into my sites to check for updates today maybe there's a security update that probably should go out today so how can I better handle that situation and so our WordPress update steps is first thing check right are there even updates available and if there's not then you know we can just let the client know hey you're on a maintenance plan we're checking for updates just a reminder if there are updates available then we use a version control create a new environment apply those updates we're going to do our visual check and then we're going to test those critical items like the donation form and shopping cart or whatever and so these are things that I log in and I have to actually go do these things and then if I find issues after I run updates like our search icon there then I have to go in and fix those issues but if everything looks good when I'm going to do my QA and merge the code and I can actually deploy those updates my colleagues in Slack said hey I updated the site and then this red box this is kind of where you run into dread and some issues is now I have to repeat that for all sites let's say I have 5 that doesn't seem too bad I have to log into 5 sites and do this stuff everyday what happens if you manage and it doesn't it does not scale very well and thinking about WordPress updates modern software actually auto updates and this was interesting to me because if you use Dropbox or Chrome or Slack or whatever I don't know what version of Chrome I think it just kind of updates and things are good and that's great for security and functionality you're always getting the latest creative stuff keeping it secure WordPress has automated updates in core but there's some issues with that your site has to be writeable in production and it's just going to update there's no QA process so even though minor updates are not super risky for breaking things there could be issues but the bigger thing for me is plugins and themes are not updated which WordPress updates don't happen as often as plugins and themes it seems like every time I log into my site there's a plugin or a theme update and so we need to update those updates but we also need to test them we can't just click update all and cross our fingers in production that's the whole reason we have staging servers and we're testing things is we don't want to break the production site so we kind of need the best of both worlds is we need automatic updates but we also need to apply the automatic testing to make sure that those updates went well so if we look at this process this thing we are doing manually in the environment applying updates, doing a visual check well we've kind of seen that these tools exist for a lot of these things if I want to check for updates I can use WBCLI and it will tell me if I have updates I can use WBCLI to run the updates the visual check we just saw that with the visual regression testing tool we can automate that there are other tests you can add for those business critical pieces and so if we get to the point where we need to do the steps now repeat for all sites is not that bad because automation scales guess what? instead of me logging into each site and doing this process manually it might take me hours if I totally automate it and let the robots handle this work they can test every single site in my portfolio at the same time it can scale out horizontally so whether I have 5 sites it doesn't matter to me because they're all being tested at the same time we define these steps and then it will just run them on whatever site we're telling it to so actually I have this running in production for over a year the steps I just outlined automated then I post the results back to Slack and we can actually go look here's my Slack channel this is on a cron running for 4 hours it goes through the sites that are responsible for maintaining applies updates if they're available runs all the tests and here's one that failed so I just get a link to the report and now my job is not going in and checking for updates and trying to hunt down issues my job is to go in and if the test do find something to go look at it and so this is changed that the spacing around this gallery and these images changed and this was probably a Gutenberg update I run a Gutenberg on the site to test out the plugin they're really coming out with a release about every 2 weeks so now all I have to do is make the decision about whether this spacing change is acceptable or not if it is I just log in and click set updates and it moves on to deploying them if it's not okay then I can go in and update the CSS and get that spacing back to where it was previously and then I just log in and click the button and tell it to continue and it will deploy with my fixed CSS and if all the tests pass I just get this nice green message and it says hey I'm going to deploy for the site FYI and so the hours and hours and hours spend doing other things more valuable work consulting with our clients building demos all these things that Pandion is really paying me to do that work not to log into a bunch of WordPress sites that's not the most valuable thing I can do and so to summarize adopting automation we need to test on every change to really catch things and not have that stressful QA period it's not feasible to do that it just doesn't scale but we can automate those things that automation does scale it's upfront work to do that but you're enforcing those rules and you're going to get that consistent testing every single time and I really do believe that the long-term benefits are going to outweigh the learning curve and the setup you're going to have to do to get this going because once it's up and running you kind of set it and forget it all of these things we talked about just to recap of the benefits the biggest one for me is that people on teams who have adopted automation are happier nobody's happy manually opening up DevTools and auditing perform but if you have to do that times 30 sites like every other day no one wants to do that so offloading that mundane kind of repetitive work developers allows them to do the work that they really want to be doing and having a happier, healthier team with a better workflow I think makes it all worth it and I do kind of want to end with automation is a journey so it took me I've had this running for over a year but it also took me over a year to build this kind of automated update process and so I started with just writing a script that had WPCI commands that were updates and applied and then I did the QA and then I learned about visual regression well maybe I can automate that visual testing and things didn't always go well I had it set up and I was like sweet visual test pass I can shift this well it turns out the performance went down like 20% on site and I shifted that because I thought I automated all my QA well no I didn't I automated the visual piece it takes a while to build up enough tests that you can be confident when they all pass that you know nothing's gone wrong so definitely it's a journey you start automating one piece so if you automate the visual test maybe you're still testing the donation form or like shopping cart manually but you can do that because now you're not spending time doing the visual checking and then with those that time you're saving invested more automation and it kind of just snowballs to the point where you get large amounts of your work from and I do have some resources here so I have an example repository with the auto updates that's on github it's tied to pantheon because the sites I maintain are on pantheon you will probably have to do some tweaking to get it working somewhere else but all the visual testing and the WPCLR commands to run updates all those things are there and then the automated workflow example this is one where you open a pull request on github and then it runs a suite of tasks it builds a site it does deployment all of those things and just some tools I generally found useful as I started exploring the stuff and some blog posts my favorite one on this list is probably Carl Alexander's twigpress you might know him as he has a great post on getting started with CI and automation for WordPress that's definitely worth reading so that's all I have thank you very much again, Nate Taylor and me tweeted out the slides there's a link and we have plenty of time for some questions if you have one please come up to the mic that way it gets on to the recording so is it possible to visual regression test individual blocks rather than entire pages because like if you have a server and a login block and it's going to affect the whole rest of your page you don't really need to see all that purple you just need to see the purple in the one thing that changed is it possible to like I guess maybe into the junction to win a header line learning test those things individually yeah so the tool I like is called backstop.js it uses headless chrome and puppeteer but rather than running some crazy headless chrome commands yourself you define a config file and actually I'll just show you what my config file looks like so you define things like you know we can delay before we take screenshots my acceptable threshold was 0.1 so the visual difference is more than 0.1% it fails the test but I can go in and define different scenarios and things and one item you can do here is selectors so I'm selecting the document which is the entire page you can select by a specific div id for example so if you wanted to do a visual test of just one component as long as you have a css selector that can get at it then you can definitely do that directly to you know what we talked about today but in general what I find is that doing using a staging server with WordPress and doing regression testing is complicated by the fact that you have both code and PHP you don't have all the configuration information in the database itself I was wondering if you could comment about what you consider best practices and in particular the issues that are inside the database because on staging you have to set the links you have a plugin you have a regression plan in there you can have you then release them you know those little pieces that aren't part of the data inside the database. Do you understand my question? I do but the tests really don't care where the issue is so if an issue is I forgot a semi-colon in PHP where the issue is setting is incorrect in a plugin the tests are just going to alert you that something's wrong. I understand and we're asking about the process of actually syncing you know it sounds like what you talked about here is mainly pushing the code how do you push configuration stuff and adjust the database to make this automated process work whether you're doing it manually so with updates I'm typically not changing plugins settings I don't update a plugin and then I just run the update if you are making a larger change and you need to get changes from your staging cycle that's just a tough issue in general there's lots of people that have tried to solve this kind of syncing and haven't been able to do it there are like used advanced custom fields they have a feature where you can export the configuration to JSON files and then you can commit it to version control you can deploy it along with your code and then live site you just import from that JSON so I hope that more plugins and things in WordPress go that direction that you can get items from the database but actually out into code that way you can deploy them but not the case you can't just shove your entire database from staging to live because you don't want to overwrite your live instances so in your case and your staging database what is in there and how do you keep it like this thing with production database so typically what I'll do is as part of the process is copy the production database back down to the staging but then you know if I have a plugin that I don't know is an e-commerce site and then I need if I'm going to test the cart checkout I need to switch into test mode you define those steps so we copy the production database over I know these are the things that I need to change I'm going to make sure that this plugin is in test mode and you just define those parameters make sure that that option is updated before you run the test and typically you do those things and update the database I'm not directly with SQL I'll use WPC a lot I try not to do direct queries if I can help it it's kind of related it's sort of going beyond what you're doing here so this is just like as you're testing everything before you hand it out so what we struggle with is when you hand a site out to a site you can decide if you need to test on Apple there's some windows and all the different prototypes and we start talking about more about where I start to cross and you don't necessarily have the access potential to all of it so what do you suggest is there a way to take some of these things and make it user friendly or not and we're adding users to our clients for the end test yes you're going to have to put in more work like I wrote a little note app that I wrote on the command line you could probably write a more robust app with the GUI if you wanted to actually I'm using an open source project like a lot of things you can do it yourself but it takes time and energy or there's services that your clients could sign up for and they will do these visual tests for them give them the nice UI they might have to pay 10 bucks a month or something those sorts of things if you do automate like we saw all the tests run when a core request is open if you can build that into your project scope that we will set up these automated tests for you so that when we leave you know if you run through this process these things will still be tested we have a BGP I'm doing this so how can we do the back end test just to not put whatever changes in the front end so if it is completed with the back end while they are developing the application on the back end so how would we do this how would we do this so you can write a test that do log into the admin so we're using Pethless Chrome and so you can actually authenticate with WordPress and log in in my case I'm just testing for plugin updates but you can definitely write tests actually have one more example here and the resources is to this repository that runs all the tests on core request and actually have B hat running if you're not familiar with B hat it can help you with more of those business critical tests so I'm actually testing the give my donation plugin and I'm doing that with B hat and so I have steps to find assuming a bit so things like log into the admin go to the edit screen of my donation form update some settings change the donation level to 500 set it as the default press publish form on the front end and I clear the cache then the amount I updated in the back end should be visible so you definitely can write tests that do those sorts of things and I've seen testing written like custom user roles and permissions in WordPress are something that is requested pretty frequently but often it's hard to make sure those are working properly you can write tests or things like that so that this role tries to go review this page I see access denied or you know these sorts of things so visual regression I said that's a good foundation because just taking screenshots and comparing them is pretty just kind of set it and forget it more of this sort of stuff you're talking about you actually have to get your hands dirty and write these custom tests because they're bespoke for every project you know this one having a donation form there might be another site with the give plugin but their donation form is probably set up completely differently so when you get into testing those more critical features you have to spend more time to write test custom to that project but it's definitely worth it to make sure you have that coverage and I think we'll call it good thank you