 Hallo, everyone. I hope you enjoyed your meal. I have one announcement to make. At 4 and 10, there will be a group picture taken just outside the disk there. Just for your information. Ich sage es noch auf duksch, das geht mir nämlich einfacher. Am 10.04 findet ein Gruppenviertel statt, wo alle Teilnehmer gerne den Fett oppositionieren, um eben ein Fettdelitz zu machen. And now let's say hello to Nico Martin. Nico is the founder of say hello, one of the top Swiss WordPress agencies. He is very interested in topics like new browser, capabilities and automation. And he likes to experiment with all of these things. So you should follow him on Twitter, on Instagram or LinkedIn to keep updated on all of his experiments, his new technologies. He was among the very early adapters of progressive webapp. And he created a plugin for this so that also less technical WordPress users can make use of all these new features of the web. He dealt with web Bluetooth with remote controlled Lego cars. And his newest project, as far as I know, is YouTube Audio that allows you to listen to YouTube videos in the background via webapp. And because of his knowledge and his experience with all these new technologies, he is a thought after speaker. And we are very honored to have you here, live on stage at the WordCamp Switzerland. Say hello to Nico. Yeah, thank you very much for the kind introduction. Well, it's actually pretty great to be back at the WordCamp. It has been three years since my last WordCamp, which was WordCamp Syric. And it is also pretty amazing to see so many people talking about software testing, because testing is usually not the favorite topic among software engineers. But I am here to change that, because with Playwright we do have a new tool that provides some pretty cool features that help you automate your maintenance. But let's start with some words about me. So my name is Nico. I am a frontend developer from Tune, from the Bernie's Overland. I am part of the Google Developer Experts Program for Web technologies, which means that I spend most of my free time playing around with all kinds of new web APIs, with all kinds of new browser technologies. I am also one-half of say hello. We are a small digital agency in Spiez. And we are quite invested in the whole WordPress community, so we maintain a couple of open-source plugins for those projects. And we also co-organize events like this. Now, one very important aspect of my career path is that I have always been self-employed, which means I have always been responsible for finding customers, for completing, developing those projects in line with the customer expectations. And when I started, it was very important for me to look at my projects from the viewpoint of a, let's say a carpenter, where he has a clear defined goal. Let's say you have a project where you make a chair or a table. So you have your goal, you have your timeframe, and you have your budget, and that's it. So when I offered a new project, I tried to work out clear requirements. I would fulfill those requirements. I would invite the client that would then be followed by a training on how to maintain the websites. But that was mostly it for the whole collaboration. I was rather skeptical about monthly subscriptions. So at that time there were, for example, there was one case in Kassensturz about Alpenweb. I guess it was the name where they forced their client into monthly horrendous monthly payments with questionable offers. So I didn't want to go into that direction. And for me it was clear. I wanted to build a website that was my craft. I wanted to build websites, I wanted to deliver them, and I was done. I later realized that my clients, often they don't want to deal with maintenance, with updates and all that stuff. And also having long-term service agreements is a pretty reliable source of income. And it's also pretty great for the customer retention. So, when I, and especially when we grew as an agency, we wanted to find a way to go from those one-time projects, build a project, deliver them, invest them done to a long-term cooperation with our clients. The difficult part here is to draw the line, because again we are a two-man agency. What should we offer as a small agency? And realistically what can we offer? We don't have a 24-7 support center and lots of employees. So we need to find a way to automate a lot. That brings me to my vision. So the vision I had was I wanted to make sure, or the client wanted to make sure, that the site is always running, it's always up-to-date. And so what I had to do was I needed a way to create periodic backups. I wanted to run updates and I wanted to make sure that everything is still working and nothing breaks. The first thing is pretty easy to do. So there are a lot of WordPress plugins that do automatic updates, automatic backups. There's also the WP CLI, that allows you to do backups and you can also use the hosting. So there are a lot of ways to generate periodic backups of your website. Also for updates, there is for example the automatic background updates where you can basically just enable those automatic background updates and it will run the updates in the background. So you don't have to do anything there. One thing I had to work around is we have a lot of our projects in Git Repositories. So we have private Git Repositories for themes and plugins we built for our clients. So I needed to find a way to use or to update those plugins. So the thing I came up with is Git Installer. So Git Installer is an WordPress plugin that allows you to install and update plugins and themes from private or public Git Repositories. It's also integrated into the whole WordPress update process so it's pretty easy to just install it and then run also automatic updates from Git Repositories, not only from the WordPress plugin repository. So I was covered there. The problem here is what happens if something breaks. And the thing here is things will break if you run updates, especially if you have a lot of plugins that will be incompatibilities between those plugins. There are big things that can break. Big things are bad, but the good thing is you will notice that right away. That means when for example your website has some internal server ever and will return a web screen of death, you will notice or you will see that something is not working properly. Also since WordPress 5.2 I guess you will also be notified in your email. So if big things break, you are pretty much correct. If small things break, that can even be worse because it's pretty bad if a customer calls you and says well there's this one feature that you built for us or if people use it but we need it to run and now a customer calls and it isn't working anymore and no one knows since when and why it broke. Well that's pretty bad. Not that even happened to me but I've heard about this. Nobody in this room ever had those problems. So the thing here what should we do if we have those situations well normally you would just run your updates, check through your website and if everything works fine you will just make a tip and everything is working perfectly if you don't want to do it yourself because well it does. You can then maybe you pay someone an intern or a junior who does the stuff for you but still that's not the solution and the big problem here is that it gives tasks that's why we invented computer computers and especially as a software developer I take those things very personally and that brings me to the whole subject of automated software testing. There's a great joke to start this whole every year of software testing so a software tester walks into a bar runs into a bar rolls into a bar dances into a bar jumps into a bar and orders to beer 2 beers, 0 beers 999, 912, 999 sorry just a number a lizard and a tissue glass minus 1 beer, 3 beer and the test is complete he leaves now every customer comes into the bar so the thing is the idea of software testing is that you will ship your code with all the other tests for some units of your code or for the whole software or your project in the end and you will be able to test for all the cases that you define but you will never be 100% certain that you covered all possible cases that could go wrong but it's a very good starting point so what are we actually doing with software testing so there are basically three categories of automated testing so first we have unit tests and unit tests are ready to test to to test single self-contained functionalities in isolation so that means maybe you have one function that returns a value you will write that test you will execute the function with a set of, let's say some input parameters and you will then check for the output parameter and if the output parameter matches the expected output that test is okay that means even if you have a big refactoring in that function you will always be sure that the function stays or is still working properly that's a great way so we have a single self-contained functionalities but it's also very far away from the actual user second we have integration tests and integration tests are there to test how certain units of our software play together so in that case for example you see that we have the sliding door the sliding door works perfectly fine also the gates that are working perfectly fine in isolation quite useless and last but not least we have end-to-end tests and now end-to-end tests are a way to test the whole flow of a user over the whole interaction of the user that uses your application so for example you have your product page there you want to be sure that you can click on the add to cart button it will be added to the cart you can then go through the whole checkout process to make sure that every step in that process is working as expected is working fine and in my opinion end-to-end tests for customer-facing applications are really important or let's say the most important test that we can write now one great tool for end-to-end testing is play write so play write is written in Node.js by Microsoft and what play write does is it will render your website in an actual browser and it will execute all the steps that are provided in the test file so you are basically able to control a browser and to do all the steps that you would do manually now in an automated test the USP of play write is now that it doesn't just run one browser it runs multiple browsers and multiple browser engines and that's the USP of play write and it also has some implementations for Python.NET and Java so if you're not familiar with JavaScript you are also able to write tests in those languages as well so how can those tests look that for example is a test or two tests first of all we want to verify that the title of the Work on Switzerland website is the right title and then we want to check whether the tickets link is still working so as you can see here our first test we are opening a website the Work on Switzerland website and then we expect the page to have a certain title the second test again we are opening the home page then we are looking for the link to the ticket page we click on that link and now we expect the page to have a new URL which is the URL of our tickets page we can run that play write will spin up the browser play write will do all the steps and when the title or also the URL is not the one we expected well then it will just fail and in those two very forecasts we already touched on two main concepts of play write we have locators that allow to locate an element on the page and we have assertions where you can test for specific things let's start with a locator so locators as I said are a way to find an element on the page but it's very important here that it's not just a query an element that you could do with JavaScript it is actually an automate function that means that it will query the element and it will also wait until the element is rendered on the page because you can imagine that maybe your rendering takes some time and the button is not there yet or it will wait until it finds the button there are some built-in locators like you can get the element by the row by the text it contains also by the old text and there are also CSS locators that means that you can basically pass any CSS select when you want and also pass a nested CSS selector so you want to have the button that is inside the div that is inside the navigation so it is possible to query elements by a CSS selector but one important thing here it is recommended to prioritize user visible locators like the text or the accessibility roles because the DOM structure of your site could change over time so maybe your theme or your plugin has some update and it adds or removes some classes if you query the element by some things that you actually address it's way more likely that your test will still be valid even if there is an update next up we have assertions and now assertions allow us to check whether an expectation is met or not so we have basically we have this expect function we can grab a locator or the page object or basically any value we can grab that value inside the expect function and then you can call a matcher on that expect function so in the first case for example I want to check whether my checkbox is checked I can check whether my button is disabled or not in the viewport so we can check for certain things in our application there are also negative matchers so in the end I can check whether a an expectation is not met and by default whenever an assertion returns false it will just determine the test with a soft assertion so we can expect.soft and then a locator will fail but in the end we are able to execute all the tests even if some tests fail before there is a whole list of all the possible matches on the playwright documentation now as previously mentioned playwright allows to run tests in different browsers and that is actually pretty pretty cool because there are 112 pre-configured desktop and mobile browsers with different browser engines so we have basically three major browser engines we have Chromium which powers Chrome Microsoft Edge and some smaller browsers like Brave then we have Gecko that powers Mozilla Firefox and we have WebKit that powers Safari and also all the mobile browsers and playwright actually comes with all binaries for all those three browser engines the important thing here is that for example if you want to test for the Pixel 5 the Chrome on Pixel 5 it will not actually emulate the whole Android operating system but it will run the mobile Chrome or the mobile Chromium with some pre-configured things like the viewport and the browser agents so in the end there is a possibility that things are not working in the real browser but having those three browser engines makes it extremely extremely reliable in my opinion so enough about the theory let's have a look at some code now since playwright is a Node.js application I think that Node.js is already installed so everything if you don't know how to do that but as developers we like to write code but often times if there is a better way we will go for the better way and playwright has a tool called CodeGen now I need to change some things because I want to show some demos so what we can do is we can run the npx playwright CodeGen with an URL and now let's imagine we want to test if the browser is still valid on the WordCamp Switzerland website or not so we can run the CodeGen command this will now open up the browser and also a console and everything I am doing in the browser will be locked to the console there we go so we have the browser and as you can see we already have the first the first step so we are going to the page now we see ok there is the cookie banner I can click on the cookie banner and you see ok I click on the element that contains this text I can now click on the close and accept there we go and it is gone I can now reload the page and the cookie banner should still not be visible so we have our first test I could now take that test and run it as a playwright test and it will execute all the steps in the browser but the CodeGen unfortunately does not have any assertions so what we can do now is we can copy that into our test file and I now need to change some things so first of all I have my banner and I will not click on the banner for now and I have my button and again I will not click on the button for now and I will now expect something to be visible I expect the banner to be visible so I will await everything that happens inside a playwright is basically a synchronous JavaScript function so I can await all the things here we go so I will wait for the expectation that my banner is visible so when we run this I will actually check whether the banner is visible now I will add a click event and I will now check whether the banner is not visible there is one problem here maybe you have seen that before but the cookie banner on the Bertha Switzerland website has a little delay so that means that when we click on it it will fade away so if we check right after the click it will still be visible so therefore we need to wait for a little time out how was it for time out let's take one second there we go after that I will reload the page and now the banner should also not be visible so that's it for the test let's have a look at some examples first of all we want to run our test so by default playwright everything headless that means there will no be browser visible everything happening in the background but with the headed option we can actually run the browser in a visible mode so now we will run our test it will open all three browsers we have firefox, chrome and safari it will run all the tests and there we go pretty cool now what happens if something is not working as expected we will run the same test again and as you can see the test will fail now what happened one very cool feature of playwright is that it will actually record a video of the test it took so we can actually look at the report we can see the test in chromium we can open that and we can actually look at the video and we will see all the steps that playwright did for us but that's in some cases it makes sense so some errors are so we are able to see some errors with the video already but playwright has another pretty cool feature and that is the trace file so we can run the same test with trace on and now since I am not passing the headed option it will do everything in the background so it will take some time and we are not seeing anything but it will actually do the same test again and it will record the video and it will also record this trace file and the trace file now allows us to investigate further and we will see what steps did playwright do so again there we go we have our results and now we have this trace file we can open the trace file and we can now go through the page and see step by step what playwright did and where it failed and the cool thing here is those are not just images and screenshots of our application these are actual DOM snapshots so we are able to investigate what is actually happening in that site where did it go wrong we also see all the network requests we see the console so we have a lot of things that we can then take and work with to see where it went wrong and since I guess since 2 weeks or so there is also the UI mode where you can actually run playwright in a UI mode and as you can see now the trace file but it's not actually doing anything it's not rendering anything but we can actually start each test by itself so in our case you only have one test so I can click on the run and now we see step by step what playwright is doing and also where it failed so it's pretty cool so we have so many tools to see where our website then we can go or investigate further on what went wrong pretty cool now how do we write good tests so first of all we need to try to put ourselves in the shoes of the user so that means there we go that means that we need to think okay our user is going to the website what does he expect and also what are our key application workflows so if we have a shop one key application workflow would be we need to go to our product page we want to pass the product to the card and we want to do all those steps and we want to make sure that these things do work also we need to test for functionality not for content so as we have seen before if you are testing for the title of a page it's not recommended because the title is a SEO expert who goes through your site and he wants to change the title your site will still be okay it will even be better but your test will fail so it makes more sense to test for the functionality of the page than for specific pieces of content and then last but not least monitor and you need to improve so that means as we have seen in the joke before there are so many things that can happen and we don't expect those things to happen so we need to make a feedback okay well we had this update and our site hasn't that problem we are able to actually write a new test and this probably will never occur again because we have one test file and our computer will just do this one test file over and over again let's have a look at one test here we are on our own website sayload.ch here we are looking for a link inside the navigation we expect the title to be a certain text string and also the h1 to be portfolio again that's a pretty bad test why first of all we are querying the locator by a chain css selector and we also expect certain text strings to be in that way a better way to test the same functionality because again we want to click on the portfolio link and we want to be sure that our site returns a response that is not 500, 300 whatever so what we can do here is that's basically the same functionality but it's a way better test because we again we go to seeload.ch then we have two things that happening in parallel first of all we click on the portfolio link inside the main menu very likely then we are waiting for the response and then when the response fulfills we will check if the response is okay that means that it is true if not it will fail same test but it's way better here we have a test that we are actually using right now for our website it's just to demo how a test could look here we are looking that our main page is working correctly then we look whether the main navigation works, that's the same test you've seen before and we also have our floating button that means if you scroll down on our website you will see this little menu button that flies in so here we are first we are looking if the menu button is not in the viewport then we will scroll 800 pixels and now we expect it to be in the viewport and also then we check if it works that expanding button or not and also if the floating navigation menu point is working as well so that's one way that we could test the navigation functionality of our website but that's just the navigation there are a lot of things that we can test on our website and that we do test on our website so let's put it all together I have a way where we can basically create deployments and where everything is happening in the background without manual interactions I will just start a new deployment there we go and now what it does is it will take a backup of our website it will run all the updates and it will then check whether everything is working fine and there's one thing here we have an update ChaosMonkey wants to update from version 16 to version 17 and ChaosMonkey is a small plugin I wrote that basically checks for its own version number and if the version number is odd it will prevent our contact form from loading so that's our contact form uses javascript so it's very likely that our contact form will not work now our test will cover that use case and it will then restore the previous backup one thing here to mention is that as you can see the test failed we can look at the results okay we have our contact form test that did fail we can view the trace file and we can see okay our contact form did not render for some reason but the cool thing here is I can run a test on our production website where I know or I can run an update where I know that it will destroy some crucial functionalities of our website live on stage without even having to worry that something will happen because it will test automatically it will restore the backup and we basically now have enough time to investigate why the error occurred so also some of you might have a deployment pipeline where you could add playwright tests as one step even if you don't have such a pipeline it's still pretty easy to write a test to automate some of the testing you would do manually and maybe if you then click all the updates on Monday morning instead of then clicking through your website you can just run this one test for the client website and you know whether something happened or not so that was it from my side all the slides are there online so you can just scan the QR code and you will have all the slides I do think we have some time for questions so if you have questions feel free to ask thank you very much for your attention I don't know I think that one was the first hi I have a question for me with update in the path there was a lot of problems in coexisting with contact forms with working contact forms where maybe for example the reception doesn't receive the email for example the other thing is so I guess everyone maybe has a problem with Google Maps integration where Google says this is for development purpose only where the RP doesn't still work anymore because you reach the limit or someone of the Google RP will not change the permissions or something like that also an example a short demo where it's made possible to check the compatibility or functionality of this Google Maps integration for example is that possible or is it I think it's possible but I have no idea how I should write the code so the thing here is how could you find the correct locator for that thing that is happening inside your Google Maps integration so their code gen is a great way because even if this is running in an iframe or in some sort of I don't know where it is running as long as it is accessible in the DOM through the browser you are able to access certain elements also in the iframe so that means you could look for an element in an iframe so maybe you know that well when I reached the API limit there will be a message so you can check whether this message is visible or not so when the message is visible the test will fail the first thing I think is way more complicated because yes emails are quite a big problem from time to time so it is very hard to test things that are not actually happening on your website because again I mean you can check the response of your contact form request but even if the response is okay there are so many things that can happen along the way so it is very hard to write test for an email the only thing I could imagine is that you could maybe send an email to yourself and then let play write email arrived in the inbox wow I don't know I think that would be a very okay yeah I think that's not so easy is it possible or do you have knowledge about that is it possible to have a check of the console errors because sometimes if Google Maps integration fails you get an error message in the console is it possible to have a check on this I'm not entirely sure about that but I think so because in the trace file for example you see the console errors and I think it should be possible to check for console logs but I haven't tried it so I would need to check that okay great thank you just a quick and small question I'm using Cypress for the same Cypress is also a testing framework but I had never heard of play write before did you use Cypress before, what's the difference why do you prefer play write yeah I tried it once and that was a couple of years ago and I can't go into specifics and how they are different because when I decided for a framework for my for my own integration of the three browser engines because maybe that changed I also heard that some testing frameworks are implementing play write as a test runner but when I decided to use play write that was just the only testing framework end to end testing framework that would actually run all three major browser engines thank you there was a question back okay thank you so very much my question is a little bit complicated what is the key advantage besides Selenium from play write again basically the three browser engine implementations so you can actually test for browser compatibility in one testing framework I'm not sure how it is with Selenium today or are they trying to implement it from what I know Selenium is only using chromium browsers so if you have some errors that would only occur in a webkit browser that would not be detected so you mean it's testing all three browsers at once okay good thank you yeah, technical on the demo that you showed us with clicking the agree button on the word cam website if you're on the same test a second time that would probably fail am I right, because the banner will not show up anymore why it's waiting for the banner each test has its own browser context so when the cookie is stored in the browser context if you are in the same test then that's what I showed before when I reload the page and it's not visible that's because the cookie is stored and the browser still has the same cookie context but if you're running a second test then it would again basically no cookies exactly new context basically a new browser without any cookies okay, cool, thank you do you charge your customers for the writing of the tests and if yes how much in percentage of a project is this I don't know because right now maybe you have seen the version number it's 00.200 something so I am right now working on it it works quite well for our own website so we use it for our own website to have the automatic updates and stuff right now we are not charging anyone for anything but maybe if this all turns out to be to be working quite nicely then we will maybe I don't know maybe we will charge a monthly fee for like additional services inside that fee or we will charge for let's say after the project we will basically go through the application and ask which tests that they want to have and maybe charge them that way but I don't know yet are you able to because sometimes you are into a scenario when let's say you have a functionality based on cookie for example and are you able to define like different structure like initial cookie to be set before you run the test so you for example can compare like old cookie to the new one you are expecting as you are no longer able as a user to generate the old structure of the cookie for example you mean a way to mock data that you will start with a predefined cookie exactly I don't know for sure I did not mock data yet but I am pretty sure that you can because I would be very surprised that's a feature that all the testing frameworks have that you can mock certain responses certain things so I am pretty sure that you can but I never tried it you can also work around and create some endpoint or whatever also just generate this a good thing to talk about I would not recommend to test for cookie structures because in the end you want to test for what is happening on the website so when I had to test about the cookie banner I could test for if the cookie is set and if the cookie has the right string but in the end that's not what the user sees the user doesn't care whether the cookie has a certain structure or if it has a different state and we introduce new cookie which has some data in it and some people might still have the old cookie with different data there so I would like to mock something to make sure the new version of the system will work for people with the old cookie before setting the new one just that I am pretty sure that this works if not, then thank you very much for your attention and have a nice rest of the conference