 All right everyone. Thank you for coming to my session and to end testing with playwrights. There's not very much time So I'm just gonna jump in a Little bit about me. I'm Petar Basic I come from Serbia and I work for a company based in Vienna, Austria as a double engineer Genomics is the company So I'm not gonna talk a lot about the company. I just wanted to mention our main Solutions because of the context in which we execute the tests So lupus digital is kind of an out-of-the-box solution where we ever you could have a decoupled front-end even multiple front-ends When you have then a content pool Drupal is a content pool in the back end And a lot more things about there. Also, we have a contributed need in the Recently contributed a lupus decoupled Drupal As an easy solution for decoupled Drupal with front-end front-ends in any other framework So at the genomics we value our testing process greatly It's a vital part of our quality assurance process and it is greatly integrated in our CI pipelines Which saves a lot of resources when some new features appears So you don't have to check for if it broke anything Before that was already existing. There are automated tests for that What we previously used during the years is more traditional framework be hat, which is actually a great framework It's a PHP framework. So you could just require it with composer and Start using it and you could interact directly with Drupal API and its step definitions and It all has already predefined Extensions that exists that you have predefined steps, which you can use with Drupal, but over the years we figured there are a lot of downsides with it so we figured there's appearance of flaky tests and actually Debugging those flaky tests is the experience with it was pretty poor So we decided to maybe it's time to find something else But the main decision so why we decided to go and search for something else are those flaky tests and The poor debugging experience with them. So in our CI pipelines in our workflow, you would expect if you Develop a new feature you if if you see the green tests and everybody everything passes you only test your new feature and If that's okay, you just merge and move on that was not the case it was a common appearance that you would see a Just random test failing with the new feature implemented You would normally think that your feature is that introduced it and After a lot of time lost there debugging checking what's going on with your new feature. You just figure out It's not your feature. It's the be had that just You forgot some explicit to wait to add to wait for some element or something So and actually did that debugging that especially in a CI environment was Practically not possible or very hard and time consuming resource consuming So there was just a decision to move on and try for something else In this process of finding something else we stumbled upon a lot of frameworks The which when the decision ended with playwright added some related post where other people had the same Processes as we did. There was also side press and code set JS But playwright actually turned out to solve most of our well everything that was missing in be had and In other frameworks themselves so I would mention some of the Playwrights main features. I bolded the ones that actually were the cause that we decided to go with it so auto wait and auto retry auto wait feature would just Playwright will wait for any Component to be actionable. So for example, if you need to press a button It will wait for until that button can be pressed and then press it the auto wait the auto retry feature will Try and execute the action multiple times until until all the conditions are met so for example, if your CI is a little bit slower and The component doesn't load in a first 10 milliseconds and the test checks if the component is there It's not there But if you're a user it will be there in just 11 milliseconds. So it checks a couple of more times and the test actually passes and For the debugging part it has tracing mechanism, which will actually record Everything that happened during the test execution. So you would have HTML network what happened with the network in the console It will record screenshots. It will you will even have a video recording Which you can actually see after the tests are executed Which helps immensely when you're debugging the tests There are also Powerful tools that come with Playwright Integrated with playwright. I'm gonna mention most in the inspector which will serve something like an xx debug will for PHP You just run the test in the with an inspector active and you could just step through your lines in the test and you will see what exactly happens like if you are doing the test manually and the trace viewer is a feature that tool that you can use to see those traces and much more features I listed here, but Yeah, not that enough time So we faced some challenges when moving from be had to playwright The main one being the communication with Drupal API in the be in be hat You could just do that straight in the step definition And some other minor challenges So how do you set up the environment in be had you could just do get them and get any environment variable You need it. So here that was not possible And of course, you don't have any predefined steps So you have to invent everything and how do you present those test results on the CI environment? And how do you share it with your team? So how do we how did we solve this? biggest issue the communication with Drupal API we decided to just use drush and You could run you could make your own custom drush command or just execute and drush PHP eval and of course in because playwrights we use it in its implementation it in No.js. No.js has this child process which you can just run and execute any command also drush command The bigger problem was how do you make it happen when you have a dock rise playwright and the dock rise Drupal? So you need to set up Docker socket in the in the the playwrights container install the Docker CLI in the playwrights container and then it actually Just works, but you have to add append to your child process a command Just execute like it would execute so Docker command in the container name I will skip most of these other challenges because they're not that important They're not that big so for environment you just had that time For writing steps. It's a manual labor. Just need to write them and read the docs the documentation on playwright is really well And for the CI setup we decided to publish its HTML report on cloudflare after each Build on a CI so it actually becomes available to the whole team immediately And yeah, I wanted to show some code will not code to execute a couple of tests So this would be an example An example of a test file, so it would be equivalent of a feature file in the be hat and Each test itself would be like a scenario in be hat So these these would be just some simple Drupal based test just testing some if the pages are working or so and I'll trigger it in the inspector mode so we can see all right, it just opens up and You can just step through your tests There's a button and it goes through the test and you exactly can see what happens and this is a normal browser So you can do open your Developer tools in a browser Yeah, and you can see the network tab you can see whatever happens in your Read your page when the test is executed Which is pretty powerful. I think I could Only dream about this with be hat and I also wanted to show the tracer how it looks like when a test is actually failing So it takes a little bit longer because playwright is auto waiting for stuff that we requested it But it's not there. So nothing to wait for anymore So we'll just run the tracer Okay, so this is the report it generates So it's a list of tests so we only have one test and it shows it failed when you enter it It shows you and on the each line where it's the exact line where it failed It has a screenshot and it has a trace Which is I think the most powerful thing you see what happened in each millisecond of the execution and How the page looked how the network tab looked so you don't actually need to don't need to run it like I did with the Inspector you can do this after the test has run so it has everything So this is where the test failed what it's this is where what it was executing when it failed it shows you everything the console the network and What it expected and what was missing? So, yeah, that's why I wanted to show yes here you can also use your developer tools and just check the HTML Inspect it and everything is there like you just open the page and Yeah Okay So our experience so far we just merged it few weeks ago. It's pretty good experience Its performance is comparable with be hat, but there's there are some issues that It takes a long time to build the container when It's running on CI and then writing the tests sometimes you forget the weight and each Playwright Assertion is running asynchronously so you just need to add the weight before Yeah, use those playwright assertions because Those auto wait and auto retry are only available if you use the playwright assertions Some tip to use the tags to tag the test so you could filter out them when you're running them There are also hook functions available where you can hook into the Execution process and prepare the environment and clean up your tests afterwards So far we didn't see any flaky tests. It's pretty reliable and stable and I got some feedback from my colleagues Which are pretty happy didn't take them a long time to get used to it. The documentation is great And they just actually love it It's I would say it's even more developer friendly because we are the ones who are writing the tests and we are developers We don't need that groovy in be hat. You just see the code. You know what it's doing and It's the overall experience is positive and the debugging experience is excellent as we saw it has some powerful tools Which turned out to be very very helpful Yeah, I made actually a small play playwright Drupal starter repo The presentation is available on the genomics comm website on the blog post so you can download it and There's a link to the public repo. It's I didn't really test it But if it's needed do you there's I left the issue No, I didn't test the starter repo It works in our projects, but I didn't want to post the whole project of course I can So I left the issue on the GitHub issue tab open So you can post an issue or contact me directly if there's any help needed Yeah on the Drupal comm site. I said Some resources and yeah some Drupal con stuff. That's it Questions Yeah, two questions. Maybe I just want to ask how long have you been using this playwright? And did you consider any other tools like PHP unit or Cypress or? So yeah, we for this kind of testing we tried using Cypress for example, but That's why I mentioned that the kind of the Projects we are developing so they're decoupled and they're running on different origins So Cypress I couldn't get it to work to switch in the same test between two different origins It just didn't allow me that plus it was really hard to get it Execute any commands outside of that so I couldn't get it to run drush Easily at least so that's why we went with playwrights How long I think six months now, but it was on and off of course I had the other stuff to do I wasn't solely on this So yeah But it took me really not that much time to get used to it as I said documentation is wonderful So will you be able to extend this suite to add any non-functional tests? Excuse me Will you be able to extend the suite to add any non-functional tests like security performance Accessibility using playwright. Yeah. Yeah, you would be able to yeah, thanks for the presentation very nice You mentioned the the challenge of the initial setup and you need to create a socket to make the different containers to talk Between each other. So apart from that. Is there any anything else that you need to do? For example within the Drupal application or it's just enough to create this bridge between the containers So yeah, that's just enough to create this bridge between containers. It's a solely a Docker setup It's actually in the starter project. I can show you right now if needed It's So you would just create a volume of where is it this one? So volume to the Docker socket in the in the playwright container and you would in the playwright container install the package Where is it? Docker C CLI and You could just execute Docker commands in the playwright container and they will automatically search actually find the container outside of it the the container where the your Drupal is running and There's nothing else needed Nice. Oh, yeah, so this line was needed as well to change the permissions for the Docker socket in the container But it's included in the starter starter repo so you can find it there How much space do the reports take up and how long do you keep them? So on CI we keep them for that build only they get deleted afterwards if you don't need them, of course And they don't take a lot No, yeah, yeah, those take some but there is a setting where you could tell him Okay retain videos only if it's something fails and only for that test So the videos will it will just be a one small video in that case But we disable disable it overall because it's not very useful And it takes a lot of more it takes more time for the test to execute if you're recording video And of course it takes space, but I didn't check how much space they take Hi, thanks for presentation I have just some experience with beehut and behave but The gherkin that they have so aligned with cucumber idea Is something that is I would say must have for us so that is a documentation for business people how feature works Wondering. Yeah, what what playwright can offer in it can offer you could integrate cucumber with it You could use gherkin and you would just do pretty much the same thing with beehut has been with beehut So have the same syntax and have the playwright in the background Like the here, but we decided not to use it because we don't actually need it and it was easier for us This way so but yeah, you can use it I didn't include it because I thought it would 20 minutes would be enough, but it's not What's your test testing strategy, what do you test with playwright and what you don't almost everything We can test that could be tested and seen the most important tests I would say are the caching tests where we actually do some changes in the content and we change how What is invalidated what caching is invalidated what it's not Then we do some just basic front-end tests if the buttons are working if the The API is working. So if you press the button if everything gets executed We everything mostly you could do with it Some stuff we don't test is like what we do actually execute with Drupal API is if you need to create a node We don't go through the whole process of adding the node in the test. We just have a function in the Drush command actually now which just clones some node that already exists and we just have a new node and test cache In validation with it. I'm really really sorry, but this was a 20-minute session and we need another put another one in We can continue. So yeah, thank you very much