 Even if it is headless, you need to have it and that would result into slow test execution. Having one UI test to ensure that the recipe has been successfully created is certainly the requirement but that is not a scalable approach to probably check your recipes listing page. And yeah, so via the UI, it took me approximately 20 seconds to create just one recipe and I'm just talking about the recipe content type here and not the associated fields in the recipe, like for example, taxonomy terms or tags, right? So those were already present in the system. Just creating a recipe took me 20 seconds. Okay, I'll hand over to Akanksha for next few slides. Thank you. All right, let's get our hands dirty. Let's create the data. So what do we need? First, we need test data and second, we need this data every time that we run the test. So we should be able to create this data while running the test. Thank you. Okay, so the first part, the first step would be creating the data. The second, we need a method to export this data so that it's available in the config or, you know, in the code and not in the database. Third is to create a test module for shipping this data with our tests. And finally, we need to run the test. So when we run the test, we import this data, data gets created, we run the test, and then we delete this data so that our environments are clutter free. Let's see how we can do this in Drupal. So, all right, so first step, creating the data. Now, there are multiple ways to do this. What we'll be talking about today is devil and devil generate plus realistic dummy content and then the drush commands to do all of this. First, let's talk about devil and devil generate. So these are two modules which are very useful for developers. And the best part is that there's a UI for creating dummy content through devil and devil generate. So once you enable devil generate, you'll see a UI where you can create test content. I will show this to you right now. So you can create multiple content types, nodes, taxonomy, menus, users, anything. It's very configurable. So you have an option to delete content and you have great control over when the node publication date should be, how many nodes to be created, whether to delete nodes or not. So this is what the UI looks like. You can generate the type of content that you want. And we have, all right, so just a moment. Yeah. So we have all these options where you can select which content type to create, how many of these to create, and whether to delete all the preexisting content types or not. Anyone can run this. You do not need to know code to do this. But the content that is created with devil and devil generate, it uses default lorem ipsum text, which does not look great. And this is where realistic dummy content comes in. Realistic dummy content, super charges devil and devil generate to create actual content and it adds images from like open source images so that your content looks great. All right. So I have a video here. So even in HTML fields, basic formatting is added so that it looks actually like the content that someone would go in and create and not something that is created by bots. Hold on, hold on. So as you can see, there is some preexisting content in this video. And we'll go to the UI, which is inside configuration, devil generate. And we have a bunch of options here. I'll be creating nodes of the type. Article and recipes, 10 nodes. And we scroll down. So there are options. You can select what language you want it in, which user should be used to create this content. And there is an option to delete all the old content. Once we do this, this is extremely fast. So we already have all of these recipes and articles created. And everything that was there was, it has been deleted. And as you can see, the data is there in all of the fields. Now that we've finally created the content, it's time to export it. But first, let's also see how we can create this entire thing with Drush commands so that we can use it in tests. So there are Drush commands to generate all of this, Drush Gen C. And then you can generate whatever you want, either menus, taxonomy terms, users, work apps. Also the kill command is the UI option that you saw to delete all the old content. So if you put the kill option, then all the old content would be deleted. There are a bunch of other options. So you can skip fields. You can add languages, feedback. So any of this can be used in your script to create test content. But each time new test content would be generated. So this is not helpful for visual regression. We want the same test content to be generated every time so that we have baseline images. And then we can compare with those images. And that's when exporting and shipping the data comes in. And for this, we are using default content module. There's another module called Default Content Deploy, which is also great. But I'll be talking about default content today. So default content is a module with which you can ship default content to your site. It resides. So any default content that you have, it goes in the content folder of your module. And the benefit is that you can export everything as YAML files, including files. So images also get exported as YAML files. And then we can import all of this to run our tests. As you can see, that is my module.info file. You have to define every UUID that has been created with this module in this file. The benefit of using this is that once you've created this, whatever content is created, when you disable the module, all of this content will automatically get deleted. There is a patch for this. It's not in the default module. All right. So we generate content with devil, devil generate, realistic dummy content, whatever you like. And then we export it with default content. So every time we enable this module, all of these notes, taxonomy terms, user, any entity would be created. And every time we disable it, it will get deleted. So now we can reliably run our tests with this content. This is the command to export the content. So once you've created it, you can export with the node ID and to whichever folder you want. And finally, running the tests. So this is a basic functional test that I've written. In the setup part, we're just installing the module. So as soon as you install the module, the entire content will get created. And when you disable the module, so I've also, there's a drush command to delete all the content that you've created, which is dcd, and the name of the module. So anything that is created in the test content module would get deleted. And anybody would get disabled. It's a simple test which gets the recipes page and verifies that the title of each node that we created exists on the view page. So once this runs, we go to the recipes page, and it loops through the node titles and makes sure that everything that we created is now there on the website. So this was way faster than creating every node manually and then verifying it. This can also be run in the CI. Like I've created a quick GitHub action to run it in CI. It does a basic install of the site with umami, enables the module, and then runs the test. So we will get real-time feedback on every PR, whether it is working correctly or not. And I will pass it on to Shweta now. Shweta, how many of you are aware of automated visual regression tests here? Oh, that's a decent number. So you might be aware that the primary challenge with these sort of tests is that the data keeps changing or the data is dynamic. And in order to implement it, you need to understand how can the data be static and consistent across your different environments. So that you can leverage the maximum out of it. Or else you would end up in just verifying the layouts and not the content. So what happens is when your data is consistent as Akanksha mentioned by just enabling your test module. So this is another quick video where we've enabled visual regression testing with Cypress. And here you see that both the English and Spanish language got validated in no time. So you see that you can immediately implement automated visual regression tests. And the bigger advantage is also certainly that your functional tests reduce drastically because you don't have to sit and assert all the titles for all the recipes. You don't have to verify their tags and all of it, right? With visual regression, it gets validated. The functionality gets validated as well apart from the UI. So let's talk about good practices with test data management. That's what TDM stands for. Yeah, so good test data should allow you to exercise and validate high-value user journeys. So that's a given when it comes to end-to-end for test automation. Ensure that the highly used user journeys are exercised in your automated suites. And you don't use dummy content as Akanksha mentioned that the realistic dummy content would help you to have content that is closer to how the real user would be using it or consuming it. So ensure that this is there in your practice. Validate edge cases. Obviously, don't forget the edge cases also have data planned so that your edge cases are exercised as well. With the help of test data, you should be able to reproduce defects as soon as they are caught in higher environments. You should be able to simulate errors also, errors that are probably caught in the production. So externalize test data using lightweight file formats like CSV or properties or YML. So we've been using YML here. The reason being if you use Excel, then reading from Excel is quite time-consuming. So ensure that you use lightweight file formats to store your data. Your data should always reside separately from your tests so that one is based on the test environment, your data might vary. You might have different data files in order to run your automated tests across different environments. And any changes to the data should not result any changes in the tests or in the automated test suite. So that's the reason why you need to externalize your test data. So generation and destruction of test data on demand for every test. Let's take a quick example of probably adding new address. So the feature varies slightly for an existing customer and a new customer. For a new customer, you would have just like at primary address probably. And that button wouldn't be visible for an existing customer. For an existing customer, it could be like add another address or add a secondary address. So you need to ensure that you are able to create that new customer on fly using your setup activity and ensure your test for the new customer address and then tear it down. So there shouldn't be any data dependency, any reliance on data for tests because another reason is that you should be in a position, you should be empowered to run your tests independently and in parallel. Also ensure that adequate test data is available to run the entire automated test suite before starting with automating the functionality or considering the use cases. Ensure that the data requirements are well thought of and the data is well captured and made ready and available provision for all the use cases and all the test scenarios for that particular function or feature. Yeah, your test data should not limit or constrain the automated test that the teams can run for sure. Consider all test environments. Usually we have the dev, test, QA or probably test or QA, UAT. So whenever you are structuring your data just ensure that you are taking care of the requirements. So data needs might vary. Some tests that can be run on probably tests or lower environment cannot be run on pre-prod for example. So ensure that your data requirements and you consider all test environments before structuring your data and structure the data uniformly so that your scripts are not affected. So for example usually you would store the URLs of different environments in different properties file, right? Okay, account for localization as well. So what I mean here is locale usually constitutes of language and country. So there would be some data that is common across the locals. So that should be written just once and data that is differing like the success and error messages based on different languages should be captured in a different file and data that is common should not be redundant. So you structure your files accordingly so that the data is not redundant for the common parts of your localization. Yeah, the most important the way your test automation suite consumes data or how the data is managed in your project might vary from in your organization might vary from project to project. So ensure that documentation is in place so that the entire team understands how the data needs to be created, modified, destroyed and how the test automation suite consumes it in first place. Talking about the anti-patterns. So yeah, so as I mentioned an excessive dependence on data define outside the scope of the test, right? So for tests that are dependent on data like the address test that I mentioned you need to ensure that the scope of the data is within the scope of the test. If you are going to rely on data that is outside the scope of your test then that would result into fragile tests. So for example let's say if you are executing the address functionality if you are restricted by executing tests in particular order then that would result in fragile tests. So for example you first create a new user and then add new address to it and only then you can check the add secondary address feature then that is not going to give you any sort of flexibility. And dependencies on external data sources. So as I mentioned if your data resides in Excel then that's going to bring in big performance issues with your execution and it's going to delay the feedback of your application. Yeah and copying production data people usually feel that yes it does give you certain insights but then it might also contain confidential information and hence it's necessary that you mask all the sensitive data and not just blindly utilize the production copy the production database in your automated tests and using obsolete or irrelevant information any changes for instance to the fields of your content type that should be reflected in your data as well so that you are not using obsolete or irrelevant information in your tests even though your tests might not fail but you need to ensure that the data is also modified when the content gets modified or when the functionality is modified. So let's understand the key benefits contribution to test automation by the entire team certainly one of the major roadblocks why everyone in the team cannot contribute to test automation is people have this understanding of it to be highly technical which might not be true at all times test automation is certainly a whole team responsibility and the entire team can contribute to it in several ways I can do a literally different talk on that but Akanksha just explained that how we are generating data in a semi-technical fashion in a user friendly fashion and with this what happens is you actually empower your team to think and begin with test automation quite early in the cycle wherein the team can focus on the data requirements of the functionality and need not worry about having those scripts always scream green that's what people are really worried about when they begin with test automation that they want everything to appear green all they have to do is once the data has been taken care of go to respective URLs so I'm talking about the act column here either go to respective URLs and add a visual regression assertion there the recipe listing page really they need not worry about any functional assertions also in that place reduce test authoring time certainly let's say that you have 10 recipes added and when you go to the recipes listing page you'll have to sit and add an assertion for every title the tags, the media and everything and yet there is no guarantee that the UI is intact just because your functional assertion passed but with visual regression your test authoring time reduces drastically you just go to the respective URL and you assert that the page looks intact your functionality gets automatically validated with it no redundant steps as I mentioned you don't have to like even if you write a loop for your functional assertions yet there's quite redundancy when your tests keep on when you move on to newer tests next tests which are also related but there would be redundant steps minimal functional assertions needed I explained that cleaner and maintainable tests for sure because your tests are modular and plus minimal lines of code and so they are much more readable and cleaner ROI is exponentially higher and how we'll understand that data creation is a one-time investment really you wouldn't have to relook at it unless the feature has changed UI validation across several browsers and devices with this technique since you can implement automated visual regression testing you can validate the UI across different browsers and devices it doesn't have to be just one browser and we all know that when it comes to web apps in particular their releases are primarily delayed because the UI needs to be verified across multiple browsers and devices different viewports and that eats up a lot of teams time I don't think engineers should be involved in verifying that manually functionality gets validated too it's not just the UI and returns multiply when the number of languages to be addressed increase as well so with localization the returns are certainly higher if you add four more languages here you can just understand the kind of return you're going to get on a smaller investment so it took me what around approximately 20 seconds to create one recipe via the UI which means it would take me around 16 to 17 minutes to create 50 recipes via the UI whereas it took me just 20 seconds to create 10 recipes including all the metadata that was the taxonomy terms and every users everything so I could create 10 recipes with metadata in 20 seconds so these are some useful references that you can read about the test pyramid the triple A pattern and I have attached few public repositories GitHub repositories as well which is the end-to-end testing with Cypress and Drupal automated visual testing using Drupal and Apley tools so these are public repositories the Cypress Drupal repository and some ways to improve test data management if you already have automated test suites running in your organization yeah and after the conference we'd love to stay connected via Twitter and LinkedIn and of course you can follow us on GitHub as well time for Q&A yes, no it doesn't we've got about 5 minutes for Q&A does anybody in the room have a question? yes, hold on a second for our viewers online sorry I'm relatively new to this subject but I wanted to ask about the example that you talked about when there's either an existing customer that has to add an address or it's a new customer that doesn't have any addresses and you said you would separate that out I'm wondering if you could be a little more concrete about how you would write your tests or manage your data so that you would have to as you say not change your tests when the data changes yeah, so do you see that there is a method called setup there and that's where you take care of new customer creation right and so you're going to have a different test file for new customer so that really depends on how you are going to set up your test file whether the test file is going to consist of all tests for a new customer or is it an address related test file so the setup would differ accordingly but let's consider that we would have a test for a new customer in this particular test file so the setup would consist of creating a new customer on fly on demand and then you would have so you have this last method called recipe listing page right so instead of that we would have a method called test address for new customer or something right and that's where you would add the step for clicking on the button and the steps for adding the new address and asserting it and in the tear down part you just delete that customer right and so you can have more such tests written for new customers in the same test file where you can generate the new customer in microseconds okay any other questions in the room no yes coming hi I know you don't speak about unit test in your presentation but I'm starting in my project to want to implement unit test and I'm wondering if you have a concrete example of unit test in Drupal because it's like on the function level and there is services and function on services but I don't know any other application to that so do you have an example concrete one not in this slide but after our talk we can certainly look at the unit tests right on our machine this is not our machine actually that's my machine any other questions in the room no okay I'll just have a quick look and see if we've got any online questions just go and look at the Q&A tab if I can no we have no questions okay well in that case round of applause and thank you very much