 Hello everyone. I guess we are about to start as it's already 15. So yeah today's topic is how can QA and developer teams work together and how they can improve the automation test coverage and overall the whole experience in a project. So I will quickly go through the agenda. Of course I will do a brief introduction on myself. Then we are going to establish a base one and actually talk more about what are the requisites of having a good QA and development team relationship. We are going to talk about a bit for planning action and delivery of our automation test. And of course, as I like every time to show some examples, I will give you some of the problems that we faced and how we actually solved them together. And last but not least, we will have a QA session. So yeah, starting with the introduction. My name is Tony and yeah, I'm a senior automation lead at FFW. This is me in office situation. And this is again me keeping my cool when things go sideways. Because I think it's important everyone to stay calm, no matter what happens. And in order to act as the best of our abilities. Because when people start to get nervous, get over excited in a bad way, they start to think sideways and we are not delivering the best. So and our clients of course get disappointed. So a little bit about the company I work in. It's called FFW. We have over 800 employees almost all over the world. And I'm from Bulgaria. And yeah, I've been in the company for 10 years now, actually turned 10 years on 1st of October. And basically, this is my first IT job. So my whole career is based in one company, which for some people might, yeah, might be a minus because you have to go around, see how other people are doing some stuff. But I believe that this helped me prepare and be with a good connection with the people I work with. We've become not only colleagues, but friends. So I believe this is one of the key points to have a good results. So let's move on about what we need to establish first when we gather a new team and or we start as the same team on a new project. So first of all, having people with the right mindset, I believe is the key. Everyone should have the same focus and the same goals. And it's really important on both QA and their side to have everyone on the same page. What we are doing, what we are going to deliver, what is clients main goal and what if the project that we are working on the product we are building, if it's aimed at end users, we need to understand how the end users are going to use it because actually we discussed this earlier during the coffee breaks with some colleagues that not always the developers understand how and why we are building what we are building. Actually, we discussed with the meter here that there are two types of people. One of them are strictly following instructions. And yeah, don't think about how actually end users are going to use it. And it's important to think about the thing that we are doing. It's important to put ourselves in the perspective of the end user. And actually what we are doing, is it usable? Is it really practical? Or we are just focused on doing our job and getting things done. So why we need to have QA and dev teams united is for me the most beneficial thing is that we can face the clients and defend ourselves when we need to do something. For example, spend some time on developing QA tests, automation tests, introducing new tools, etc. In order to be more convincing in front of the client, we need to be united. And to be united, we need to have the proper mindset. So in order to bring the developers on our side, as I'm a QA, we need for me to let them know how actually the tests works. No matter what framework we choose to use. I think we need to spend the time to show them how it works and best case scenario to have them set up the framework that we have locally. So just to give an overview of the audience here, how many of you are developers? So most of you, I guess, are developers and I guess you are interested in how to achieve things more. So why this is the question? Because a couple of years ago, in my experience, developers, most of them didn't actually understand why we have the tests, why we have automation frameworks, etc. And I believe that by spending the time showing them what exactly is the purpose and how actually the tests are helping us in the long term by spending a couple of hours, couple of sessions together to show you, I mean, to show you from QA side to show the developers what we are doing. Of course, they will, developers will be a bit more interested in this topic because it's kind of technical. It's not so, for example, some scrum stuff or some other stuff that are purely theoretical. And by showing them how and why, they will start more to think about the work that they deliver, how they can help us case deliver it better. So one thing I mentioned, and it's on here on the slide, it's best if developers have the automation tool of the team set up locally. So when developing new features, for example, or a major change to do a local run and see if they're not going to push something that will break, for example, the whole development environment. Of course, it's not always the case. It's not always possible. But this is the goal that we need to aim. So about planning and good results. Good planning means good results, of course. But what actually is good planning? First of all, we are, I guess, most of us work with stories. And when we are on a planning session, and we start discussing specific story, first thing from QA side, I believe is to discuss with the developers if a story is automatable. I don't know if this is a real word, but let's use it. So we should aim, of course, to automate all the things. But of course, it's not possible. So it's really important to decide and scale. Is it worth it? Because let's say we are developing a certain functionality, we will spend a lot of time automating it. But at the end of the day, is it going to be really the most crucial functionality that we are doing? Or there are more important stuff like, for example, imagine you have a social network and you have a registration page. Of course, this is your most important thing on the site to work. But then you have block pages and some search, which is more important. Here you need to think like the end user. Is it more important from the end user for the registration to work or to have the search of the block pages? Because on social networks, not many users read block pages. And this way, you need to structure the most important things, start covering them and go down to the less important stuff. One more thing that is also important during these planning sessions is to estimate both the manual and QA work. As you may know, QAs are working. There's two main differences between QAs. Manual and automation QAs. And it's important to estimate both of the work. Because sometimes it may happen that the manual work is much less than the automation work or vice versa. And one more extra step is to have split between the QA members of the automation and the manual work. What I mean is that, of course, if you have more QA members, sometimes you have only one. But if you have two or more, I think it's important to one of them to do the manual work and the other one to do the automation work. Because this way, you are doing a cross check and everyone can make a mistake. Everyone can miss something. And when you have two people different working on the same thing, you will, of course, catch something that the other guy missed. And I've seen a lot of examples of bugs that are not reproducible manually, but the automation tools, as you know, they behave quite different. They catch some issues. And yeah, when you discuss on this planning for a story that needs to be automated, but let's say you are pressured by the time and you can do it within the current story scope, of course, you can go ahead and move it away as a work and have a separate backlog of issues to handle in terms of automation work. And what I believe is the right way to go is to always bring some of this work into your sprint or whatever time frame you have defined to work with. But always, always take some of this work because this way you are going to constantly improve your coverage. You are constantly going to deliver better product for the clients. And this point is not only related to QA and DEVS. It's related to all the team members. They all need to push with suggestions. Yeah, Siri activated. And it's important everyone to be aware of how actually the automation tool works and what can be done in a specific example. So in our case, in my experience, we even had the clients understand how the automation tool works and they are giving us suggestions. Do you think this is a good item to be automated or something like this? So, yeah, I don't know if I'm ahead of time, but as I said, I really like to talk with examples. So I will just give you a couple of real case issues that we faced and how we solve them with the help of developers. Of course, I'm very grateful to all my colleagues that spend the time to give us solutions because, yeah, not all QA guys are so technical. Of course, developers are working all day with technical stuff and they have a better understanding. So one thing we introduced during last couple of years is having all our tests running nightly. So at each morning, we have an overview, a report of what's working and what's not. And I believe that this helps us keep even the deaf environment stable enough. Of course, there are always red flags every morning, but it's easier for the QA team to have an overview, send the proper feedback to the developers and get things solved as soon as possible. One other thing is, as you know, when you delete a user in Drupal, some content is deleted and some is not. And when we are running our automation test, we tend to leave no trace behind, meaning that every content created by the users that the automation test uses needs to be gone. So we faced a problem where you delete a user, but of course, there are items like terms and manual items that leave behind. This made us do a custom QA delete module that was, of course, developed by our deaf team. But this module tracks all the entities that are created in the Drupal by every user. And then when deleting, we added actually an extra option for cancelling user, which says delete FFW, QA, delete content and user. And what this module is doing is actually deleting everything, not only the notes, but all the entities that are created this way. After the test run is finished, there are no test items, no dummy content, and nothing is left. Of course, this was done with the purpose of keeping our development environments clean, not having so much dummy content. And this will also prevent doing mistakes if we decide to run the test on production, because we don't want to have any test content life. One more thing, as you know, Drupal has weird ways of naming fields, etc. really, really long selectors. And as you know, IDs are the best way to end the fastest way to select an item in terms of automation tools. So whenever and needed, we asked our kind developers to add some custom IDs so we can use them instead of the long Drupal selectors. Of course, there are cases where you can still use the ones provided, but you can't have a case where you don't. One more thing that we faced as an issue was a capture. Of course, it's created to avoid bots, and when running automation tool, it's actually controlled by external, no matter if it's a Selenium or any other driver, but yeah, it's hard to bypass captures its purpose. What we did is to create a custom endpoint where when you hit it, it sets a cookie and in our dev environments, when you have this special cookie, which is randomly generated in each run, the environment knows to bypass the recapture and move forward. This way, we can submit forms that have recapture and other and use it on other places where it's used. So and one of the other things we implemented and faced as an issue was working with dynamic data. Imagine that you have an eBay search listing and constantly the items that are listed there are changing and they are coming from outside source feed or something like this. You're just consuming the list of items. So each day, you don't know what is going to be there. How we solved this is we created our internal API, which the automation to cause stores the data and then runs the automation to and starts behaving like a user and it knows to expect certain items with certain titles so it can search for them and expect them as a result because and also you can have some properties like in this example, color, series, etc. And this is also very usable in live productions sites when we use when we're on our smoke test after each deploy. We use this API to gather the data because we of course cannot create dummy data on life and we tend to not do it on our dev production because we have external feed in our case. And I think this is a really good solution for dynamic content. So thank you very much. And yeah, no, no, the question is not for me. The question is from a sum. How can you unite the tools to use for both sides developers in QAs when they are different for this? All tools or the automation tools? Yeah. Yeah. Yeah. Maybe you can. How can you unite them? So you're saying that you don't want to have an extra tool in your setup? Yeah. Well, how can you unite it? Maybe you can work together. Currently, in my case, in my experience, we just have it set up extra. Because the developers usually you maybe use Docker or something else and everything is there already. And maybe you can work together to introduce the automation tool of the QAs also in the Docker and have it there in place and have it installed already when you spin up a new container. Yeah. And maybe just want something for me because I have been a developer and he has been my QA. I know very well that, you know, when we choose tools, we try to choose tools which are, you know, with the same background like we use Behat. Behat is using PHP, right? And Drupal is using PHP. So yeah, we very often have our developers able to use to write Behat test. And now with the JavaScript frameworks, it is it is the same. So at the end of the day, we speak the same or write the same language. So no other questions here. Thank you very much. Yeah, thank you.