 Hello everyone, with us we have Benjamin Bishof. He's a Test Automation Engineer at Travago and today he would be speaking on the topic of plugins and decorators, Travagers, end-to-end test framework in the spotlight. Welcome Benjamin, the stage is all yours. Thanks a lot for having me. Hello everyone in the world. Let's get started. So this is of plugins and decorators. It's about two major things that we built into our in-house end-to-end test framework and I thought you might be interested to see what we did. Maybe it's an inspiration to you or maybe it inspires you to do something completely different because you think this is not a good idea. So what I would like to do is give you a little introduction, tell you why we decided to make a new framework at all, some of the challenges and solutions and some last thoughts. Little disclaimer here, it's very technology heavy, very cold heavy as well. The cold examples are in Java because that's the language I use daily and if there are any questions I can actually answer them and like I said what works for us might not work for you because good solutions solve very specific problems and I'll tell you what exact problems we had. So that's me. Thanks again for the introduction Benjamin Bischoff. That's my name. I'm a test automation engineer at Trivago. I've been working there for the next year. In total I'm 22 years in tech, 15 years, 100% of full developer and then I transitioned gradually into the world of testing. So now you could call me a software development engineer in test and I write articles on my website software-tested.blog in case you're interested to read what I have to say. On Trivago it was founded in 2005 in Düsseldorf, Germany and it was the first German hotel meta search that lets you search for your hotels based on different criteria and dates and filters. So that's what we do. So you might have seen our product here. That is our main website that you can search hotels on. So you see at first glance that there are a lot of components on this website. There is a lot to do, a lot of filters, so it's rather challenging to test. So why a new framework? When I joined the company in 2016, there was a framework already in place. I will not mention any names because it is still around and still used, but for us it was just too bloated. There were too many technologies crammed into it, too many ways you could do things. It was too chaotic also because there was no real structure to the test. For us, really important, it was not able to handle A-B testing properly because that's what we rely on heavily and it was rather hard to solve company specific problems. So in general, when building a new framework, there are a lot of questions you can ask, what problems should it actually solve, who the target audience is, if there are any dependencies and interfaces, tools that are already in the company, things like where should it run, how should it be triggered, what should the reporting be like and all of this. So we went through all those questions and then did a lot of evaluations as well and then in the end settled on a Selenium-based framework. Why Selenium? Because it's proven, it's a proven browser automation technology. It is compatible with most browsers and devices. Super important for us, it can handle multiple windows and tabs. A lot of of newer frameworks cannot do that, but our main web application relies on tabs and windows. So this is really important and it has the standardized web driver protocol, which makes it very nice to work with. The test framework we wrote is a pure Java framework. Like I said, I started it in 2016 as a follow-up to the old framework we were using. It's based on the latest Cucumber, BDD and Selenium versions. So we can actually write very descriptive tests. Because it's Selenium, it supports various browsers and OSs and mobile devices as well because the protocol that it's using is the same that it's used by say Appium, so we can run web tests on mobile devices as well. And another important thing, it has a plug-in API and this is actually the second challenge I will talk about today. So the challenges. For us, a big challenge was element resilience and hierarchy. And I will tell you what I mean by that. But first we have to look at the Selenium web element. That is something you are probably all familiar with because it's the building block of all Selenium code. It's the representation in code of a DOM element on the website. Typically findable via locators in Selenium that is the buy keyword. So it can be by tag name, can be by CSS, by Xpath, various methods, and it supports various interactions like clicking on it or inserting text. So that's the basic thing. What it does not support is automatic stale element handling. If you are using Selenium on a more complex website, you might have encountered that. If an element flickers, it is reloaded, Selenium might give you a stale element exception. That means that the element is not there anymore. And I totally get why Selenium doesn't handle this for us because it is remote control for browsers, nothing more. There is no real element hierarchy. It is kind of there now because you can chain locators. You can search for locators inside of other locators, but we wanted to have a real parent-child relationship. And a minor point, but an annoying point for us was that there is only one fixed standard timeout for weights. You can use fluent weights, of course, with other than default timeouts, but we wanted to have a more automatic way to do this. So what we decided to do is program a custom Web element. And to understand a little more of what we're doing, here's a short look into page objects versus page components. So page objects, that is a standard way for a web-based test. That means wrapping the page functionality into classes. So you have everything you can do on a page through a single entry point and very encapsulated units of code. If you see the right side, the orange border, that is the page. And on the lower left, you see search page dot search for London. Search page is the page object. And search for is the method in London is the parameter that in this case is entered into the search box. But we wanted to have page components, something that actually is mentioned on the Selenium website as well, that you can use page components instead of page objects. That means wrapping single components into classes. And then combining those components into pages. So in this case, you can delegate the actions through a page to the specific component. Again, the orange border in this case is the component. So we have search page, which is the page dot search, which is the search component. And then the search for methods that is part of the component. That's what we wanted to do. Now, don't let this put you off, please. I will highlight the code parts that I wanted to mention. This is how it would look in our test code. So in this case, the search page extends a page. And the page is our base page. And this page has just access methods to all of its components. Like here, in the previous example, when you do search page dot search, it returns you the search component. And for this, we have the get component method that is part of our base page that is able to return page components. And it can return those components that extend our component class, which is a base component in the framework. A little side note. Generally, we search for elements through data test IDs. So we don't really rely on any other locators. But we have our own data test IDs in the page code. So that makes it much, much easier to access those components. And on the right side, you see the find by with the orange border. So the add find by annotation that belongs typically to Selenium to search for Selenium Web Elements. But in our case, we use it to search for our custom Web Elements. And I will show you later in the code how we can do this. This is the whole component in Web Elements flow. And don't worry, I will try to make it as clear as I can in this short period of time. This could actually be a many, many hours. And I was struggling to compress this down to a single presentation. So let's try to go through this. So what happens, as you saw, we request a component from the component factory. I will show you in the next slide how this works. Because we use Spring in the framework, we also retrieve the component. And when I say component, I am page component from the Spring context. And then to handle the find by annotation that is typically used by Selenium, we apply a custom field decorator. And the field decorator is actually part of Selenium as well. So this is where it starts. Like I showed you, you do the get component, search.class, which would return you the search page component. And we have a factory class that is able to return those components. Like I said, this is where it is retrieved from the Spring context. It's important for us. It might not be important for you if you don't use Spring or any other dependency injection framework. Just ignore this line. The important one is this one. That is a field decorator that is typically used in Selenium to find elements by custom locators. But we use it differently. And I will show you how. The important thing here is that we create this custom field decorator using the current web driver. And the properties we pass to the framework that is typically system properties to have some specific settings. And this is where the magic happens. The Selenium page factory init elements method, this one initializes the elements, the components of a page using this field decorator and using the requested page component. If you have any questions afterwards, I will try my best to answer them. And also, I will try to make this slide set available for you in case you want to look something up. The page factory, that's interesting. I found this quote from Simon Stewart from 2017 saying, the page factory design ain't my best work. I'd recommend using it for inspiration for something better. I have to slightly disagree here because for us it was exactly what we needed to solve our problem with the web elements. Okay. So we have the field decorator and now the magic continues. We are registering a callback on the components to intercept methods. And this is where this happens. This is our custom field decorator that implements the Selenium field decorator interface. And the important part is that this interface has a decorate method. And this took quite some time to figure out. But the class loader here is the web driver. And the field is the variable that is annotated by the find by annotation. So here we check if we have a find by annotation. And if this find by annotation annotates our custom web element. And if it is one of our custom web elements, we add this callback method. For those of you who are into Java reflection, we use the cglip library to do that. It's called an enhancer. And this enhancer makes it possible to intercept any methods that are called on a specific class in our case on the custom web element. And what that enables us to do is whenever you interact with our custom web element, then it is initialized. It's like Selenium works when you do a find by annotation on a web element. It is not initialized yet. It's just a proxy that waits for interactions with a web element. And then it's being initialized. This is done to save memory and to make it faster. And we mimic this. So this is the intercept that you don't have to look at all the parameters. But the way it works is it checks if the object we are calling a method on is of type custom web element. And if it is, and if it is not yet initialized, we set the web driver on it, we set the Selenium web element, we set all the framework properties, and we also set the buy. So each custom web element knows exactly how it was found. Then when we have done all this, we invoke super. That means we call the original method. So this intercept method, like its name implies, it intercepts the method call, does some of its own initialization, and then redirects to the original methods. So it sits in between the caller and the custom web element. And then we execute the methods that we want to call using the initialized properties. Because at that point, we can be sure that it is initialized like a click. And we do that by passing the function as a lambda function. So, like in this case, wait until element is clickable, and then do the web element click, which is the Selenium method, the original one. So our custom web element, because it has a reference to the Selenium web element, uses the Selenium method in the end to click. And because we pass this as a lambda function, we can execute it here. And if we catch a stale element reference exception, which is the stale element exception that I talked about before, oops, I'm sorry. Then we can try to refind the web element, because it knows how it was found. So we can use the same by condition again, and then try to call this method again. And refining elements, that is where the parent-child relationship comes in. We check if our custom web element has a parent. If it has a parent, we try to find the web element within this parent. That is the page component context, the page component bounding box. So when we find elements inside other elements, it's just searching inside this specific DOM element, and not through the whole page, like it typically does. And again, if we catch a stale element reference exception, we try to refind its parent. So this is something that we wanted to have. This is recursively finding elements and parents all the way back until an element does not have a parent anymore. And then we go back down and try to find the web element again. That makes it really, really stable. There it is. And the last thing we did here is what I talked about, the default timeout that you have, that we wanted to bypass is this. So we have find element and find elements for this methods here, but we also have find element directly and find elements directly methods. That means that we don't want to wait. In case we are really sure an element exists or we are really sure an element does not exist anymore, we can set the web driver minimal weight, which in our case is 50 milliseconds. Then try to find the element and then reset the web driver regular weight again. So this dirty trick, that's why I wrote on the right, yes, I know it is a dirty trick, but that speeds up our tests many times. Because especially when an element is not on a page and we want to check that it's not there, we don't have to wait 20 seconds until Selenium says no, the element is not there anymore, but we can just use a directly method and then instantly get this. Yeah, if you have subsequent interactions with the custom web element, so after it's initialized, it's directly invoked without the initialization phase. So we skip initialization and then just redirect it to the original methods. That's how it looks. There's the intercept method again and we check is the custom web element initialized. If it is, we skip all this and we just invoke the original method. So from this point on, it's exactly as fast as the original Selenium code because we don't have all this initialization. It's just something in the middle that does nothing from this point on. And some of you might think, why so complex? Because Selenium, it has a lot of this stuff built in already. Yes, but we want it to be free to use our own custom web element or Selenium's web element, especially in Selenium 4, when there were some more methods available, some more locators like the relative locators. We wanted to be free to use those without having to release new framework versions all the time. So in this case, we can use a find by annotation, but instead of using our own custom web element, we can also use the original web element of Selenium and use the new methods there. In later framework releases, we also added those relative locators to our custom web element, so we don't need this anymore. Yeah, and like I said, we did not want to introduce another annotation to keep the usage very straightforward and also not to confuse the users. And should we decide to dump our custom web element, there won't be many changes in our framework because we can just use Selenium's web element. And a very important point to remember, it is only complex for the framework creators. It is not complex for the users. They don't see what's going on on the inside, so it's very important to hide the complexity from the users. Okay, that was the harder one. Now let's go to the easier challenge, extensibility of the framework itself. This is typically how the tests run. So we run a test scenario, open a browser, in this case, a local browser, perform some actions on the website, make some assertions, and then lock results and make the test pass or fail. But later we wanted to have more functionality. We wanted to use a test cloud, first our own internal one, then a commercial vendor one. I won't mention any names at this point. And we also wanted to send failure messages via Slack. We wanted to queue certain tests for rerun, not all of them, but certain ones. And this is only a snapshot like from 2017. After that, we added a lot more of those arrows and boxes. But you might spot the problem here, direct framework integration of this functionality. It is just too much functionality in the core framework that leads us to the situation we had before, like this bloated framework that does everything. In case of browsers and browser clouds, if you add this directly in the framework, it might be too tightly bound to specific cloud vendors. So for every cloud vendor you might have to use in the future, you would have to add more methods. If you want to add functionality to the framework, especially if you're not one of the core contributors to the framework, you need deep architecture knowledge to do this. And that doesn't make sense to just add a simple thing. And also every additional feature causes new framework releases, which is also not a good thing. So we decided to have a plugin API with the requirements that writing plugins should be very simple. They should be very small, separate projects. They should not require core framework changes. And they should also not influence core functionality, which might happen if you directly insert your code into the core framework repository. And plugins should just implement interfaces that interest them. Very, very small interfaces. I will show you exactly how we did this. The framework should notify matching plugins. Each plugin should be able to have its own set of system properties. And plugins should be activated as needed. So even if we have those plugins in our test as dependencies, we should still be able to enable or disable them as we want. So here we found the service provider interface in Java SPI. It is 22 years old. It's available since Java 1.3. And it did exactly what we wanted. It consists of three parts, a service interface. So that is something that providers need to implement. So it's exactly our thing. A plugin should implement a specific interface. And that's it. Then we need the provider registration API. This is a plugin manager that keeps track of what plugins are currently there. And the service access API. That means we communicate with the plugins. And this is not too important in our case, because this is not something the users use, but only the framework itself. So this is what a basic plugin interface looks like. It has the method get plugin name. That's it. That is the plugin interface that all other plugin interfaces extend. Because we just wanted to have each plugin to provide a specific name. That would just make it easier for logging and also for property handling. And this is the part that makes the magic possible. A marker file. Like I said, this is Java specific. And there might be things like this in other languages. But I'm not aware of those. So what you can do here is to add a file to a plugin that is residing in source, Java, resources, meta inf services. And then has the whole path and class come to Rago plugin extension plugin, which is our main plugin interface. And inside this file, you have the class of your plugin, the main class of your plugin, like in this case, custom cloud plugin. I'll show you on the next slide how this works. And this is used by the service provider to scan matching plugins and find implementations. This is the plugin loader class that just checks what plugins are currently active. And this is the part that makes it happen. Here, the service loader, which is a part of Java util. So it's built into Java. The thing that has been there for 22 years, the service loader loads all the implementations of plugin.class. That means all the classes that implement the plugin interface. And then it has a list of classes. That's it. That is the whole thing that makes it work. And this is one of those plugin interfaces I talked about. This is an interface that's called web driver created aware that extends plugin. So it is discoverable, like I showed. And this passes a web driver instance of the current web driver to all interested plugins. And we use this to take screenshots and also retrieve web driver logs. Here it is. That's the only method handle web driver created. And it accepts the Selenium web driver. And here is what the framework does on the left side and what the plugin does on the right side. So in this case, the framework creates a web driver. And then because this plugin has the web driver created aware interface, it has to implement the handle web driver created method that accepts a web driver. And there's another interface that the plugin implements called cucumber after step hook aware that is triggered when a test step is finished. And in this case, in this demo plugin, it just takes a screenshot and stores it somewhere. And you see on the bottom right, it uses the web driver that was passed in the handle web driver created method. Oh, I'm sorry. There we go. Those are all the plugin interfaces we currently have. And there might be more in the future. But there are some that are for web driver handling. There are some that can react to certain properties to certain cucumber events. A lot of things. So what happened through this new mechanism is all of this turned to this. So we still have a very straight line, the white boxes. But here the first step is the plugin discovery that I showed that registers the plugins in the plugin manager. So the plugin manager always knows all the active plugins. And everything the framework does, it can tell the plugin manager, hello, please notify all the web driver interested plugins or notify all the plugins that want to know when the web driver is destroyed or when the properties are available. And then the plugin manager communicates all the plugins that are in the plugin manager. And then the plugin manager communicates with the magic plugins. And this is how we can inject all kinds of functionality here. I'm aware this was a lot and this is rather complex. So let's give me, let's give you some last thoughts here. Some of you might ask, why didn't you make a commit to Selenium itself? And this is because refinding elements is out of scope of Selenium. Like I said, it's a remote control for browsers, nothing more, nothing less. So this refinding and custom web elements stuff, this is way out of scope. And it's very specific to us. It might not be something you need there. The Selenium web page actually says it, Selenium automated browsers, that's it. First, the problem than the tool, that is something that I had to learn the hard way. It is absolutely crucial, you understand the problem fully first before jumping on any tools or any solutions. It's especially bad if you have a solution already before you know the problem. And you have to evaluate the tools in regard to your problem and you have to also reevaluate the tools regularly because requirements might change and tools might change and there might be better tools available. Know your tools. Before switching to other tools, you should always check if your current one is able to solve your problem as well. And if it's less effort to do so. And it can be very valuable to dig deeper, like we did with the whole page object handling and the Selenium internals that solve exactly our problems. Also don't dismiss the old solutions. One very important point for me is decoupled progress from technology. You can make a lot of progress without switching to newer technologies. Because old tools oftentimes mean proven tools and bug-free tools. And if you have older tools, you have a better chance that more of your colleagues know about them instead of jumping on a new tool that might then it might be only you who knows about it. And that creates one of those buzz factors or like one of my colleagues likes to call it tractor factors. Like when you know about a new tool and you get run over by a tractor, there's no one else that can maintain it. And reinventing the wheel, it's often said it's bad but I say it's absolutely crucial because not all wheels perform the same. They have to complying with the terrain, with your specific problem. And if you recreate a wheel, you learn a lot about wheels. And with this, I want to thank you. Hope there was something in there of value and ready for any question you might have. That was an insightful session. So we have only one question from Bhargav who says, why CSS is stored in string and passed to at the rate fine by in later steps? Any reason behind this approach? Can you say that again? I didn't quite get this. Bhargav asks, why CSS is stored in string and passed to at the rate fine by in later steps? Is there any reason behind this approach? Yes. There is a reason behind this approach. This was just a sample code. But in our real tests, we have a lot more locators, a lot more selectors that we have to work with. So instead of having those all over the code, we wanted to have them in a central place, which is the top of our classes. So we can make changes simply by looking at the top of the class and then changing the string. Instead of going through the whole code and then checking where do you use this and where is it defined? So everybody of us who work with the test code, they all know where to find those strings. It's always the top of the file. It's just convenient. If the attendees have any other questions, you can post it in the chat or the question and answers tab. I think there's another question by Karthik who says, how can we learn more about building better architectures? That is a very good question. And I can tell you, please read about design patterns. Please read about code smells and just try together resources that are typically aimed at developers to make your code and your architecture better. Because that is one of my things, one of my missions really to promote better code in testing. Because I see a lot of bad examples, especially on platforms like LinkedIn. People who say, I wrote this new framework. And if you look at the code, it's just a bunch of libraries thrown together without any thought. So yeah, you should, if you want to improve your architecture, you should definitely check out resources for developers.