 Hello and welcome, everyone. Thank you for joining in myself, Karan Kural. And I welcome you to the session on Become Black Ops QA with Selenium 4BD DevTools API by Anton Angleff. Thank you, Anton, for joining us. And we are glad you can join us today. All right. After this first delay, I hope that you already enjoyed your coffee and we are ready. So as you can see from the title, we are going to talk about new Selenium 4 and CDP. And as you know, many browsers provide a set of tools that use the work of developers. They are also known as DevTools. For example, Google Chrome developer tools use the Chrome DevTools protocol called CDP short, which can be used nowadays in Selenium 4 tests. And many of its features are marked as experimental and often deprecated quite fast. And as you know, the protocol is not very stable and some of the features depend on the browser versions, as you will see in coded demos. So also, as the name suggests, it is available for Chrome browsers only. And if you want to use it with Firefox or Safari, you cannot. So to solve these problems, WebDriver maintainers have started working on this next generation cross-browser browser automation-slash-tasting API called WebDriver Bidirectional Protocol. So it is a better idea usually to couple your automated tests to a specific browser version. So it's recommended to use bidirectional APIs whenever it's possible. So you should be careful with the CDP usage in your tests. However, it might be useful as we will discuss. And yeah, so keeping all this in mind, let's review the endless possibility of these two APIs. And also in Maya, I'm Anton Andriov. Maybe many of you know me from my website, AutomatedPlanet.com and read some of the many articles or from the YouTube videos on the landing test channel. Also, maybe you own one of the books about design patterns, about high-quality automated tests. So one of these, I have my own company. We are many people working on enterprise, big-scale projects establishing test automation. And as a hobby, I write books and talk to conferences. So here is our short agenda. First, we'll talk about geolocation testing, globalization, localization, time zone, and other types of testing. First, we will see examples how to do this via CDP APIs. Next, we'll investigate the bidirectional APIs and how we can use them to troubleshoot tests and improve the overall maintaining process of tests. We'll talk about, for example, the PIN script method, collecting JavaScript errors, DOM mutations, console walks. And the next big part is related to capturing and monitoring networking traffic. And I will show you what type of validation we can make. First, all of these are related to simulating, yeah. So all of this is related to simulating networking speeds and other parameters. And also, we'll talk about what is fast and what are the proper metrics to work for and how to collect them using CDNM. Let's begin. First, a little bit of theory and then we'll move to code examples. Because, you know, you need to understand the different types of testing we can utilize the CDP. Yeah. So probably you have heard the terms internationalization. It's like also called globalization and localization testing. Where they mean different things, I mean the localization and internationalization. And let's review them and later I will show you how to do these things with CDP. They have the same goal to ensure an exceptional user experience for global market users. And if we need to give a definition to localization, it's about adopting the UI and the content for a specific region and look out. And also it takes into consideration the region's cultural and linguistic specifics. And during this type of testing, we may verify translated text, addresses, format sequences, keyboard usage, visual lines to the culture, time slash currency formats, alignment with data. And if we need to manually perform testing, we can use the DevTools as depicted in the image. You can change below here the coordinates, you can change the time zone and the look out. You can do that from the sensors view in the DevTools. Here is another example, an AliExpress user can choose the preferred language and the page contents are translated into that language. This is where localization comes into the picture where the contents are localized to meet the requirements of the specific culture and region. And this is an example of AliExpress in Japanese. And what about the globalization also called as internationalization testing? It focuses more on product features and capabilities that allow the app to be used globally. For example, be able to change the language of the website, the previous AliExpress example. This you may check are support for multiple languages, support for different time zones, support for various number formats, text in other formats like left to right, right to left. And another globalization feature of the website is offering personalized items based on where you're located. And of course, time zone testing is important. Here I found an image. Here's an example for Kindle, Amazon Kindle come down where for a short period of time, you get some discounts, right? And here is another image. However, this is related to the time zone of the user. I took the previous screenshot with U.S. time zone set while the following one with Australian time zone configured. And Canon Bear Australia is 16 hours ahead of Washington DC, United States. So the deal wasn't yet active there. So time zone testing is something that you need to consider. And lastly, if we need to compare localization to globalization testing. We want to test that everything continues to work when we change the settings. In contrast, we do localization testing to verify that the functionality usability interoperability and other testing methodologies to build a product with global appeal. For example, in internationalization testing. If a linguistic accuracy has a lower priority, then, yeah, you can ask your questions in an agent of the talk will discuss them. Yeah, so for example, in internationalization testing, linguistic accuracy has a lower priority than finding issues in product design that could impact globalization capabilities. On the other hand, this is top priority while doing localization testing. And last thing is the name is just a location testing is a type of testing where you perform tests on your website from IPs from various countries worldwide. And here are some of the features that can be tested with Gail location browser testing. The first one is called go fencing, which is a method to send notifications to your users through mobiles by leveraging the geographical geographical area specified by the user while using your web application. For example, last time I traveled to Munich, Germany, I received a notification for my booking complication regarding local places to visit. Another one is called gail tagging from the left, which allows you to put geographical tag over social media elements such as photos, videos, cure codes, etc. And using gail tagging, you can implement geographically based authentication through QR code identification or on your web application. And the last thing is called gail blocking. And it's used to apply restrictions to your web app concerning the walls and the norms of a country and many online streaming companies such as Disney, Netflix, HBO and more use gail blocking to deliver rich media content based on their country norms. For example, you may have seen the current image while accessing the video through a thumbnail on YouTube. And let's see an example of how you can do that with CDP. As I said, you can do that with chromium. So this is Chrome driver edge driver. We can go first, we need to go first, get dev tools, and then we need to create a session. And after that, this is Java. So in Java, we have this send method where after that you need to use the particular domain, whether this is simulation network performance security and then this particular domain classes they give you access to the various methods that you can use. Later in a minute I'm going to show you the documentation. So, at least for setting the geolocation, you have three options here. The first two are really important. These are the longitude and longitude. In Java, we use this, there's the case is written in a way where since you can miss some of the parameters that can be empty, sometimes you may use optional empty or optional and then set it. So today from the emulation domain, you can set the time zone and the local, you can find the list of all of them on Wikipedia, for example. And there are two places that you need to check. The first one, of course, is the official documentation. And as you can see they are separated from DevTools and bidirectional API. We'll talk more about it, but as I said, CDP or Chrome DevTools, it's coupled to the Chromium engine. So, yeah, basically it's available in Chrome only. While the bidirectional API it's available on, it should be available on browsers. And as you can see, there are not so many examples really. Here, we have only the maybe most used of them. However, as you know, we can do many, many things with CDP and maybe the best place to check what are the available options because maybe if you use it in the past, you notice that there are so many methods in this particular domain. And what are the domains? This is, by the way, the official Chrome DevTools GitHub page or documentation. And here, as you can see, there are many, many different domains. This is actually the emulation domain. And as you can see there, like, we saw three examples and there are three things in the Selenium doc. However, there are so many other options here. And it's the same for all of the different domains. And this is how actually while I was working on my new book, actually I was experimenting with these different options that we have here. However, this is an excellent place to see what is set as experimental because when it's marked as experimental, this means that it can be deprecated soon and I mean it's not reliable to put it in your tests. And when you click on them, you can see like a description what this is doing and there is a description for some of the parameters. Sometimes they're not really described well, so you'll have to guess and try it. In a couple of examples, I looked into the official Chrome source code to find out what these are doing. But yeah, this is something that you need to check. But this was related to if you run your tests locally on your local browsers. However, as you know, many of them in our, you know, when we are on our testing continuous integration usually we use some kind of Selenium grid, no matter whether it's a local grid, or you run them in Docker or you use your or you use Docker based grid or commercial platforms such as slender the source labs browser second others. There's no problem now to use them in slender for you just generate the capabilities from. If it's a commercial grid, then they have this capabilities websites that they can generate the capabilities for you, you need to be careful there because in order to use CDP. Of course you need to use chromium but also they have this flag where you enable the CDP and also. The other thing that you want to use Selenium for that there is a version of Selenium. And then the another important thing that you need to do is to initialize this augmenter class, and then co augmenter augment driver, this is basically will improve the base of the remote web driver and it will add the capabilities of calling get DevTools. Otherwise the code after that is the same. And let's see an example for the geolocation testing before moving. I was, you know, all the code that I'm going to show you actually it's open sourced and I did. And I'm recording almost each week videos for another test YouTube channel regarding Selenium for and we have all of these free videos there that they're really won't come all the subjects that I will touch. I won't be able, you know, because they're like separate lectures of them. And during the lecture about geolocation testing, I use this website to demonstrate the geolocation testing. It's like a simple GPS coordinates website that basically when you press this button, it will detect your current location. And then you can provide a second location and calculate the distances. This is just for for the example of using these sensors about reading the GPS coordinates, however, many often this modern web applications, for example, the Starbucks app. They have Starbucks here in Bulgaria and you know they they have on their web store. They detect where you are and they show you like the nearest shop. And if you have to test something like this, you need to be able to simulate the coordinates right and the simplest way to do it with is with Chrome DevTools you can change the coordinates. And many often in this enterprise projects that I'm involved we do exactly that. So, in order to demonstrate that, let me show you some code for the geolocation again. This repo lambda test Selenium for Java tutorial you can call it from GitHub. The geolocation testing. The thing that they did is like in the in the first test in in here I use Maven J unit. I start the Chrome maximize it and then in order to simulate something I create a session, and I use the simulation set geolocation right. So the thing that I showed you before, the interesting part is here where I created the data driven test where basically I provide different coordinates for the different capitals of the cities around the world. And I calculate the distance between me and this capital. And I have here the predefined distances. And this test will be run for every of this particular sets of coordinates. And as you can see every time I'm initializing the new Chrome. Then I'm creating a new DevTools session. And I'm overriding set the geolocation override. And then I just, you know, go to the example as I told you basically, I click the button that that detects where I am. And then I click distance and check whether the correct distance was calculated or not. This is not really so important what exactly we do the Selenium. This is the important part where we can we use it. That's what I wanted to show you. Now, trouble shooting as a tool creator for me, maybe one of the most exciting new functionalities that even you cannot find in the Selenium docs but I'm going to show it to you. It's about executing JavaScript on every page without using the JavaScript executor every time. Like I'm the creator of maybe one of the biggest C-sharp open source frameworks using Selenium. So in the past, it was still doable with prior Selenium 4. And we had this functionality score for example, highlighting on user interaction in the browser, which is quite useful, especially when you need to troubleshoot your test because you are seeing what your script is doing, especially if you're recording videos. Another useful thing that over time we found that it's really helpful are these toast messages. What are the toast messages there? Sometimes these pop up messages are called toast. And the name is because it pops up from the bottom of the user screen just like a piece of toast. And I'm going to show you how to implement this tool with JavaScript. And actually, we are going to use Selenium 4 to integrate it into our tests. As I said, it was possible before, however, now it's easier with the bidirectional APS. Still the maintainers can correct me really. I'm not completely sure because it's not part of the documentation whether this is CDP or bidirectional. I think it's bidirectional API because really you cannot find such a method in the CDP docs. So here is what I'm doing. First in the beforeage, again, here I know I haven't organized everything in a framework or library. It's just here to ease the example. So we create a new session of the DevTools. And from Get Domains, you can get the domain JavaScript. As you can see, this is how I presume that actually this is bidirectional API because you don't send this commands from different domains. From JavaScript, you have this pin method. And the thing that this pin method will do is actually that we are going to reuse this JavaScript and basically WebDriver will initialize every time when you want on a particular page, which is awesome. You don't need any more to want big scripts, etc. every time with JavaScript executable. And in order to implement the highlight here at the bottom of this reusable script, or you can pin it second time, it doesn't matter. I created this simple JavaScript function called highlight that will accept the font element that we located with WebDriver. And we get the default background in the default border of the element. And then we set a new background color, which is this red border. And then we use this native setTimeout function that accepts a code back. And basically, the thing that it will do is that after one second, it will return the element to the original background and border. And later, if we want to highlight something, you just execute this highlight function, which again accepts the element that we found. And as you can see here, through the JavaScript executor, we execute the script calling this already defined from the pin script highlight method. And then from arguments we're providing the element that we found. And this is how when you execute your test, this will highlight it and then unhighlight it in one second automatically with the JavaScript. And then, then as you can see, I have another method that in a minute I'm going to show you how I implemented it. I use this library. It's called getHepiredFundit. It's called jgrowl. Basically, in order to use it, no matter where, you need to have a link in the head to basically to its script and CSS. And after that, through the dollar sign from jQuery, you can call jgrowl and provide some message and it will pop up. So why this is useful? This is useful again when, for example, you have a problem with some of the tests, you can enable this type of troubleshooting. And I found it especially useful when we have these big enterprise projects, we usually set up heavily a lot of data with internal APIs. And this is really useful to show what we created or what we typed in a form or how you found an element and stuff like that. So this can be useful. And you can see that later during the video recording or screenshots and failures. So in order to implement that, again in the in-script, I do a few stuff. This is the part for jgrowl. On the window on-volt event, I subscribe this widget here. And the thing that I do is I check whether the website has already jQuery. If not, I create a script stack. I basically change the type to JavaScript. For the source, I got this link from a jQuery official website, which is their CDM. And then to the head of this particular webpage, I'm appending the jQuery JavaScript stack. So if we already have a jQuery, we assign the door sign that is popular for jQuery and this is how we use it. We wrote the script for jgrowl. And the second thing that I do with jQuery is that to the head, I'm appending the CSS stylesheet. And later, we can use this method here through the JavaScript executor, I can refer to jgrowl scripts. And create these toast messages with the message that I provide. And also here, it's up to you whether to put some kind of a slip if you don't want to pop up these toast messages too fast. And how we hook all of this to WebDriver? Well, we can use the event firing decorator, which is like the nature of extending WebDriver. You need first to initialize this WebDriver listener. And then you have so many other methods that you can override. But basically before or any element is found or called, you can highlight and send a message. As I said, there are many of these points where you can execute that. Or as I said, you can do this by hand, but I usually prefer to have an automatic way in the frameworks and libraries. And then we just add the listener to the event firing decorator instructor, and we call the decorator to add this to the original driver. So this is working for remote WebDriver as well. Just don't forget to call the augmented. That's all. And then other troubleshooting stuff for bidirectional API. So here I use some of the bidirectional stuff on the same example with the coordinates. The first one is that you can find the documentation is called HTML DOM mutations. And basically the thing that this code will do is that when there are changes to the HTML DOM, you will collect all of these changes in a collection and later you can do something with them. Why this is useful? Well, when you troubleshoot websites that have a lot of JavaScript involved in, for example, injecting ads and changing them over time, this is my example of how I use it in the past. I saw what was injected, whether it was blocked or not. And yeah, how we use it, it's really important for all of these bidirectional APIs that you need to use synchronized list, not a simple array list, at least in Java. Or in your particular language, you need to use a thread safe collection, because everything is happening in parallel of these DOM mutations or requests, etc. So you need to put them in this synchronized list. And then usually, in some kind of a way, you have these listeners. In this case, you call unlock event, and you use this lambda syntax to add it to the list. So basically, almost the same for collecting the JavaScript exceptions. This is awesome feature where it's really important to watch for JavaScript errors, especially now when all of these asynchronous websites, they heavily use JavaScript. Many often, there are some problems between the dependencies. And even if you don't see like a functional program right now when when you execute the stuff, sometimes there are problems with other related third parties so it's good to watch for. And the same thing about the console messages. Basically, you call this app listener and then walk entry added, and then we add them to this synchronized list. Later, it depends how you want to play it, I usually print them on the console and this means that when you execute them in Jenkins with corrections whatever you're going to see all these exceptions and why the console walks are important because many often you will see like warnings like this, meaning that even if you don't see an error, this doesn't mean that you should ignore all the warnings and that you don't need to notify the devs that they need to change something. Or if you use a more sophisticated framework, you can even fill the tests if there are any JavaScript exceptions. Yeah, so many new capabilities regarding troubleshooting. Let's return to the slides. The next thing that's the next big part is about capturing and monitoring network traffic. In the past we use proxies to capture and modify the edge traffic. However, if you don't care about the browser, it might be even much easier to leverage the power of CDP. And this functionality opens the framework to be used in broader range of automation scenarios like security testing. For example, in your event handlers, you have access to all basic HTTP request response properties you can examine inject or do whatever you want with them. You can use them for performance testing. You can use the HP event handlers to get file sizes you want to send or receive and even build regression tests to ensure you know your files don't grow beyond the specific size. In a minute I'm going to show you an example. And another thing that you can use is the so called black hole proxy pattern. It tries to reduce the test instability by getting rid of as many of third party uncertainty uncertainties as possible. And modern websites have a lot of third party content loaded on every page. There are social networking buttons images coming from CDS tracking pixels analytics and much more. And all these items can destabilize our test at any point so black hole proxy takes all HP requests going to third party websites and book them. This doesn't mean that you don't need to test them, but you need to test them in separate tests. And of course, part of your location testing is to simulate how your website was using slower internet speeds. And this is especially important in the case of mobile web testing, and we can use the methods provided by the network demand that you will see in a minute. And of course here from network emulation, something really, really important. If we return to the example of the example of Starbucks. Actually Starbucks website is one of the good examples for progressive web app. They're totally different. They're totally different lectures about progressive web apps but in short, because this is a location testing. You need to reuse the code base and basically provide you the working feel of the same app, but on mobile on desktop, and on the web, and they need to behave like like a real desktop app. And this means that basically how this works is that they use this native new capabilities of HTML five and behind the scenes they are fetching all the resources and this means that they can work. And with network emulation, you can basically turn off the Internet and test how whether this is still working. And let's see some examples for, for example, capturing HP traffic. So here are some examples. Here I use most of the time CDP. One exciting new bi-directional API is that you can register a basic authentication. You can find that on the docs. Another bi-directional API is about network intercepting. You can use that for security, for example, you can add headers, you can set statuses. It's pretty good. In order to implement the black hole pattern, this is really easy. You need to send the enable network command. I know this is ugly, but this is how it works with this optional empties if you don't set them. And then it's really easy because you have this network from the network domain set blocked URLs. And here through Ragex expressions, you can provide full URLs or disable all particular resources. And later you can add listeners, for example, to check that something called failing can etc. This is how it is easy to implement the black hole. Another really exciting feature or capability of CDP is smoking API calls. Basically, when adding a listener, when you enable the fetch, you can pause a particular request and then you can change it. Why this is important because many of these modern web apps, they have this front-end calls to many third parties. For example, e-commerce websites, they use these payment providers. And in the past, something that happened to me is that these payment providers PayPal or others, they charge you for, for example, calculating the different taxes, right? And even in one case, actually, this taxes API for the test environment, it just returns every time white random numbers. And as you can guess, when you receive a random number for tax, you cannot basically verify anything. So we had to implement this mockup server returning hard-coded values. And however, back then we didn't have this capability of CDP. So we had to our developers, they put it on the test environment, they just changed the payment provider with this mockup server. But right now you can do that with this capability of CDP, or you can even modify. Also for these modern websites, you can verify, you know, these progressive web apps, they push notifications. You have this chat application so you can verify them as well. You have this capability for web sockets and event sources. And if you need to capture the HP traffic also, you have this listener for responses received. And again, in synchronized list, we can put all the responses. And when you put all of the responses in, you have the list of them. We can create such assertion methods where we can check for that there are no error codes, that particular request was made. This is especially useful for checking analytics calls with particular data. Actually, in one month ago, we had such a use case. And this doesn't mean that you don't need to integrate with, for example, Google Analytics stuff and check that everything is there, but anyways. And also, you can check for no archimages if you enable the compression server, this is excellent way to verify that. And for progressive web apps, when you turn off, you know, the internet, you can disable the cache clarification, check that everything is received from it for the progressive web app. And if we need to emulate the network, just really quick, if we talk about getting metrics, maybe we won't have so much time left to discuss that. Anyways, regarding the emulation, as we discussed, it's really important to right here, you can enable the network again, and then call emulate network conditions. And as I said here, if you check that to true, it will turn off, we will check how your web app is behaving offline, and you can change the download and the upload speed and change here whether this is 3G or 4G. And then in order to get performance metrics, you call this send performance get metrics and we print them. And this is a full list of all of them. But yeah, anyways, keep in mind that most of them doesn't mean anything to you, but really this five here, they're really important. And maybe this to hear about the many people forget about them, but actually, this is related to if you use them, you can catch like memory leaks and this is actually something that we find out during testing of one of a single page application that we're working on. These three are related to performance. Again, their whole video about that on YouTube that you can find will send it as a resource. Also, you have this matrix, you know, in the commercial clouds already with Google White House about the performance so you can use that as well. And you can check the website of Google. This is the last thing that I'm going to show you. It's called Web Dev Matrix. In here you can find all about this matrix that Google team defined. It's really interesting. Again, you can watch the whole video about the performance matrix. It's something interesting and you can use these capabilities of CDP to reuse your functional test for performance as well. So again, regarding the resources, I will pull out the presentation. You can download the full source code on GitHub. You can check all of these YouTube videos on the LandTest channel. Also on my blog, on Ultimate the Planet, I've talked about a few of the stuff. And of course you can check the Selenium docs and the DevTools documentation that I showed you. Thank you. Thank you so much, Anton. That's a really good session. We get to know a lot of things in the session. So we have a lot of questions, although we want to finish and take all the questions now. So what we'll do, so those questions, those people who are left with the questions they can discuss with Anton in the Hangouts table. So we're going to take top two questions. So I'm going to read it out for you, Anton. The first question is from Kartik Hola. Does CDP work only with Selenium 4 or can this be used on older versions? It's Selenium 4 and above. You cannot use it with, for example, Selenium 3. This is something related to the new release. New, new. It was like half a year ago, so it's not so new already, but yeah. Sure. I hope it answered your question, Kartik. The next question is from Gaurav Mamgen. In case of running geo-location test cases over cloud platform, which can be hosted over any time zone, would it be any issue with DevTools? Again, it really depends on, if I get the question right, regarding the commercial clouds. If you use them, they have their own capabilities usually to change the time zone, so it's better to use the native capability of the cloud grid. And also, this is related, I forgot to mention that, but related to geo-location testing. It depends on which of the clouds we use, but basically, if you don't use the CDP to change that, actually they have built-in way to run the test exactly from particular IP, from this particular country. This is not an emulation, but it will be from the real IP, because many often in some of the projects that I was working on, actually not all of the apps use the native sensors for detecting the coordinates, but sometimes they use these paid third parties for detecting the IPs, and this is how they do it. So this won't work in every website. And thank you, Anton, for this lovely experience you shared with us, really.