 All right. So hi all, thanks for joining in. We have Ivan today here and he is going to help us actually master our Selenium Tests. Basically, that's what his talk is about bullet proofing the test, Selenium Tests, and he is going to help us deep dive in. And this session is brought to us by the Sauce Lab, so we'd like to thank them. So the stage is yours, Ivan. Okay, thank you. So my name is Ivan Krutov. Probably some of you have already seen me like two years ago in Bangalore. Unfortunately, this year is not possible to meet offline, so today I'm talking in online mode. And today we'd like to talk about the bullet proof Selenium cluster. So this will be like a live demonstration how to easily create a bullet proof Selenium cluster from scratch and start running your tests in a dozen of minutes. So a few words about me. Being a software developer during the last decade, my main experience is related to Java and Golang programming languages. My main activities during the last seven years are called with a buzzword DevOps. That means that I'm creating and maintaining various infrastructure. And one of the biggest products I'm working on is a big Selenium cluster. Also, which is important for today's talk, I'm one of core maintainers of Selenium and related products. So first of all, how big is the cluster I'm working with? Compared to a typical Selenium grid having 50 browsers and executing 10,000 browser sessions per day. My cluster has more than 5,000 browsers running in parallel and executing more than two and a half million browser sessions per day. These browsers are distributed across five data centers. The average load in the cluster is about 4,000 requests per second. The traffic is more than two gigabit per second. And this cluster is certainly working all the time. This cluster has all popular browsers and platforms, including last 10 versions of Firefox, Chrome, Opera, all versions of Internet Explorer, Microsoft Edge, Android emulators running on real hardware servers, IOS simulators running on Macminis. And for some teams, we also have real devices, phones and tablet PCs connected to the same cluster. So some of you could also be using one of the tools we are working on, including Selenoid, an open source Selenium protocol implementation running browsers and Android emulators and Docker containers, Moon Enterprise implementation of the same protocol running everything in the Kubernetes cluster and also browsers and online testing platform like sales labs, browsers, tech testing bot and so on. So if you have any questions related to one of these tools, don't hesitate to ask these questions right after my talk. I think most of you would agree that Selenium is nowadays a de facto worldwide browser automation standard. Selenium appeared in 2004, but since 2009, its architecture is being called Selenium WebDriver looks like the following. It consists of a Selenium server handling test commands, a browser installed in the user's operating system, and between them we have a WebDriver binary, a standalone binary translating test commands to browser specific commands. So for every browser, there is a separate WebDriver binary. For example, for Chrome, we have a Chrome driver, for Internet Explorer, we have IE driver, for Opera driver and so on and so forth. So Selenium WebDriver architecture seems to provide a good decomposition between browser automation components. But anyway, its reference implementation, the reference implementation of this Selenium protocol seems to suffer from issues. I'm speaking about Selenium server. So for example, it has manual and complicated installation. You need to download, install Java, JAR files, run everything manually and so on. You can also have a lot of issues when running browsers in parallel. Your test can be slow, can freeze and so on. By default, Selenium server is using mutable and browsers having mutable settings. That is to say that you can accidentally go to browser settings. For example, configure a proxy server incorrectly and everything will stop working. So browsers are fragile. It's also difficult to run several different versions of the same browser on the same host, just because usually installing a new browser version will completely remove the older one. Also, you can, you may encounter browser and WebDriver compatibility issues and you will need to dive into the WebDriver documentation to understand how to fix this. So all these, all these numerous issues led us to creation. We just decided to create our own implementation of Selenium protocol and we named it Selenium. So Selenium is a GoLang application, a standalone server using Docker to launch browsers in containers. When you request to launch a browser to Selenium, it's using Docker API to launch a container with this browser. And then everything is running inside this container. So when you finish running your test, this container is automatically removed and your operating system remains just in the same state it was before launching the test. Selenium, as I said, is running browsers in isolated containers and every such container has inside everything needed to run to open pages in browser including an Xserver, an application allowing to start graphical, graphical applications like a browser, compatible versions of the WebDriver and the browser and other stuff like fonts, flash player, encodings and so on. So everything needed to correctly render the pages in the browser, which is very important. So now it comes with a set of ready to use images for all recent browser versions including all Firefox versions starting from 3.6 and above, all Chrome versions starting from 4.8 and above, Opera versions starting from 3.3 and above and also Opera, Presta versions. We certainly also have images for Android. We also have documentation how to build images for Internet Explorer so everything seems to be covered. Okay, so these five seconds. So this was the motivation of creating Selenium as a solution but as I said, our goal is to be able to run hundreds of browsers in parallel and nowadays even the most powerful server can run up to 100 browsers in parallel. That is to say, in order to run thousands of tests, we need to distribute the browsers across several or probably several dozens of servers. But the problem is that even having a lot of servers, all these browsers should look to a user like a single Selenium server. So we need some technology for scaling Selenium. The traditional Selenium approach for scaling Selenium is called the Selenium Grid. So probably some of you have been working with it yesterday. They do a workshop related to Selenium Grid. And traditionally, Selenium Grid consists of two main parts. It has a hub which accepts all user requests and one or several nodes connected to this hub which are actually being executed browser sessions. This traditional architecture has unfortunately also a lot of issues. So for example, this hub seems to be a single entry point to cluster and it seems to be also a single point of failure. So let's say if this hub will go down, the entire cluster will become unavailable. So also such cluster, it consumes a lot of memory just because of Java and also there is no stuff like authentication and authorization. So having all these considerations, we also just created a different approach for scaling Selenium. This approach, this tool is called GoGridRotor or simply GGR. And the main difference between GGR and Selenium Grid is that GGR doesn't require a permanent connection between hosts with Selenium and this load balancer itself. What it's doing is randomly and automatically distributing browser session requests across all available hosts. And such architecture allows to first of all have a stateless solution that is to say you can have any number of GGR instances behind the load balancer. And the last thing I would like to say about GGR today is that this load balancer was being used now for running 5,000 browser in parallel. So this is a proven solution. So that was everything related to motivation about the tools I will be showing today. And now let's move to the demonstration itself. So here is the program demonstration. I have already told you about the motivation. So the first thing we are going to do today, I will show you how to start Selenoid and its user interface called Selenoid UI on a virtual machine. So I will be doing everything in a cloud platform. Here I'm using digital ocean but you can do the same in any cloud platform like AWS, Google Cloud or Microsoft Azure or anything else. So I have already created three virtual machines for today's demonstration. Two machines for running Selenoids and the third machine for running the GGR load balancer. So you can just guess it by virtual machine name. These virtual machines are rather small. So for example, you can see that we have only two CPUs and two gigabytes of memory and 50 gigabytes of hard disk here. So let's start first of all the installation of Selenoid on the virtual machine. So we copy the IP address of the virtual machine. We enter the virtual machine and we can check that we already have Docker installed here. That's the only prerequisite for running Selenoid. So here we have some recent Docker version and we can also check that we don't have anything running on this machine. We also don't have any images so everything is empty. That's a clean Docker installation and these virtual machines are clean. So now to install Selenoid, what we need to do, we need first of all to download a standalone installation tool for Selenoid which is called CM that stands for configuration manager. So you can download this stuff directly from the GitHub. So here is the GitHub page for the CM tool. All this stuff is open source, certainly. So I'm going to release this page. I'm finding the latest release and here I can just directly download and copy the URL for a binary for your desired platform. Darwin for Mac, Linux for Linux, Windows for Windows. So here I'm showing everything under Linux so I'm downloading a version for Linux. So you can use your browser to download this tool if you're installing everything to your workstation, but in case of virtual machine, I will just use a built-in downloader command called Wgets. I'm just downloading the binary, everything is downloaded and I can just check that I have now a new binary on my machine. The problem is that by default when you download anything from the GitHub, even the binary, it has no execution permissions. So let's just add the execution permissions to this tool. Having done this, we can just execute this binary as a regular command. So you can just type CM and this is a program. It also has the help flag allowing you to get the help for every command. For example, we have a command called solenoid here. We can type CM solenoid help and it will show you some subcommands and so on and so forth. So every parameters are documented here. So in order to start solenoid, we need to type a very short and straightforward command. It looks like this, CM solenoid start-minus-vnc. So this command will do the following. It will first of all download the latest release of solenoid, then it will download two last images for two last versions of Firefox, Chrome and Opera on your machine. It will generate automatically configuration file and start solenoid. So everything, how much time it will take depends on your internet connection speed, but usually it takes up to five minutes maximum. And while it's doing this, let's do the same on the second virtual machine because we'll need this virtual machine for configuring the load balancer. So I'm just doing the same stuff here. Sorry. I'm doing the same stuff here. I'm also downloading the binary, the binary, the second machine. I'm giving just the same execution permissions. I'm executing the same command. So here we need to just wait for this command to finish. And when this simple command finished executing, we will already be able to run the tests. So let's just wait for the latest images to download. So now it's downloading the Chrome 85. Here is the Chrome 84. It will quickly download the images. It will now download the Opera images here. So everything should be rather fast because the majority of the images like fonts, Flash Player and the base operating system are being reused across the images. So you can see that it's rather fast. Now it's downloading the video recorder images, image and starting solenoid. So in like three minutes, you can now see that you have a bucket container running in your operating system. Now we also need to install a graphical user interface for solenoid because by default solenoid was created as a server-side application. That's not strictly requiring to have a user interface. So to install user interface, we need to issue a very similar command like this. CM solenoid UI start. And it's working very fast. It's downloading one image and launching. So now we have two containers running in our operating system. One of them is solenoid UI running on standard HTTP port 8080. And the second one is solenoid itself running on standard Selenium port 4444. So depending on your operating system, you may also need to open the ports in the firewall. For example, in here I'm using Ubuntu and I need to just allow accessing these ports. This is done like this. We allow 4444 and also the port 8080. So this is how we install solenoid and solenoid UI on a virtual machine. This takes only several minutes. Let's also take a look at what images were actually downloaded to this machine. So we have an image for solenoid itself, an image for solenoid user interface, an image for video recording software, and several images, six images for Opera Chrome and Firefox. Two last versions you can see here. So now let's take a look at the solenoid user interface. So in order to open the user interface, we just copy the virtual machine AP address. And we can open in our browser this AP address on port 8080. So let me zoom a bit. So here is what is called solenoid user interface. This user interface mainly consists of two main parts. The first one is the sessions list. So currently we don't have any running browsers, but when we launch our tests, we will see the browsers here. The second part is called capabilities section. So here you can easily copy paste, ready to use desired capabilities for any desired browser version. So for example, if we choose latest Chrome version 85 here, and we choose Java, we will be able to copy capabilities just to our code. We can do the same stuff for Python, for JavaScript, and so on. So this is basically how solenoid user interface looks like. Now let's start working with this user interface. The first thing I would like to show you is before we dive into running tests, is how to do some manual testing with solenoid user interface. So it's very straightforward. You just select a desired browser version here. You click on the button, create session. And now we will see this browser version running in my browser screen. So it will look like this. It's now starting the browser. And here I'm able to see the real browser being ready to execute some manual commands. So I can just maximize my screen, maximize the browser window. And I can just start working with this browser as usually. So for example, I can open our conference website like this. And if I need to somehow debug something on this web page, I can just click and open browser, the Chrome developer tools. So everything is working similarly to how like you are running this browser on your workstation. So here are the browser developer tools. So everything looks very similarly. Okay, so this was the manual testing. This was the manual testing. Let's now take a look at how to run an automated test because usually we need solenoid for running automated tests. So here I have, let me dive into presentation mode. So here I have a really straightforward solenoid test using Java and using standard solenoid stuff like creating a new remote driver. So everybody who have already been working with solenoid should understand this code. This code is doing the following thing. First of all, it opens the page. It's an anonymous search engine like Google or Bing and so on. Then it just finds a request text input on this page. It types the word solenoid and presses the enter key. And then it just takes a screenshot of the result of pressing the enter key. So in order to run this test on our solenoid instance, what we need to do, we need to simply update our solenoid URL in this test. We copy paste an IP address of our virtual machine. We simply insert it in our demo test and that's it. We can now run our test. So before we actually run the test, let me enable some features I would like to show you. So for example, we will now start our test with video and log recording. So an example test overview that's done. Let's enable video and log recording. This is as easy as adding several capabilities to our code. Enable video, enable log. So rather straightforward. Now we can launch our test. We will now wait for this test to execute. And we will check the log and the video file, save it during test execution. So my test is starting here. I will now be able to see the session running. So it's now used. I will now see a session here. Yeah, here's the session. The session is running. And when the session will actually finish, we will be able. So the test has passed it. We can now, for example, open the screenshot. Here is the screenshot of back.go page. And now we can go to Selenore UI and check how easily we could get the videos in the log. So the first thing here, we have now the videos tab, which is able to show you the video of the session I just recorded. So I can just click here and see what was happening inside the browser. So here's the browser. Here's the back.go page. And here is my test being executed. So here's a video of my test. Currently, there is no such tab for the log file, but you can easily get the log by using a very similar address. So you go to Selenite, the port 4444, you type slash logs. And here you will be able to see the log files for any session you executed. So here I can just click on a URL and see the logs of the browser. So this was an example of running tests with video and log recording. And here is how you can take a look at the log file and the video file. So for video files, there is a similar feature. You can open the video file using a direct URL like this in the browser. So sooner or later, you will certainly need to download these files or remove them. So for example, you can attach such a file to your test execution report. But in case if you need to delete this file, you can issue a very, very straightforward request. So let me show you how to do this. You can just use a regular HTTP client like curl and you simply issue an HTTP delete request. You specify the URL for the file you wish to delete. And now we can see that there is no such file here. So files are being automatically removed. So basically that was the first like impression of how Selenoid and Selenoid UI look like. And let me now show you how to create a cluster, how to easily scale this solution. So if you remember, here we have three virtual machines. On two of these virtual machines, we have already installed Selenoids. And the third virtual machine will be a virtual machine where we will be running our load balancer. So let's now quickly install a load balancer to this virtual machine and see how to execute a test against this load balancer. So we copy paste the address of this third virtual machine. We access this virtual machine using SSH. Here we also have a Docker installed. And now in order to install GGR load balancer, we don't need to memorize anything. We can just go to documentation which is publicly available and simply start copy pasting commands from browser to the terminal. So it's opening slowly because of the streaming, I think, just five seconds. Okay, so here is the GGR documentation. While the page is opening, let me just start copy pasting commands from this documentation. So first of all, I need to create a configuration directory for this grid rotor software. And then I need to create a user's ht password file using this command. For example, to install this command on Ubuntu, you need to add a package like this. So if you try to issue this command right now, you will see that it doesn't exist on my virtual machine saying ht password not found. And I just need to install one more package to my virtual machine like this, install. So I'm now installing the ht password tool. And now I can continue copy pasting commands from browser to this virtual machine. On this step, I'm creating a file with users and their password. So I'm creating a file like this, etcgridrotorusers.htt password. And I'm adding user with name test and password called a test password. So if you take a look at this file, you will see that it's just a raw text file having the username, then a colon sign, and then the password in encrypted form. So the last two steps I need to do in order to configure the load balancer. Here's the step that I need to run some Selenium implementation, but we already have two Selenoids running on virtual machines. So I will now create a so-called quota file. That is to say a file describing which hosts with browser to use for every user. So for example, now we have a user called test. And for this user, we need to create a file called test.xml. So we just copy paste some XML. We create a text file, test.xml. We insert our example XML configuration and we will not just change this file a bit to use our virtual machines. But here, for example, I would like to configure my load balancer to serve Chrome browser. For example, version 85. So I'm filling the default browser version, which is 85. I am creating a version 85 and specifying the region that is to say the data center name, which is really optional here. I'm using FRA, that means Frankfurt, because my virtual machines are running in Frankfurt right now. So the last thing I need to do, I need to copy paste the IP addresses of the virtual machines to my file. So here, I insert virtual machine number one. It's Selenoid port 4444. Then I copy paste and I insert the second IP address here. And that's it. So if I need to add one more browser version, I can just copy paste the entire stuff like this. So I can add, for example, Chrome 84 like this. So having all these files created, the last step to run a load balancer is to run a Docker container that will be using all these configuration files. So you just simply copy paste a command from documentation like this. It will automatically pull a very small image and run a load balancer container. So here is a running container with the GGR load balancer. So we can now start running our tests. Okay, we also need to allow 44444. And we also need to check that here on Selenoid 2 instance where we actually have a running Selenoid. We also allow to access 44444. So now check that everything is working as expected. We can simply open one more terminal and simply ping our GGR instance. So I'm just copying the GGR IP address and I'm issuing a very straightforward request, for example, HTTP, an IP address, 44444 slash ping. So I'm getting some response saying that GGR was configured correctly. So now let's run an example test against GGR and simply let's copy paste our... Sure, I'm finishing. So I'm now going to show you how to run the test against the load balancer. So here I have just the same test I was showing you before. And the only thing we need to do in our test to use the load balancer, we need to update an IP address. And we also need to insert the username, test, and the password, test, password. We created several minutes ago. So I think for some of you who were already using online platforms, such notation should be straightforward. So you should have already seen such notation. And if we now open the GGR logs, this, logs, logs. And if we now start running our test, we will see that in reality, our sessions are being automatically distributed across the virtual machines. So I'm now running the test. And here in the log, I will see that my session will be created on one of two solenoid hosts. I actually, I created at the beginning of my session. So it's attempting to run a session on this virtual machine ending at 171. So I'm selling out two. If I, and the session is actually created and everything is passing, if I run one more time, probably 12 run on the second host. But as far as this distribution is random, probably it will go to the same host. It's just really random stuff. So I will just try to run one more time. A session on the load balancer. No, it's creating on the same host right now. But anyway, if you run several times, you will see that host session requests are actually being distributed across the host. So that's pretty much everything I would like to show you today. So at the end of my talk, let me show you a slide with useful references and links. So here there is a GitHub URL with solenoid repository. In the same organization, you can find repositories for GGR, these demo projects, everything is there. Here is our Twitter, our telegram channel where you can ask any questions related to these tools. Our website with a lot of like URLs to useful articles related to efficient solenoid infrastructure and our YouTube channel where we are actually publishing some related videos. So thank you for your attention and you can ask your questions. Yes, so thank you, Ivan. That was a very good session. We have a couple of questions. So I'm going to check on the Q&A and we did all for you. So there is a question from Suresh about do we have solenoid Docker images for IE or age browsers? How would I say we have ready to use instructions on how to build such images because of license limitations, we have no right to distribute the pre-built images. But anyway, there is a probably I could just send to discuss channel. Seconds cube. Because last year there was a Slalom conference in Tokyo. I'll just send the URL. There was a Slalom conference in Tokyo where I was speaking about how to build such images. And in fact, we are providing, we open-sourced ready to use instructions, step-by-step instructions on how to build Windows images running in Docker. So simply I just copy pasted a URL to discuss section. You can just go and check. It seems to be rather straightforward. You can easily run a virtual machine inside a Docker container. So it will work. It's normal to work. Yeah. Thank you. So next question we have from Suresh again. Can we use solenoid UI with GGR? Yes. You can do this. The only thing you need to do. So basically, let me, I will be showing it. Basically, you will need to run one more small binary like GGR itself. These binaries needed to actually collect all the sessions from the entire Selenium cluster. So what I'm trying to say is that you can easily achieve this stuff by adding one more running application to the cluster. So this is doable, certainly. And you can configure your user interface to create the sessions against the GGR and see all the sessions being executed inside all the cluster on every host. This is doable, certainly. Yeah. Next question we have from Bebo. Is it possible to get HAR network logs and console logs using solenoid? Basically, yes. Because currently solenoid images for Chrome, for example, they come with built-in Chrome developer tools support. And as I used to say, you can first of all create a Selenium session using standard Selenium client libraries for your preferred programming language. And then you can directly connect to Chrome developer tools, WebSocket and fetch touch information without using any external proxies. A well-known approach for getting HAR files is using some external software like HAProxy or stuff like this. There are different proxies allowing to get these requests. But currently, with developer tools, you can do this directly without using any external software. Yeah. So that's the end of our session. Thank you all. I hope to see you in the next years of line, because I prefer offline formats.