 Yeah, we go. I just popped in again. And yeah, I think we had a great session and I don't see any of the questions popping out in the chat box I think everyone is cleared about the session Like they are just got every answer from this presentation slides. So, yeah Thanks, everyone for attending this session out. So let's get started Yeah, hello, everyone. And yeah, welcome to the dev conference again We would be having a session right now over the selenium proper grid on the open shift for platform which would be taken over by our Red Hat speakers Jatin Maldi and Deepak call here who'd be joining in the session And if you have any of the questions, you can just drop off in the chat box. Meanwhile, the session goes on So let's get started Hello, everyone. My name is Deepak. I am a quality engineering manager in Red Hat and and With me is Jatin My name is Jatin and I've been at Red Hat as technical support engineer for OpenShift in working OpenShift for long time. So today we're going to talk about scaling the Selenium grid. So Deepak will take over The details around the Selenium. So open to you Deepak. Thank you Jatin Next slide, please All right, so let's start with what actually Selenium tests because This is DevConf and I'm sure there are people who are not from the Testing background There are people from other development project management and other backgrounds So to give to give everyone a brief introduction of what exactly is a Selenium test So Selenium is an open source tool which uses this on wire protocol to Help modify the DOM inside a browser. So there is a Selenium server and then there is the native browser Selenium client libraries, which actually implement those methods so what you can do is you can write your tests in in one of the five or six languages which Selenium supports and then you can run your test by opening the browsers in Selenium server have virtual machine or anywhere and then you can run your end-to-end tests so Right now Selenium supports the multitude of browsers like Chrome Firefox Safari Internet Explorer Edge as well and then Opera and it used to support HTTP unit as well, but I don't think it it to support it right now next slide please All right, so the the problems with the running Selenium tests have always been the conflict between the browser version as well as the Selenium version so As you all know it's an open source project. So the speed at which the browser vendors increase the Development of browsers does not necessarily match with the speed with which the Selenium team Develops the Selenium version so there is always this mismatch between the Selenium version and the browser versions plus different different set of test engineers use different kind of Selenium infrastructure for example I've seen most of the test engineers actually Going ahead and running the Selenium tests on their local host Which obviously is not scalable which is which not And you cannot put those tests in your CI systems The second problem is Using VMs because if you aim to do the cross browser testing Like you want to include Microsoft Internet Explorer as well then you probably need to have VM setup for Selenium servers there You can then point your test to multiple simultaneous browsers like Chrome, Firefox and I And run those tests on those VMs. But again VMs Also have their share of problems like they are not extremely scalable and prone to same old browser and Selenium version issues All of this all of this Changed drastically when the Selenium project decided to build the Selenium containers for each of the browser library As well as the Selenium server So I think around three to four years ago people started using Selenium Docker containers which were which were very fast scalable and They used to just get rid of your Browser and Selenium compatibility issues But as As you all know the problem with containers is that they are actually actually disposable entities. You cannot you cannot build a High availability Selenium cloud procedure with just Docker containers so Plus you never know the state of the container whether it is up or not and everything around that And that is where the container of the station platforms like Uber it is coming to picture And this presentation is more about how you can build a sustainable scalable and high availability Selenium infrastructure using Using Selenium Docker images and Okd Next slide please. All right, so I'll hand over to the expert in this field And then he'll talk about Everything from Kubernetes okd and how how to build this setup And then I'll probably take over in the end and I'll show you how I can run my tests and point my tests to this Private Selenium cloud Over to you Geffen Thank you. Thank you so much Deepak. I think by now we have seen What are those basic pinpoints while using Selenium? and its literacy formats moving to containers one of the most important help which Testing or a QA team gets is the environment which they require to run multiple tests in multiple formats is Has become more easier with containers But just having container doesn't help because if that container goes down and If you want to bring it back up You just have to rerun it using the container runtime But in the in in that time there are some cis which which definitely get Failed just because there is no availability of your Browser or any kind of your tools which is required to run those tests So in that case you just not require a container runtime.q. You definitely require Orchestrated tool that is exactly where we have a Kubernetes Since Kubernetes has been one of the best orchestrating tool and has proven a lot in In scaling workloads making them more easier Using containers and using the different features which is right So what's exactly a Kubernetes? So Kubernetes is a declarative language for launching containers What's a declarative language? So that means I I just have An ability to make sure that even when I'm not around my machines my containers or my environment is always up and running So when I say environment, I mean those tests Infras which are required to run my test So let's say if I am an I am a developer and my concern is that matters should run No matter The if there is no one from the infra team or any other team available So I should not be dependent and in that case. I just have to run the test to make sure that my My Objectives are completed. So in that case this orchestrated tool make sure that he gives you those environment specific containers So that you can run the test and even if due to some reason those container goes down like for example if the underlying machines are very highly used and are And and they and there's a memory spike to it So it doesn't mean that your tests will fail because the orchestrator tool will make sure that the containers which were running on that machine Goes to some other node from the same cluster so that The developer doesn't feel that the environment is not accessible So this was a highly collaborative open source project And now it's been Also worked in open source to make sure that more and more ability of this orchestrated tool can be taken And we make the best out of it. So that is a second largest contributing member with many ideas coming from the It's like it's it's been very good to have them Coming from the open source enthusiasts and they're trying to make it more and more better if day after day So why do we call it as k8s? So there is a there is some acronym behind it It's like so between k and s there are eight letters which definitely makes it cubanitis That is exactly where we call it as cube or k8s as a short form It allows you to start stop update manage clusters running the containers and Many more things. So we'll come to that. We'll see how this Okay, d which is one of the flavor of cubanitis and built over cubanitis Does this help does help our developer to get his tests running? easily so uh So additional functionalities to make containers easier to use in a cluster. Yeah, so as I said, it's it's easy to reach just a login to one of the URL and you get an access to an environment an entire environment which is self-sustainable to make sure that whatever you ask for is available at the at the other end cubanitis does not And will not expose all the features of docker command line. Yeah, so it's not that the cubanitis is dependent on some other tool but it definitely requires a dinner and time at the back end because whatever the Cubanitis is or whatever is our procedure tool meant to run It requires someone at the back end to run them And that is exactly where the container runtime comes in the picture because your containers are definitely going to be managed by the runtime And over the runtime we have different components of cubanitis who will make sure that whatever is running stays in the running mode Whatever goes down the exact desired number of those containers come back up in some other nodes as well So there are some primitive keywords for cubanitis like masters notes board replication controller So what exactly is a master? This is the this is that component of cubanitis which runs the control plane In this control plane you have an api server A server which or an api server which looks for requests when you run some qctl commands when you run oc commands This is the api server which makes shows that This requests are being addressed While we have controller and the scheduler along with hcd which makes shows that The desired number of state whatever is asked for is always kept in the Live step in the in the current format. So that's exactly what the controller is and at the end if you want to run or scale up more Events or more containers you need someone who will schedule them So it's it's like that is exactly where the scheduler comes into picture with You can automatically scale through different components like auto scaler To run more workloads as and when the request is missing But at the end of the day all of this should be stored somewhere in in In a manner that they can be received received and updated as it was required So that is exactly where your hcd is used when you store and store all of your data And make sure that whatever is Whatever is when to change this also Then comes your node. These are your As any ends which make shows that whatever is the workloads These workloads are managed as per the resources available on the node a node runs a cubelet along with it Which makes shows that The respective node is in healthy state is giving its responses to the master And if the node is out of service like if the node goes High in memory or out of disk We have an ability or this orchestrator tool has an ability to even notify the same That way the node will not accept more workloads and the cubelet running on the node will tell that this node is already pretty full I'm trying to drain this out and take out all the containers of this and then schedule it on the other Then comes what pot so pot is just a collection of one or more containers. So containers are processes which which are mainly Managed by your container runtime and a collection of these containers can be pots Well, you can even run one container in one pod and you can run more than one containers also in one Then comes your replication controller. This controller makes shows that a number of The desired number of replicas for any of the application whatever has been mentioned in the declarative object. For example, the deployment config or deployment is always met That means that let's say if you make sure that you want Two replicas always to accept 20 requests considering 10 is being 10 requests are being handled by One application in that case There would be two pods by default when you deploy them And let's say if one of the node having the pod goes down This can this replication controller will be monitoring the state of the Will be monitoring the state of this project and also will be monitoring the number of replicas Which are expected to be up and running in this application and that is where when it sees that there is no There is there is one container less or one pod less It notifies the same then the api server with the help of the cube later on the node Make sure that the another container is also up and running Well, you have the pods you have the containers, but how do I access them? How do I make sure that there's a load being Shared between two because I cannot tell that okay go to this node and share it with this pod And if you want to move it to this one more go to pod b So in that case you need a service which is this entity is a virtual entity in Kubernetes or in okd where This service manages the underlying pods and whatever traffic Which is expected to be as an integral traffic or communication between two services running on the Kubernetes Is being done through this services or this layer which we called as svc So when I say that svc is also required or is is used for Internal communication, but how will a service know that these are his endpoint or these are the three endpoints which the Which the traffic is meant to or or where the traffic should be going when it's an incoming or outgoing packet So at that time a service Is dependent on objects like labels So labels play a very important role in Kubernetes where the discovery or service discovery is been done Through labels. So whatever selectors you write in the service The same selectors will be looked in every pod to make sure that if those match Then the service Enables this pod as one of its endpoints and then even send the packets or even send the incoming traffic to the End application which is a pod running inside of or running as the data And all of this is packed in one namespace. That is exactly where you have that isolation Part of making sure that more than one developer can use the platform and even They can keep the data Isolated to make sure that we do not leak or we do not have any security features between projects because Having one namespace running some application and only allowing one user at a time is not what an tool is meant for We want to make sure that multiple applications or multiple end developers are able to access the Platform make use of the platform the resources the platform is providing and then run their workloads So coming to okd in specific as we mentioned as our title of the session as um running a selenium grid on okd 4 So what's exactly is okd or what's exactly is okd 4? So okd is an origin community distribution of hubanatis It's optimized for continuous application development and multi-term deployment So when it's a multi-term deployment is making sure that no project Details are shared or accessed by some other project in a normal deployment okd adds developer and operational centric tools on top of cubanatis to enable the rapid application deployment development Easy development and scaling for a long-term life cycle maintenance. So if you see in the right hand side of my Uh presentation, you can see this is a developer more where The developer directly comes uh on the platform and tries deploying his application As in when he wants to and he doesn't have to make sure that he requires an administrative access Whatever access he has is what he can use and deploy his application in his selective namespace He can monitor them. He can search for some specific labels or some specific objects using labels And even he can build using different strategies. We also have some more projects or some more objects like config map and secret through which sensitive data could be shared with the application Like username passwords, which could be embedded in secret and shared to the application making sure that These are not exposed when the application is talking to other services inside open shift on okd So okd is also referred as origin in github and in the documentation So it was previously called as origin But then it was rebranded to bs okd and that is exactly what we call it now So in previously we had okd 3 which had a 1.11 version of kubernetes While now we moved to okd 4 where uh the current deployment is 1.18 for what for the latest okd 4 You're going to see in this presentation okd makes launching kubernetes on any cloud or bare metals More easier it's it's it's simple to install and you will see how fast it gets installed Definitely, we're not going to play a video of installation because because it just takes half an hour To get installed, but that's even more and more interesting when you see the next slide So how do you install it? So in order to install you need the right set of binaries the client tools so that you can utilize them to install the right images of okd over Any of your hardware or any of your tools or resources you have at the back end So you use the oc to library or sorry the oc binary to download and extract the tools You can just run this command and copy the image this image is available at the release Page over here. Just click on the link and you will find The latest releases so when we developed this we had This version as one of the latest versions we copied the binary to the To the directory where we wanted to start the installation from so this is a bastion host and on this host We want to make sure that the installation Goes ahead So uh, you will be prompted uh to choose a platform to install a w uh to install this open So aws is currently the best place to start with okd 4 While uh, it already has fedora core os machine images set Other providers you can uh move to um You can move to other providers, but you will need to set up this fedora core os images on it So, uh, the machines which you which you see in okd 4 the underlying machines run Uh coro is operating system. So in this okd 4 we run fedora core os And over that we build uh the complete cuban it is orchestrated to in this there is a temporary control plane bootstrapping where A temporary control plane is set up and then uh the production Control plane is being given the access and then the workloads or the installation stuff So to look for more, uh Installation formats you can definitely reach out to this link and look for other Variants of installation you can take a look at other cloud providers if you're looking for bare metal Even that is possible and um, it becomes more and more easier So this is a short snippet of how the installation is done. Uh, I have tried to copy the complete steps in this In this snip so that you understand what exactly gets installed and where it's getting installed. So this Is the binary I pulled from the at release Play page and uh, when I check the version I see this is open shift 4.5 And this is the build. Uh, and this is the release image which gets, uh, which is actually getting installed So the moment I have this binary and I have my cloud Uh credentials configured at the back and so consider that I've been deploying this on aws I have configured my aws secrets that's the access key and the secret Before the installation has started Or before starting the installation to make sure that whatever or whenever I start the installation there is no There is no issue with getting this Okay d4 cluster up and running so, um So so let's say that you have aws environment, uh Or aws credentials configured on the bastion machine And now you can run the command open ship install create cluster with increased love level to take a look at what exactly you see Or to see what exactly is getting installed So once this is hit, uh, the install config or there is a yaml file, which is fed, which is fed to the installer We we run terraform in the back end to create machines with, uh, when it's a cloud provider installation And that machines require some basic details which are configured in the install config once this install config is fetched It tries and applies create different resources on the aws It creates your iam roles. It creates your machines It creates the bootstrap node and all the other required objects once that is done It waits for 20 minutes initially to make sure that the api is up So this is your temporary api and this api, uh comes up It it waits for 40 minutes, but you can see that within 30 or within 10 10 to 12 minutes. We had this, uh api up or the bootstrap status completing that means Uh, the the timeout value is kept more but before that the installation itself or the installer is so fast enough to make sure that The the experience of installation becomes more and more easier once the bootstrap is completed as I said, this is a temporary boot We just use them to make our production normit up and running That is exactly where once this is done the the resources of these are destroyed So we do not keep any stale resources as well on aws or any of the other Cloud providers we use and these get deleted and once it's deleted the actual control plane comes up So this control plane now is with the help of the operator framework, which is a wrapper around open shift or over over okd Tries to bring the other uh required objects on uh open or on on So once that is done, it says that the cluster is initialized and uh, it's it's just waiting for Getting the console created and I think it doesn't take much time. It just just says that okay I see that there is a console there. Uh, there is the namespace and I also see the other Objects available so that way everything is created and this is where your install is completed So at this point you get the access to Use oc commands from the command line or also you get an access to even Fetch your console using the using this link and also the username password in here So this definitely marks that your okd is up and running and you can see that uh over here Let's say if I'll just log out and just show you How it works. So once I I get the installation there I see this page where it says me or where it says that okay, welcome to okd And this is the platform where you can lock in to Or where you can just just use the credentials provided in the previous In in in the previous slide or not slide or maybe the on the on the Just use it to install or log in to the console. So I'm just taking that Username and let me just take the password as well for it. So Just a second To write my password. Yeah, so that's the password. Um, I just copied and That is our login So, uh, this way I I should be able to to access the The console as well and also, uh, use it to to run some other, um Applications workloads or just check how the console looks like so as of now I just used a cluster admin access which is created for me with installation This can be removed once you have the other Authentication mediums attached as a data operations. So this way you can see I I I have a complete environment here It says I have five nodes. I have these many pods. I don't have any storage classes of now. The utilization is quite good some Some activities some alerts It says I have some of the other control plane which is down But then that that's exactly what I was saying of our architecture tool. It tries and brings itself up and running very fast So to see what exactly it's getting created. You can see the settings with source that I have currently this build and uh, these are the operators which are Which are governing the the cluster making sure that the cluster is up and running for the long time So these are different operators. You will also see where and you don't have to do anything to get this up It just gets installed by its own as per a process of installation So this is exactly how you can see the different projects and the the complete overview to To the to the environment you just created on uh using the okd so I just navigated to developer mode and um, you can see I have a topology I can just click on any of the project. Let's say I have this project and I see that I have three pods And if you want whatever you want to see if you can just select it over here, it will show you Just click on those. You will see the details of the ports. I have three ports running These are the details about this products under this namespace. These are the labels. These are selectors If there are any annotations, you can see them. You can edit them So this way the developer has the access to even Go more ahead and see what exactly is on open or on on this okd You can see the consumption as well the utilization of These pods and if you want to even navigate to monitoring dashboard It takes you to some other console and you can definitely see what exactly is the utilization of these containers So so this is how the complete okd Is is is done and you should you should be able to You should be able to use the the same formats to install so that it becomes more and more easier Coming to the autoscalar part because since we are now Open and now we know how to install okd. It's very important. We even install Or or or even have a working Okd or sorry a working selenium grid running over it So let me just just run through these steps to to to make sure that we we have this selenium or We have we have been using selenium here, which is a combination of salando plus selenium to make sure that Anemic scaling as an vendor test test requests goes up and goes or goes beyond the The replicas we have so let me just use the Terminal please. I hope you can see my screen and the terminal. So this is my okd You can see I have five nodes. Let me just make it a bit big And I'm just I'm just trying to use these commands to to run a selenium over here So I just have the repo. Yeah, I do have the repo here After that, I'm just going to create a new project. So OC new project Selenium Selenium demo So it's just it's just a command Like it's just one command Run which gets a complete project for you. I'm using the CLI. You can definitely use the UI that is also Something which is very easier and more recommended to be used while develop while developing The only reason of using CLI is I'm going to use Helm commands, which would be a bit difficult for me to run from the From the UI. So that's exactly why I'm I'm using this. So once I have this and Project, I'm going to use Helm tools to to to run the repos and and to even get my Templates up and running. So let me just try and see what I have so Helm repo list Okay, I do see that I have a selenium repo here. Well, it doesn't take much time to just add them So this is exactly what the command you have to run Helm repo add Just give it a name and give the link of your GitHub project So that the the charts which you have created gets loaded over here So once this is there, you just have to run Helm repo update so that you get updated All the updated details on whatever changes you made in the charts into your Repositories which are which are hosted over here. You can see we have some stable local I have a local repo as well on my host on this current passion host and then I I haven't have just added this selenium as well. So coming to the Next part as I just added the repo and I just have to create a template So or or or using the template. I just have to create the the complete selenium to just Run in my environment so that I get the complete scalable Selenium grid and that is exactly where I will just use this template command so Helm template iPhone iPhone name Selenium. This is something which I taken from The commands you see from the presentation and I am using charts which are available in this repo or under this Under the repo which we have just added and this way I am going to use a namespace which I just created which is iPhone iPhone namespace And that was selenium in our UPM demo So it will just say that it has created a service account It has created the service for me. It has created a deployment and some rbacks which are required to Make sure that only authorized users are able to access it So let me navigate to the project OC project selenium demo And then we'll see get pods. I should be able to see that. Okay. I have a selenium hub over here Let me just run the logs and you should see that this is a selenium hub As you can see this hub is up and running and then you have Notes which can register Using this url when you have clients the the clients come to this url And it says that okay the hub is started and the testing and all is all enabled You should be able to use selenium now and it also says that it has currently added two hosts like the two nodes so Just you will see get pods and you should see that we have two Two nodes as well So just coming back to our ui in the ui as well. You can even see over here if I navigate to the project selenium demo You should be able to see that I have one pod and the the same pod. I have uh, the the The hub running very well, and if not this way you can just go to the admin page Let's say if you have that access and just navigate to networking and just to the routes This will show you that there are some routes created now Since we just have a service and have not created an error you can just go ahead and click on create Just give it a name. This is uh hub url And the host name will be picked. Uh, we do not have to do it. I will just use the selenium Grid this is what I want to use the support if you have we want to secure it just use the tls And just secure them if not just hit create and that that's exactly what the hub ui looks like so Just click on the hub url and you should see the hub running if I navigate to console I see that I have one a node here and one node over here So that's pretty much how you get the complete, uh, autoscalable selenium infra And this infra is now ready so that the developer can go ahead and run some tests on it So that was all around the demo of how to install and uh bring up the selenium grid I will now give it up to or I'll give it to deepak so that he can run through the tests And show how exactly the the testing helps or how exactly the the resources get scaled when the tests are run So over to you deepak. Thank you. I'll share my screen Can you Can you all see my screen? Not yet, but yeah, hopefully it's People to see it Never mind Can you can you share your screen yet and again with the uh, yeah Selenium grid console This one. Yeah Yeah, so this is what the console looks like Okay, so I have uh, I have selenium tests written in java uh with test ng as the runner And I I have four tests in each class which I want to run parallely on Both firefox and chrome So what I'll do now is uh, I'll start my run and you'll see that right now. We just have one Uh, one instance of the selenium node running and as soon as I start my test I will see that there would be some auto scaling And here we go. I have started the run And I'll ask uh get them to Refresh the page and show you whether we are actually increasing the Nodes So the test run is going right now So you can see that two requests are waiting for slot to be free And it will now uh in a moment Create another replica of this Uh Selenium node. Here you go So this is totally uh uh Autoscalable depending on depending on certain hub parameters like max instances per session Your new instance timeout. We recommend you keep the new instance timeout to equal to uh Equal to what your hub or your test timeout is so that uh, your your tests don't fail Without any reason So as jethan said we use selenium because uh, the actual selenium grid the official uh, selenium grid project has some issues with auto scaling So selenium itself is you can think of selenium as the Selenium grid on steroids. It has some inbuilt VNC capabilities as well. Let's say you want to view what's actually going on In your test. So you can open the debug mode and actually see the it also retains your videos for some time uh Apart from that, uh, the actual selenium project is also going to merge with the official selenium project Uh after release four. So it's uh, I think it's okay if you use selenium as well So as you saw, uh, the test plan is complete. We uh, based on because I had a parallel test. So it raised the selenium instances to two and then automatically uh Came back to one instance again so you can uh The thing which we are trying to achieve is that historically the setting up of uh end to end test automation infrastructure Has been a pain point for test automation developers always so as I said they they Some of them use their local host to run the selenium test some of have this painful setup of VMs and some of them use Docker containers locally or on VMs. So this gives you some enterprise level selenium cloud where the your engineers only focus on creating the tests and not on the infrastructure so You you get you get has uh jethan created this uh route So you get one route which can serve all your uh end to end testing needs. I mean browser testing needs so This is pretty much uh about uh back to you jethan if you want to add something Yeah, so this is pretty much how um You can achieve those dynamic scaling As for whatever tests you run since we just had to move from one to two you can see that uh, initially we had uh One uh hub and one node. So if if I just click on one of them that one won't be your hub on which uh, you have the The the complete selenium hub running and the url for this is over here You can see the see the console or you can just see this is how the The the the ui looks like if I I guess if I just append it with Wd-hub it will just show me some more details around the hub as well So that is exactly how uh, you add you you run some command or you run these uh url to To to let the test uh know that this is where the the selenium is and uh with the help of the orchestrator tool uh The requests are reaching your hub and as per whatever configurations are there on the hub it's been shared with your Uh underlying nodes So that is where the tests are also helping and you don't have to set up any environments By yourself. You just have to create once and uh, just set up the right values as People mentioned. I'll just show you that those as well So you just click on the deployments and we have some environment variables which says that okay This is the requests. These are some limits. Which should be always catered to so that the node doesn't go down These are some max numbers which we have set and as he said that This is the sessions we have enabled and even it should be that uh visible that let's say if there is no instances to To address any of the tests. It doesn't just say that okay just give an error It waits for the mentioned time and then once it's uh, it's it's available It's it's been shared by the other tests. So that way the tests are also running It's getting on the scale and you are able to get all the results based on this Container grid or based on the complete Container grid. So I think that is pretty much what we had in the demo I guess if there are any questions, please Raise those to us and we are here to help you on with it. So anything Deepak, you would like to add in please quiet No, I think that's pretty much it as jethan mentioned If if you don't know hub and serenium hub and node Which actually comprises the serenium grid Was designed in a way to where the hub acts as a delegator It receives the request from multiple clients or one client sending parallel testing requests and then delegates those tests to multiple browsers, which it has controlled as part of its strategy with multiple Uh customizable parameters like it has four chrome nodes and two firefox nodes and Maybe an opera node and stuff like that So that's what hub is all about but With with an orchestrator tool You just take it to the enterprise level where your test engineers don't worry about We have to run your test. It just it's just one URL and always up And scales automatically Perfect. So that's it from our side Thank you so much for being the patient and taking a look at this Thank you Uh Thanks, everyone for attending this session and my gratitude towards jethan and people for presenting this session out very well This was quite an interesting session. I think yeah, people love this session and I see a couple of questions Or like people if you do have any of the questions you can just post over the chat box like to your Right side of your screen so that the speaker like we have both of the speakers available to answer all of your questions here So go ahead Let me jump in and I think we don't have any more questions I see on the chat box. So once again, thanks jethan. Uh, and thanks deeper for the amazing session