 So good morning everyone and welcome to this session of Selenium Conference 2020. So today's my topic is Rise of Shadow Dome and let's solve it through by using two different approaches. First is WebDriver Selenium we can say and second it's WebDriver IO it's a wrapper over WebDriver itself. So without wasting time let's move ahead. So before starting this session let me introduce my company where I am currently associated. So currently I am working with 3 pillar global it's a service based company. So we are implementing products for our clients and so that they can enhance their revenues and they can grow in this competitive market. So in short we can say we are a product lifecycle management and a software product development company. And this company is founded in 2006 and we have four global centers it's one it's in India, one it's in Costa Rica, one it's in Romania and the headquarters in USA Fairfax. In my company we are implementing different different products by using latest technologies. We are implementing web based applications we are implementing native mobile applications like iOS, Android and some hybrid by using React Native. And we are also giving cloud services to our clients and last but not least we are also giving Q automation services by using different different open source tools. So 3 pillar global has been recognized as a great place to work certified organization in India. So this is for its outstanding company culture for it's a second straight year. And this prestigious certification from the great place to work institute is awarded to those companies that foster excellent workplace environments for their employees. So we are lucky we have good culture in my company and let me introduce myself. My name is Virender Singh professionally I have a good experience in area of Q automation and currently I'm associated as a senior Q automation lead with 3 pillar global and as my company following agile culture. So this culture always motivates me to explore new technologies, new tools and which is coming new to the market. And being a leader like I'm working as a senior Q automation lead. So leadership always push me to help my juniors to help my peers so within the project or whether it is within the company or whether it is outside of the company through the social networking and etc etc. I am always open to help anyone in my field. And apart from this professional things I am also a triathlete and currently preparing for iron man 70.3 miles competition which is which will be held in next year now. And these are my contact details. If anybody wants to reach out to me then can email me at virender.pnnu at gmail.com or can approach me through the LinkedIn. So today's agenda it's today's agenda it's like I'll be covering three main topics there first topic it's web components and custom elements with shadow dome. So this is the one thing the second thing then will solve it by using web driver and the third thing will solve the same shadow dome by using web driver IEO. So let's have brief idea about it but what we are going to discuss today. So first of all we'll talk about web components then custom elements and the shadow dome. And how this shadow dome is different from the light dome we'll discuss more about it as well and how it creates problems for the automation guys will see all these things as well. So basically all these things are new age HTML which is creating problems for key automation guys will solve it through by using two different approaches. So first approach it's like we usually encountered problems because of this shadow dome. Suppose we are web driver guys and we face problems because of the shadow dome then how web driver interact with this shadow dome and we will we would have a demo at the end and second approach it's like the problems encountered to the web driver IEO guys those are working on this tool and then we will see like how this web driver IEO tools natively support how to handle this shadow dome then we will see the demo about it. So moving ahead before moving ahead let me tell you the problem statement which I encountered few months or around one year back like I was sitting in my company and someday and one of my cooly came to me like he has to prepare some POC on his new application actually he was even not able to interact with the elements by using our simple selector approaches like X paths or CSS selectors he was not able to find those elements it's a like it's a weird situation for him because because if one of us is not able to find out the elements itself then it's a shame for us so I was quite busy at that time but I asked him to provide me their application URL so that I can look at it when I will get some time so when I came back to home then I looked at that URL and I also tried to find out those elements by using same simple approaches like X path and CSS selectors even I tried JavaScript but I was not able to find those elements then finally what I have seen on the web developer tools like there is one thing which is written like shadow root then I google that the term and then I came across like oh so this is the shadow dome and we cannot interact directly shadow dome elements by using these simple selectors we have to use some we have to take a different approach for it okay so this is the problem statement which I encountered before moving ahead first of all we need to discuss about the web components so what are the web components so whenever any application renders on a browser then it always contains three things first thing it contains HTML so this is the structure of the elements second thing it always contains CSS like styling of those elements third thing it always contains the JavaScript like through JavaScript we can take actions on those elements so we always see these three things in our web browser through the web developer tools as we know like dome is it's a very unknown term to each and every one like it's a structured representation of an HTML document and through JavaScript we can interact with those elements and we can manipulate those elements within the dome because JavaScript has a complete access of the dome okay let's talk about web components now web components are basically it's a W3C standard feature that is being incorporated in HTML5 it's because it's a new HTML it's also in the Google's polymer library it's also in the Google's lit element library and there are some more libraries in the market where we can see these web components web components are a set of APIs that facilitate the creation of new custom reusable HTML elements like we can see here reusable HTML elements we can see header we have used header twice we have here and we have header here we have putter here we have putter here and that can be used in web pages and web apps with your functionality and that is isolated from the rest of your application code we can see here so these are completely modular and the reusable components we can see on this web application or we can say web page basically this web page it's divided into different different web components or we can say different different smaller modules we can say and these modules these modular components are reusable across the web application and this web component has four different building blocks we can see here one is the custom HTML elements second is the shadow dome third is the HTML templates and fourth one is the HTML imports but in this session we'll only talk about these two building blocks because we need those we don't need these two so we'll talk about the custom HTML elements we'll talk about the shadow dome okay so here we can see these web components can interact with each interact with each other very easily and but they cannot access each other's properties for example one web component has one property and let's say color is red and another web component has a different color maybe the color is green then they can interact with each other if they require it but they cannot impact each other's properties right suppose but here is here is one hack for example if we have any global properties on that application let's say it's color is black so in that case that property would be applicable to each and every web component in that application because it's a global property and as well as it's an inheritable property we have known in inheritable properties also like background color it's a non inheritable property but if we make it global then it won't impact any of the web component okay let's say color is a global property but we can override this property within the web component for example in header itself if we override that property then we can change it we can in line override it in any web component it's all up to you okay yeah moving ahead so let's talk about the custom elements custom elements are here we can see custom elements are like these are very kind of HTML elements with custom templates behaviors and tag names made with set of JavaScript APIs here we can see we have used HTML element class to make those custom elements here we have inherited this we have extended this HTML element class right there and by using this class we are creating a new custom elements by using this method we can see here custom elements to define we are defining this new tag my element and before and just below that we can see my element is it's a this is the HTML which will be rendered on the browser itself by using this code we can see right there let me open one file there here we can see this page what I have done here it's the same code which I have mentioned in the slide itself we can see my element right there and under that under that custom element we have one button if you try to access this by using xpath then we can very easily find this button right and if you try to find by using css maybe let me use css right there then again we are able to find this button because still the shadow dome is not applied on that code it is just a custom elements which we can access by using any selector like xpath or like or css or by using the javascript we can easily access those elements here we can see we haven't applied any shadow dome right there and that's why we are very that's very easy for us to access those custom elements now let me apply shadow dome on this so here we can see the same code the class extending again HTML element and below that we can see the same custom element but here in between this method we have applied we have attached shadow dome we can see right there and we can see the mode is open we'll talk about the mode as well down the line so currently the mode is open I mean the shadow dome is openly applied on those custom elements and here we can see the shadow is appended with that button as well okay let me open this let me go there right now so this is the same code which I have mentioned in the slides if we click on that element then we can see under this custom element we have a shadow root now and the mode is open for it now we can see the button is under the shadow root it's a part of that shadow dome if we try to access this directly then we cannot access it let me try to find out the x path first we can see it is not visible like we are not able to fetch this element by using x path now let me try this css again css sorry it's a javascript let me try css as well like nothing is fetchable by using these simple selectors so this is the problem like it's a super annoying for everyone like if we are not able to find the elements by using our different different approaches then it's a it's a really a big problem for us now the question arises in mind that if the custom elements gets encapsulated after attaching shadow dome then how to interact with these custom elements inside the shadow dome so we have one thing like as we have seen query select rates entry point for the whole dome I mean I'm talking about the main dome which we usually see in the developer tools so query select rates API it's entry pointer for any dome now what is the entry pointer for the shadow dome so we have a shadow root API this is the entry point it's it's again API this is the entry point to enter into the shadow dome so we need shadow root API with query selector method to fetch the elements although shadow dome is a useful solution for web developers because it's there are some security issues but it becomes a challenge for automating testing because those elements inside a shadow root technically do not exist in the main or the light document dome it's a problem for us so let's try to fetch this element by using shadow root here we can see the query selector then the my element this is the out of shadow dome and now we can see the button button this is inside the shadow dome so that's why we have used shadow root dot query selector if you try to fetch this then we can easily see it is visible to the user we can find the elements very easily so one thing the mode is open currently if I make the mode closed let me make it closed here we can see the mode is open if I'm making it closed yeah I can hear you Shweta just moment let me try this I think I'm not able to move this thumbnail because no I'm not able to try okay thank you Shweta okay so now I made it closed now we can see the mode is closed and let me open this element again if I refresh it and if I try to fetch this element by using this shadow root then we are not able to interact or the fetch that element because closed mode means the shadow root is in that case shadow root is always null I mean the element is not available to the user so what is the use of open mode and the closed mode so we can ask the developers for example they do this because of some security issues we they make it closed mode so that we any any person cannot interact with those elements so in that case what we can ask to ask to the developer suppose we have to automate those pages in that case we can ask developers to make the mode is open on the sandbox or the we can say on the test environment but on the production environment they can make it closed mode so in that case like security would be fixed by the developers and also we can automate our scripts very easily we can verify our regression speeds on the sandbox itself or on the test environment itself we can ask to the developers about this okay if I make it open again if I save it if I refresh it and if you try to fetch it once again then it's visible to the user again like we can interact with those elements okay let me move to the next slide okay so let's understand the how shadow dome works within the browser so here we can see at the top it's a light or the main dome which we usually see and there is a document interface that is entry point for the light dome or the main dome but within the main dome we can have any number of shadow dome so there is one entry point for any shadow dome it's a shadow host so in our case it's a we can see here just moment it's in our case my element it's a shadow host for us because this is the entry point into the shadow okay so here we'll talk talking about the shadow host it's a it's a kind of entry point in into the any shadow dome and below that entry point we can see any number of shadow roots and we can have single level shadow roots here we can see it's a single level shadow root and we can have multiple shadow roots as well for example in one shadow root we have we can have another shadow root and we can have multiple elements in another shadow roots so here I would say again like shadow dome it's a shadow dome is a kind of web standard that developers use to encapsulate dear custom stml code and style components and with JavaScript as well and shadow dome elements are not visible to the documents.query selector API because these are only visible to the shadow root I mean the documented shadow root API okay and here we can see the scoping like we have boundary for these shadow root elements so if we have to enter into this boundary then we have to use shadow root API without this we cannot enter into the boundary and we cannot interact with those elements let me open one application just moment I have already opened I think so to do app okay so so this application it's it's implemented in the by using the polymer library it's a it's a google's library and the earlier which we have seen these are the stml 5 based app code which we have seen so in this here we can see we can see the multiple level of shadow roots here we can see we have to do app it's a entry point or we can say shadow host and under that we can see one shadow root then one element another shadow root then there are loads of elements we can see the text area here and if you try to fetch these elements if I try to fetch these elements it's a multi-level shadow roots just moment let me make it bigger here we can see one ID then we have another ID shadow root query selector adds items then shadow root query selector enter text area we can see here it's a it's a multi-level shadow roots so so this is what I have shown right there let me go to the next slide so this is all about like like which browsers are supporting this shadow dome here we can clearly see we have two versions of shadow dome it's a version zero which is already deprecated maybe it is used by some developers still on already implemented applications but currently the developers using its shadow dome version one and here we can see it's a almost supports every browser like we can see edge the latest versions of edge browser the firefox we can see the chrome safari opera we can see safari isafari as well it's a partially support to this shadow dome maybe not fully but there are few which is not supporting maybe it's opera mini but it is supported by android browsers opera mobile chrome for android and we can see a list of browsers which is supported by this shadow dome like we can in short we can say shadow dome can be rendered on any kind of latest browsers so that's why it's getting popular nowadays and developers using it the two purposes one is security and maybe second thing it's supported by each kind of browsers nowadays let me go to the next slide okay so it's time to talk about the web driver approach with the shadow dome okay we can interact with the shadow dome by using JavaScript which we have already seen and then how the web driver can interact with those elements the web driver can only interact with those elements by using JavaScript executor API we can we have to use this with the web driver then only we can interact with those elements and we can take actions on those elements because there is no native support by web driver itself but we have work around for it and we can very easily interact with web elements shadow dome web elements let me show you one quick demo here i have written like very basic code about it i have used this local application which is implemented in polymer library and below that we can see this is the method which i have used to return the shadow root elements so that the driver can take actions on it and which i'm using these elements right there it's very basic code i i haven't improved it a lot but just to show you the demo guys and here we can see after getting the shadow root elements then we can append another elements and we can take actions on those let me execute it by using testNG so this is the same application which i have shown you guys it's in the blue color colored here we can see it will add some it will add some test plans on the right side it will add two test plans for today and two plans for later i have applied some weight on these scripts so that you guys can see it okay this is done and we can see it is passed so the main thing is interaction with those shadow dome elements and later we can just apply JavaScript executor and we can take actions so this is the approach which we have taken by using web driver and let me move ahead now it's time to take another approach it's by using web driver IO so web driver IO basically it supports natively by using two different two different approaches one is the shadow dollar here we can see it has two different inbuilt commands one is shadow dollar one is the shadow double dollar shadow dollar basically we use to access an element inside a given element shadow root it it will fetch the element which is a single element if there are multiple elements kind of list of the elements in that case we have to use shadow double dollar and we can fetch those elements one by one by our regular approaches so here we can see we are getting here we i have used javascript i am just fetching one element by using shadow dollar this is inside the shadow root and again we have another shadow root element it's enter text area so there are multi multi-level shadow roots and we are fetching those elements by using this shadow dollar let me open the code and let me show you the demo here we can see here we are fetching we are returning those elements by using shadow dollar and below that we are just taking actions on those found elements and finally we are calling these methods inside the test script which i have used there let me run this so it will open the same application then it will take the same actions but this time it is by using javascript and then web driver we have used java so it will add two test plans for today and then later and it is passed and we can see all of the actions has been taken and it is successfully executed so now we have seen two different approaches first is by using web driver second approach is by using web driver io so here i just want to suggest one thing to everyone for example if you guys any one of you guys using web driver already by using java as a technology and some of the pages the developer implements by using this maybe any library maybe a stml5 or maybe lit element library maybe polymer library and in that case you can take the first approach because you have no need to write you have no need to change your tool in that case you can continue with your automation scripts by using the first approach but if in case you know the javascript and you are starting your project automation from scratch in that case you can go with the second approach because web driver io it's giving native support for the shadow dome but i have seen the code the shadow i have seen the code for web driver io internally it is using the same thing which we are using with the selenium web driver because it is it has written some just wrapper method over the same thing and internally it is using the same javascript and all so here i have mentioned some references you guys can go through these references like about shadow dome if anybody wants to know about the shadow dome then can go through this link if anybody wants to take a shadow dome approach with selenium then there are some methods which you guys can have dependencies by using form dot xml file and here you guys can see the shadow dome approach by web driver io so you will see these shadow doler and the shadow double doler methods and below i have this is my github account where you guys can see the test applications which i have shown you guys and the selenium approach and the web driver io approach so i'll i'll make it public today so that you guys can get this and you can try at your end so that's all about this session if you guys have any questions or any doubts about it and i hope i have cleared each and everything but i am open for any kind of questions thank you virender for that i think the audience had a question regarding whether the code would be available so i think you've answered that towards the end yeah okay so we have quite some questions posted so i'll take them one by one so the maximum vote is for the question what is the difference between light dome and shadow dome okay so light dome basically it's our the main dome which we usually see forget about shadow dome first suppose i want to automate my application the first of all i always look at it it's a light dome or we can say it's a main dome so the entry point for the main dome still it's a query it's a it's a what i would say like like i'm not getting exactly now but still i would say like for shadow dome it's a shadow dome it's a part of the light dome we can say it's but it renders separately that's the reason we cannot interact with the shadow dome elements and another thing it's like developers make the mode is closed in that case we cannot interact with any element at any cost but if the developers make the mode is open then then only we can interact with those elements by using a java script shadow dome basically it's a part of light dome we can say it renders inside that light dome okay uh we have another question again on the differences what is the difference between shadow dome and iframe okay iframe it's a like it's an older technology earlier the developers using iframes but nowadays they are not using iframes because there are some cross-site scripting issues but now they have started using shadow dome it's it's a kind of iframe but it is completely quarantined from the light dome this is this is what i can say now it's kind of similar things but it's completely quarantined but iframe was not quarantined from the light dome so this is the only difference between iframe and the shadow dome elements okay the next one is what is the purpose of creating shadow dome okay uh purpose of the creating shadow dome is that suppose i am creating one of my plugin and i just want to sell these plugins to different different vendors or we can say different different web applications in that case what will happen this plugin won't won't interact with their html codes and their code won't won't interact with this plugin code plugin html or the gss or we can css so they they will render on the same browser on the same uh on the same dome but they won't interact with each other they can work flawlessly with each other but they they won't hamper anything of each and like they won't hamper uh each other's code so that is the reason they are using shadow dome nowadays so this is kind of security issues we can say and second is they are they are not accessing each other's properties so this is the main reason okay um another one how to check if the shadow dome element is closed can you please come again swetha i didn't get this question so uh how to check if the shadow dome element is closed okay even i have uh shown there just moment let me open it once again if i make it closed i have made it closed and if i just refresh it then we can see the shadow root it's closed we can clearly see right there and in that case if it is closed then we cannot access this element this is the simple thing which we can see by using the developer's tools okay um we'll take a couple of more is that um why and when do i think web developers use shadow dome i think that is also answered broad yeah i think it is already covered because there are two reasons okay how many ways we can find these uh shadow dome elements here uh i would say like uh javascript is the only uh scripting language which has complete access of the dome so javascript we can use javascript and we can access those elements at any point of time but we have to use uh shadow root api to fetch those elements that's the only thing which we have to use otherwise the javascript works well with it because simple uh selectors strategies like xpath and css they won't work at any cost so the it's a it's a useless for shadow dome elements okay um then if shadow dome is closed it means we cannot access with any workaround yeah in that case we don't have any work around to access those closed elements so so that's why i have explained earlier if this is mandatory to make the elements closed mode so in that case we can talk to developers and we can ask them to make it open on the sandbox or the test environment and they can make it closed for the production environment this is the only solution which we did in in our company in my in my project otherwise there is no work around to interact with the closed elements okay thank you so much virender for answering those questions um it was a wonderful session we already have a lot of good feedback coming in on the channel uh on discuss thank you so much shweta and thank you so much everyone for joining this session i hope i did not waste your time and i hope you learned something new uh for your career uh the session was brought to you by sauce labs