 First of all, I'm really humbled that you have a lot of good sessions to attend to and you decide to come here So hopefully you'll get The things I'm talking about and hopefully you're gonna understand how AI is going to be used in software testing my name is Rod Subramanian and a little bit about my company and what I do So I work for a company called testim.io TSTIM.io What we do is we have an AI based software testing tool which helps to do front-end testing UI testing and end-to-end testing and Coming to my role Just like Gil who was giving the presentation before I'm a developer evangelist Which means that I write about software testing speak about software testing and also Work with the founder of my company to make our tool better for our customers, right? And also work with the customers directly to help them use our tool, especially when it comes to code and stuff And I have a background of Selenium. I did seven years of cucumber Java and Appium So I kind of know the framework pretty well. So that's about me So today I'm gonna Cover these topics as part of the stock, right? So I'm gonna talk about what are the current challenges with software testing then how AI can help to solve these challenges and Before getting to how I can solve these challenges. I may cover the basics of AI. I Know in a previous presentation Gil mentioned a couple of stuff about Basics of AI, but I want to make sure when I use AI terminologies. Everyone is on the same page When I talk about AI deep learning machine learning and stuff like that and Then finally I'm gonna talk about the future of AI and there's some other things Those are some other things which we are currently working on as well So for let's set some context, right? So even before talking about what are the problems AI can solve It's really important to understand how software Testing has evolved in the past few decades, right? So I know for people in the end may be hard to see but In the 1980s and 1990s period, right? testing was predominantly manual testing and we had teams Which used to deliver which used to build the product for like six months and then finally they'll give it to the tester saying hey test this right there. So pretty much the development methodology was waterfall and What happened was there's a lot of wastage of time costs and effort because what the test will find a bug and Then once he reported a lot of rework Had to happen and then a lot of changes happened and because of that people started missing deadlines And the customers didn't get their product on time, right? So that's how software testing was in 1980s and 90s and Then from 1990 to 2003 we started seeing these bulky automation tools Which started cropping up right testers were super happy that and finally we have something which could help us in testing But the problem was they were so bloated and it was really hard to understand what they're actually trying to do right and it was a steep learning curve and At the same time We started seeing people experimenting with different development approaches right like rad which is rapid application Development and scrum XP and those kind of approaches. So that's what happened in the period of 1990 to 2003 Then we come to the 2003 2010 period this was the period of Innovation I would say because The open-source framework started during that time and there was this group of like-minded people who wanted to Solve different problems with software testing right and and what they encourage other people in the community to Help them solve problems with software testing. So one example of an open-source framework would be Selenium so that kind of started during that time right and and so We started seeing a lot of these Open-source communities forming during the 2003 2010 period at the same time people got tired of waterfall way of Developing software right so agile became huge. So we had scrum XP and people wanted to release Software fast, right? They wanted to write this fast Find bugs faster and really really really fast. So That was 2003 2010 period and then we had 2010 to the current period which we are in right now where Everything's about scale. So whatever I mean by that. So as The software is becoming really complex, right? there's a need for a lot of Server space faster processing parallel execution and And we have so many resources which you need right as your Testing complexity increases. So in this era from 2003 till now There were two things which have made a big impact in our software testing industry So one is crowd testing where people Regulates that collaboration was really important and what they did was they give their Applications to other passionate testers outside the company to get feedback about their application, right? so crowd testing when was one big thing which started and Another thing which happened during the same time was cloud testing because we had server space Faster processing need for parallel execution. We have sass left and other Amazon cloud and all those people have come into play So those are the two big things which have happened. So that's the current state, right? Now it's all about the future. The future is going to be autonomous testing using machine learning and AI It's not going to be about static locators It's going to be about dynamic locators, right? And I'm going to talk about that in a bit I'm going to talk about that in detail with the demo as well and and it's all about Increasing collaboration making automation simple smarter faster and more stable. So that's the period We are going to see and that's going to be the future, right? So it's important to get this context and it makes sense as and when I go through the slides So let's talk about the current challenges with software testing, right? So recently we did a survey in them we asked people how frequent how frequently they released software, right a majority of them They start testing early, but they keep testing, right until the last minute and because of that a lot of times Release cycles are getting affected and then they're not able to release software and time, right 57% were Responded to that and Finally at the bottom we see 2% of the people who are releasing every minute of every day, right now If you're going to transfer this into a graph this how it's going to look like So we have the laggards in one end Who release software every six months, right? And then we have the innovators on the other end who are releasing software really frequently every day, right? So let's talk about challenges in each of these phases and then you're going to understand how AI is going to help to solve These challenges in each of these phases, right? So the first challenge is the challenge of skill set so people Who are in the laggards phase who release only every six months? They have this challenge of skill set where getting skilled testers is really hard and and and it takes a lot of time and it's very expensive, right and That's because of that challenge Most of the testing is manual and it just takes so much time for them to release a product, right? So that's the laggards phase then you have the late majority phase where There are people who release product Who do the releases every month, but they have this huge challenge of authoring, right? So what do we mean by that? These are organizations who have testers. They have some skill testers and they start authoring tests But they've failed to notice and pay attention to good practices of automation like reusability low coupling Parameterization and all those things which you need to keep in mind while doing automation They fail to realize it and because of that there's a lot of problems and they end up when you're releasing once a month then you have the early majority of folks who Majority of them who have released once in two weeks and they have this huge problem of initialization state, right? So what I mean by that say You've written dentist right now and then it's all passing your super happy yay, right? Then next day you come only to notice that all the dentists are failing right now So what happened the problem was the state of the application actually changed and now all the dentists are failing, right? That's because a lot majority of those problems are because of the initial initial state of the application, right? So say for example of Amazon and you have an automation script which says you have to add items to the cart, right? so you write a automation script which add items to the cart and Awesome the test pass now you rerun the test the same exact test. What's gonna happen is It's going to add another item now There'll be two items in the car you rerun the test three items in the car four items in the car They keep going right people fail to realize that you need to make sure the app is in the initial state Like you have to have zero items in the car before running your test again, right? So similarly log in scenario log out scenario once you log into your application you may want to make sure you log out Right, so people fail to realize this initial state which causes a majority of your problem, right? So that is the third channel force challenge is maintenance and that's the biggest problem You're facing right now, but we just test maintenance, right? So Say now we have 500 tests, right? And it takes about five hours to run it and then only to find out that like hundred tests are failing Now what what do you do? You actually have to go look at every freaking test Which is there then troubleshoot it put a fix and rerun it wait for another five hours So that's this huge problem of maintenance, right and a majority of the people Have the exact maintenance problem and that's one thing which I'm going to talk about how we Are trying to solve it and how other companies and trying to solve the maintenance problem using AI So I'm going to talk about that and finally scale I already talked about scale where we need service space and you want to run more tests as frequently as possible So these are the current challenges of it software testing and test automation in general So I'm going to talk about how AI is going to help to solve all these challenges that I just talked about So before I do that, I wanted to cover the basics of AI just in three four minutes because People are mystified by this word AI they use AI machine learning deep learning And I know in our previous stocks that they mentioned a couple of differences between these two But I want to make it simple, right by no means I'm an AI expert But it's really simple if you understand the concepts. I'm going to just cover that quickly so You have this huge circle, right and The outer circle is called artificial intelligence this huge space. So what is artificial intelligence? It is giving the ability for machines to work and react like human beings Right, that's artificial intelligence Under artificial intelligence You have machine learning, which is a subset of artificial intelligence. So what is machine learning machine learning is the field of study which gives the Machines the ability to learn without being explicitly programmed, right? So that is machine learning and and and it evolved from the study of computational theory Which is study of learning algorithms and pattern recognition. So that's how machine learning evolved. So that is machine learning, right? And Now whatever you hear in the media and for the past 40 years about autonomous cars and everything related to AI a Majority of them is related to related to deep learning Which is a subset of machine learning and and so what is deep learning again? Some of you may already know but deep learning is mapped to the neural network of human beings So you have these different neurons which starts interacting with each neuron in your human body and then gives reactions and Makes you act in a certain way because all these new new runs are learning from each other So they took that took machines and that's what is deep learning, right? And all your autonomous cars and whatever you're hearing about right now is all deep learning to give you a very simple example of how it works so Say for example You want to you're giving the size Up a house you want and the price you have to pay for it and once so you give this input and then Price is the output. So what the input output combinations called a training set So whenever you meet any AI folks, they're going to talk about training set is just giving input and output combinations for the AI To learn to train the AI right so say yeah, you give the size and the price and you want to see what the AI comes up with, right? so First you give the size and the price and then you give the number of bedrooms, right and the price Then the AI starts figuring out, you know what family size is something you want to think about based on the inputs You have given and then you start giving zip code Then AI is good and determined AI is going to find a relationship called how walkable, you know that area is right It's trying figuring out based on all these input-output combinations Then you give the wealth and the price of the house then it's going to figure out the school quality so So all these things in the middle Those that's what the AI does and it's a black box and I'm just giving a really simple example of how we could figure something out But that's how it is So you give all the inputs and the outputs and then keep training the AI and then you're going to find a lot of Relationships, right? So that is deep learning and that's how it works and Your autonomous cars that's an example for deep learning and these autonomous cars you're hearing it in the past three four years, but The first research was done way back in 1989 at Stanford University and in fact they had a Video them training an autonomous car The car's name was Alvin. So this is a video from it to if you see Learned to steer by watching a person drive Alvin is designed to control the nap lab to a modified army home V equipped with sensors computers and actuators for autonomous navigation experiments The initial step in configuring Alvin is training a network to steer During training a person drives the vehicle while Alvin watches once every two seconds Alvin digitizes a video image of the road ahead and records the person's steering direction 30 by 32 pixels and provided as input to Alvin's three-layered network using the back propagation learning algorithm Alvin is trained to output the same steering direction as the human driver for that image Initially the network steering response is random about two minutes of training The network learns to accurately imitate the steering reactions of the human driver pushes the run switch and Alvin begins driving 12 times per second Alvin digitizes an image and feeds it to its neural networks each network running parallel Produces a steering direction and a measure of its confidence in its response The steering direction from the most confident network In this case the network trained for the one-lane road is used to control the vehicle as the vehicle approaches the intersection The confidence of the one-lane network decreases as it crosses the intersection and the two-lane road ahead comes into view The confidence of the two-lane network rises when its confidence rises the two-lane network is selected to steer Safely guiding the vehicle into its lane on the two-lane road This video is published in 1992 and we just hear about autonomous cars for for the past four years, right? So the reason I was stressing on the confidence level and The data we provide to the AI is because I'm going to show a demo of how People are using AI and software testing and you're going to see all these concepts the exact same thing reflecting in the tool as well So I just wanted to give you an idea that artificial intelligence was not something new So for you to To see how you can train an AI in live There's an open source application called quick draw has have any of you heard of quick draw from google So That's the best example of how you can actually keep training AI So based on the things you draw the AI keeps learning, right? And that's the best example for you to know how an AI actually works. So I have an example for that too so Once I get out of this so it's called quick draw and if you go So this how it's going to look like so you're going to draw different things and train the AI and anyone can do this this best example of deep learning here, so You click on let's draw now. It's a draw a guitar, right? Or garden hose or belt or speedboat. I see key. Oh, I know it's guitar So you're just praying you're training the AI right now. Now you have to draw a crab That's going to be a tough one actually. I'm really bad I see circle or blueberry or bracelet or teapot. I see frying pan or see turtle or spider I'm stumped See so yeah, my drawing is really bad Sorry, I couldn't get it Now you're let's try a pair I see moon or here or circle I see apple or cherry How do you do a pair of minks? I'm stumped. Sorry. I couldn't guess it. Okay, I can drop pencil for sure I see lime or hockey stick or magic wand or stream bean I see the Eiffel tower or wishbone or baseball bat. Oh, I know it's pencil exactly, right so So that's that's AI live in action, right? So that's an open source drawing tool and Google that so stumps you gets all this freaking data because now I'm using it for the presentation So they keep getting data for free and they're training the AI, right? So just want to give you an idea about how an AI works and what are the concepts behind it, right? So Now that you have this idea of the different challenges we have and how AI Works and what AI is so I'm going to talk about how we use AI in software testing, right? So Like um, there are different companies trying to do different things using AI But one thing remember at the start of the presentation I was saying saying the future is going to be moving from static IDs to Dynamic locators. So I'm going to talk about that for a bit. So I test them in my company. We use what is called dynamic locators and what that means is um We get multiple attributes for every single element using AI which is working underneath the hood, right? so for example say um, you have a login button and you write a selenium script and say I'll click on login, right? And uh, you're super happy and the next day you come and see the test fail because some developer Just decided to change the name of the login to summit and the test fails, right? And you're super pissed you have to take time to troubleshoot the issue Then you find the problem and then you rerun the test, right? And then there's so much time Waste over there. So that's where we bring the concept of dynamic locators And that's going to be the future of software testing and that's what people have started using So for the same example of login button, what happens is This AI underneath the hood It's going to go through the DOM and extract all the object trees and properties and create a list of location strategies, right? So for the same example of login button If you're going to change the name of login to summit the test is not going to fail The reason is we already have a list of location strategies which the AI created So it's going to say oh, wait a minute the name change, you know what? I'm not going to fail the test. Let's go to the next best location strategy Which could be ID, right? If the idea changed you'll say, you know what? Let's go to the next next look Next best location strategy, which could be tagged Then see I said keep going on and on because for every single attribute it gets Every single element it gets multiple attributes, right? That's what dynamic locators is And that's what we use on our tool So to show you a live example of how it actually works And this is how people are using dynamic locators in their testing, right? So Let me Take an example here Okay, so this this how our tool looks like let's just start creating a test, right? So I'm going to create new Um, I'm just going to start recording a test So I do refresh because I think I'm having poor internal connection. So give me a second and I know with live demos There's always There's going to be a hit or a miss. So I have a record of video too. So I come prepared But you know what? Let's uh Just try this out Because I know everyone is on the wi-fi connection right now. So It's taking some time Cool. Okay So right now we have the editor. Um, let's see whether it works now. So Cool. So I'm just recording a test, right? This is a demo application I created and I'm just going to click on login Enter username password click on the login button Um, I'm going to select dates from a date picker Just because I can and then um, I'm going to click on the select destination button and then In the bottom you have a book button. I'm going to click on that. Um, enter my name Email address and I think that should be Enough for now. So so as you can see, um The ai underneath our platform It actually records all the steps with the screenshots and actually displays it over here, right? But coming to the part which I want to talk about is the dynamic locators which I'm talking about, right? So say for example, let's take the select destination button so Here You can see the list of location strategies, which All the DOM elements have been Parse and then displayed over here for the select destination button, right? So what the ai has done is in real time as in when I was interacting with the element it just just Parse all the attributes of that element and displayed over here So you can see the text attribute um indexes then you also Have class tags and everything there's multiple attributes of the same element and Remember I was talking about the scoring in the video that because we use a similar kind of algorithm here I'm hovering over the stars next to each attribute and The ai says that the score for the text attribute is point nine, which is the highest you can get So it probably is going to use the text attribute to locate that element But say if we change the text to something else Then it's going to go to the next special location strategy Which could be class or type because both have the same scoring, but it makes the decision in real time so To show you a live example of Even if you change an attribute the test will continue to run because of the ai so i'm going to do something over here um So i'm going to put a break point next To the at the point where we click on the select destination button before we click on the select destination button And then i'm going to change the attributes of select destination And then i'm going to show you how the ai actually smartly figures this change. So i'm going to run this test It's going to run the test Until the point is going to click the select destination button. So it's going to click on login Enter my username password click on the login button Go through the dates from the date picker Cool and then now the test reach a break point because i haven't clicked the select destination button yet, right? and So now you see this loading icon, which means that okay, there's a break point. That's cool. So now Okay, so now i'm going to manipulate this attribute of the select destination so i'm going to do an inspect And don't worry if the demo phase again i have a recording too, but um I'm going to Whoa, what do you guys see? Okay, so i'm going to Move this button Say i'm going to move it to a location So if you see i just moved the select destination button from here to here and To make it Better i could do another thing I'm going to change the name of the button from select destination to Selenium conference 2018 right okay So if this was a situation in most of the Automation tools you use what's going to happen when you run the test It's going to fail right exactly because i pretty much did every freaking thing possible with this button so that It should surely fail, but now see what happens Then i run this test right so i'm going to click on play And there you go it just still keeps running It's going to continue the test enter my username email address that just keeps going right and Let's see what happened underneath the hood so Again going back to the select destination button So i don't know whether you guys notice but before it was 100 confidence but now the confidence level decreased to 83 percent because i manipulated the attributes and What actually happened was The since i changed the text attribute and the location It went to the next best location strategy which could be again class or tag and then it made the decision real time right and And it takes screenshots and actually shows you that hey You have this button but the attribute changed but the functionality didn't change so i continue to run the test But i just want to let you know so So it's telling you with screenshots so for example the baseline image was select destination, but What it found was selenium conference 2018 right so that's the beauty of using dynamic locators so when once you extract multiple attributes of the same element right so the test becomes more stable and and and When you have 500 tests and next day you come you don't have to worry about like oh man 250 tests fail and Now i have to put 24 hour days to actually fix it right so you don't have to worry about it So that's the beauty of dynamic locators and that's what people are using right now To and that's going to be the future of testing people are going to move from static locators to dynamic locators and Everything is possible all this is possible because the ai Underneath the hood so it's a good thing at least for this automation right so That's why i just wanted to show you a live example of how dynamic locators work So that's going to be the next best thing So this the coverage gap in software testing right now so the complexity Of different features of the software are increasing exponentially But the tests are growing only linearly right so we have this huge gap in between and that's what ai can help to solve right so That's something you want to keep in mind Now coming to the future of ai and some of the things i'm going to mention We are actually Trying to do in our company right now But again, it's a work in progress because as i said ai is a black box We need to figure out a lot of stuff about It's reacting to different put output combinations right so Let's first talk about autonomous testing so What's going to happen is and what we're trying to do in our company and i know a couple of other companies are doing the same is We're going to connect our production apps to our software testing cycle right, so what does it mean so The ai is going to start observing different flows which the actual production users Are trying to do with your apps and it's going to create tests based on the actual flows right And it's going to create tests for you and it's also going to identify Identify repeated steps like logging in logging out and it's going to cluster them for us it's going to create reusable components automatically for us right and We do it to a certain extent right now but we still Want to get to the point where we can start analyzing production User flows and create this and finally you have this test which are created Based on actual production data and that's autonomous testing and that's what everyone wants to get to right and and This what we are trying to do right now with using ai so the next one is ui tdd so everyone has heard about a tdd bdd all the dds out there, but there's one more Tdd to make it more complex which is called ui Test ui based test driven development. So what do we mean by that right? So what's going to happen is so we have these mock-ups and And developers so in general at least in companies have worked Worked in for the past 12 12 years We usually have mock-ups and then the developers will look at the mock-ups and say oh, this is a feature Okay, let me start developing right now with ai what you can do is while the developers are developing the feature based on These mock-ups the ai can look at these Images and start creating tests for us right based on the features so You're gonna so the ai is going to create the test based on the mock-ups The developer is going to develop the feature using the mock-ups and then finally Once the feature is developed You run the tests which were already created by the ai and make sure it's green right? So now you have ui based test driven development right and currently we are working on this aspect Of trying to do ui based test driven development right it's super fun, but it yeah, we again need a lot of data Probably I should do something like quick draw and make you guys do something so they can collect some data But the point is yeah, this is another thing which we are trying to do right now so Finally another big thing is automatic response. So every time You make a change in your ui say for example You change something in the login flow in the ui right you don't just go change just to test the ui you also want to make sure that Responds and request response to servers and database still continue to work because the developer would say No, I didn't change anything But of course we as testers Don't agree to that and we would want to test that whole end to end flow right? so See for example something changed the login in the ui right? Before what you have to do is you have to wait for the server To process the request it it will probably Access another database maybe and then give the necessary information and then in the response finally we get in the ui And that was a part of that was a little time consuming aspect of testing as well, but with AI what's going to happen is The a is going to record all the server responses and requests and you're going to mark it so You don't have to go all the way from A server to another server to another database. Can they get the request? Get the response back and back to the ui. You don't have to wait for that Even that time can be saved by marking Recording all these server responses and then based on whatever you do It based on the marked responses It you immediately start getting feedback right and it makes your test Much much faster, right? So that is something which People are working on currently and that's one of the things we are working as well using AI so How companies like us and other people are Trying to use AI to you know, pave the way for the future, right? So our main focus is on Solving the challenges which currently exists this automation. The main thing is the maintenance aspect of it So that's what we're currently working on but also Autonomous testing helps to eliminate the quality velocity dilemma. So what do I mean by that? So in terms of quality So you are now you've already created all the user flows based on the actual production data, right? So you're maximizing user coverage because the you're actually mimicking what the users are really doing, right? So Now the all of a sudden now QA has become a science and we make decisions based on all this data, right? We make risk based uh based be based on based our decision based on risks and data and and The biggest thing is you're already ahead of the curve because The what the AI does is it proactively? Finds problems and keeps fixing them in the background even before it comes to Comes to the screen or you see it, right? That is because of the self-healing mechanism of the AI because it it can it just keeps doing its own thing, right? And we can we don't have to worry about a lot of stuff So that's why it prevents bugs as opposed to fixing them Then coming to velocity aspect So we create much more scenarios in a short period of time, right? Because all these Tests are automatically created for us. Of course, we has human testers. We will go and check whether that's why you actually need it Right? That's why there's always a human who needs to be there. But the point is you can create much more scenarios um He can create this proactively by monitoring production apps and It's easier to do rootcast analysis as well. So Those were pretty much the things I wanted to actually cover um, hopefully now we got an idea about what the basics of AI is how people are using AI For software testing what are dynamic locators and how the AI actually Used the dynamic locators to make decisions in real time. So that's why you Everyone is seen right now if you need more information. I have my email address and everything over here and uh before the stock I already Wrote a summary of all the things I talked about and also have extra information about tools and programming languages people are using in this blog post so once if you just go to testim.io And click on blog you should be able to see that or if you have this Slide if you just click on it'll take you right there, right? And before I end the session, so I wanted to do a couple of things, right? um I know we have time so What's the time? Okay, we have five minutes left. So I wanted to do two things, right? um first thing is I Never realized that There'll be such a big crowd, which is phenomenal. So i'm going to take a selfie and you guys should say hey or something and just want to Prove the people that they were actually people in my talks. So Say cheese Okay That is one thing and another thing is I got some freebies Um, so we have uh testing t-shirts over here. So be free to take them and I didn't again Didn't know there were these many people so it's like a first come first serve basis And I have one request though once you take the t-shirt I would like you to just wear them and then we'll take a group picture and then we can blow up twitter as well so um So that's pretty much it and thanks for sitting there patiently and thank you and I'll take questions right now So Hi rash, uh, my question is that you were talking about the dynamic locators So in the application that I test what happens is uh, we have elements and most of the css the developers what they do is From the css, uh, everything is almost same right and the id is dynamic So in my application if I have two buttons with the same name just their position is uh different, right? Are you getting what I'm trying to ask? So, uh, if the if the buttons are placed differently, but the css and the x path almost it's same Just the position is different and the id is dynamic. So just as you saw in the demo, uh, somehow the developers somehow, uh, Interchanged the the location of the buttons interchange So the functionality of the buttons is different. So in that case, will it detect that the buttons has changed or will it click the same button? I mean because the the position of the buttons has just interchanged, no Right. So just to make sure, um, I got the question, right? Say you have four buttons called summit and all four buttons have Exact same css Tag names everything's the same even the id's are same. No the id's are dynamic. So we cannot use id's but and the names are also Same. Yeah, right and so somehow the two of the buttons got interchanged by uh, because of the developer or some Some mistake So in that case, so my question I Coming I'll answer that but my question was if you have four summit buttons Who actually cares if it actually changes it's the same button, right? No, no I mean like the name is same just it's just an example. The name is same but the functionality is different Yeah, the functionality of the button is different. Yeah. Okay. First thing is I would say that's a terrible coding there Um, I I don't know if they were in the last session, but we were talking about product managers and business people so First that has to be really fixed no matter what tool you use And next thing was coming to the ai aspect of it. So what happens is Yeah, as in when you keep running the test it starts gathering data if it sees that okay all these id's are changing dynamically then it starts learning, right and Again only based on the amount of data and amount of times you run the ai is going to learn So After a certain period of time it will start detecting the detecting the changes and start self-healing Which means that it's going to detect all these changes even though the id's are dynamically changing and everything is the same So it is going to becoming smarter It'll become smarter and start trying to identify all these changes and then work properly But even if you do use ai it's going to take some time for it to learn But I would first say is go to your developers and business people to actually change it But to answer your question is the ai can do it, but it will take a lot of Test runs and then more data to do it, right? So only once we have That scenario and keep running it then we'll start learning about stuff. So hopefully that Kind of answer your question because it's hard then is there any plan like if you If you click any button then the page that renders after it. I mean, is there any plan of you know, is there any way? I mean, maybe you have some plan of Implementing that into your ai application. I mean what happens after the button is clicked the dom that is rendered Maybe that can also be used later on to identify. What is the functionality of the button? Great. So we already do that. So for every element we interact with we take screenshots and Ai already knows what's the next thing which is supposed to come So for example, you're clicking on the same example summit button and say a different page was supposed to Display it already has taken the screenshot and if it's failing it's going to keep looking Why is this page not coming right? So let's start learning. So the point is that is already implemented So it is going to detect that dom change and make sure the right page gets displayed based on the click Thank you. Yeah, thank you I'm sorry guys. We are out of time. Oh, I'm so sorry But raj is going to be here and we can definitely reach out to him Oh, yeah, and their t-shirts here and we're going to take a picture as well once you get the t-shirt So and I'm going to stay here. So feel free to ask me questions. So thank you. Appreciate everyone. Thanks raj