 How many are there who actually do not like to be interviewed? What do you think we guys are here for? One is the question is about this session and the next question is about this conference. Why are we guys here today? Are we here for the 18% number that is there? Are we here to interview? Are we here to convince? Or are we here for something else? How can we advocate today? Open source. Open source. Anything else? Why are we here at this conference? Meet the session also. Collaborate. Collaborate. So I'll start with a very small video that will sum up why we are all here for this interview. So pay attention. Now you will look at how the video is actually shot. It's quite easy to miss something that we are not looking at. And that happens in manual testing as well as automation. So because in today's world, a lot of our applications are involved. We've all been so far. A lot of things keep on changing. They keep changing so fast that manual testing is not able to cope up with this. Testing has become important not only at manual level but at automation also. And that is the key thing that we are here. We are not here just specifically for Selenium as a tool. We are here as a community who wants to do automation testing in a better way. And that is why automation is becoming more and more important in these days. Not just to give you a background about myself. I used to work with Infosys for close to 8 years. I started my IT career here in Bangalore with Infosys training. In 2012, I quit IP whenever in the US. And my passion has always been there for automation. So I wanted to start a venture of my own. I took a break for close to 1.5 years, wrote my third book during that time. And after that, I started this company called Tablabs. So I'll be talking more about my company in the next slide. But my focus has not been on Selenium or UTP just as a tool for automation as a tool. So how we can use automation to actually help anyone with any kind of task. So it's not just about testing that automation is important. It is even there are a lot of things that actually happen on the production. Which could be log analysis, which could be processing batches, payments, all these things. There is a lot of automation that goes into that process. Today's presentation that we have, it's not about any sales pitch that we are here for. I am here to show you things about the migration that we did for a client. We had created their UTP soaps 3 years, 4 years back. They were running quite fine. It's not that we were having a huge failure. Or there was a very big problem with their UTP soaps that we started with an idea for migration. So before I jump into that about the venture I started, we started just last year. We are a small company which is more experimenting a lot in the automation domain right now. And our prime focus is automation. But we are not limiting ourselves to Selenium or UTP or anything. We are actually right now experimenting in a lot of domains. Which could be node.js, esp, icon, server, site optimization and all those stuff. And we do create bots, automated bots, automated fiddles. A lot of stuff we do right now. We are more into services domain for now. And we do plan to use, gather some experience in this field. And later maybe come up with a product that could help the masses testing better. The agenda for today's presentation would be I will be talking about what was the application that we were testing. What was the existing test suit about? What were the existing challenges with the current test suit? And why did we start with a migration? How UTP versus Selenium? How different was it? And the results of that migration. So anyone here in the room has done a similar migration for any clients? Okay. So that's why you raise the hand if you would like to do that. So the application test was a web app. It was for hospitals. Hospitals would actually manage their patients, doctors, medication, diagnosis. Everything from that interface. They had a mobile app also. The backend was ASP. It's not an MS SQL. Now as a UI tester or as a Selenium tester, it does not matter to us what that backend was all about. And the front-end was HTML and jQuery. They were not on HTML and Python, but they were using normal HTML stuff. But they were not on jQuery and 75% of the fields on every screen were Ajax auto-populated. And each behaved in a different manner. Some were behaving in such a way that all the list of whatever possible values are there will be populated. Someone based on what you are actually entering into that domain. That is what you get as a result set. Now about the existing suit, it was around 30 test cases. These were large test cases. So it's not where we go in one screen and say, yes, the screen is working. No. Every test case, we were going around 10 to 12 screens and performing different actions. And making sure that the board was through from start to end was working. Now we had a single machine which we used to run this. We used to use the native QTP reporting. The runtime of the board was close to 3 hours. Not exactly just under 3 hours. And execution that we used to manage was using HP QC, which was the version 11. Rousers are trying to test if it's on I family and Firefox was sometimes they would use Firefox as well. Now inconsistency failure with our existing suit, 10% of the test cases would fail for no reason. So that was inconsistency wherein let's say you populated a field using Ajax because somehow the page did not record it. And when you go to the next page, it says that the field is empty. Now those kinds of errors we used to get in this application when we were working with QTP. Now QTP had nothing to do with that. But it was that sometimes the replay is so fast or sometimes that click does not happen in the way we wanted. Because of that, there would be a failure on the application and we would just rerun the whole field test cases. These are the challenges that I was talking to you about. So we had to launch this 15 to 25 minute execution time for each test. And these are the ones which had 40% chances of failure. So it is that we had to most of the times rerun these test cases to get the whole suit to pass. And it is nothing to do with the test case but rather the application as the interaction between and making sure whether the application is not in that state. So now you have let's say 10 screens in between where every field you have Ajax autocomplete and you don't know which screen could fit. So you cannot win the replay logic at every place and you only get that error when moving to the next page. So we wanted to do something better in that place where it would be easier for us to actually handle such situations. Next thing was in QTP, the worst part is it uses vd script as the language and the stack traces that you get in the language. They have no information about the call stack. So you just get an error that let's say divide by 0 error. You sometimes do get a line number but in case you are dynamically loading your libraries it would say line number 1 divide by 0 error and you have no way to easily trace that. Now we picked the add-on for QTP also to actually enhance that and we call it power decal. So that was released as a pre-wear tool 2-3 years back by me. So that is used by some of the companies right now but it is only to enhance your stack trace when you work with vd script and it's a debugger written in Seesha. How did we come up with an idea of migration? Because the pain points that we had were not so strong where it let's say you would want to go and evaluate that let's kill QTP all together. The license cost that we were paying were maintenance because we had already purchased and we were paying like $1500 a year which is not much for a corporate client. So the idea of migration was not to reduce the dual cost. What we did is we were just experimenting on Selenium from our own side and we just thought let's create a simple scenario that we do right now in QTP. And the difference that we noted in the execution time was average test took 2.75 minutes in Seesha compared to 4.5 minutes of QTP. Now why would there be a huge difference? Is the application performing better under QTP? No. Application would perform the same. So how is there a huge difference between them? So what happens is when we run any test from QTP which is the test runner for QTP the QTP makes a connection to your local QTP instance. Similarly it starts the test so there is some boot up time on the Google page and once your test is done the tool stops which means you cannot keep anything loaded in the memory which could speed up your next test. So it's a failure of architecture that QTP had. So that's why you see there is a huge difference of timing between what we did in Seesha and what we did in Selenium. Next thing because this test was only using Seesha and no framework altogether. So of course you will see better performance. Once you start loading stuff, once you start creating wrappers you might see some substantial things but it all depends on how you have created that framework. So now in this demo session I will be showing you what we did and I will be giving you a very small demo script of how easy it is to create a script in this print. Now people who are familiar with QTP and Selenium there is a huge difference when you jump from QTP to Selenium. Somebody who is actually working on Selenium and jumps to QTP might find things to be lot more easier but somebody who is used to QTP, I used to work for QTP around 8 years and jumping on to Selenium so I had good developer skills but it is not easy to jump from one tool to another where one tool is giving you everything that you need and the other tool is giving you a very base set of an API on which you can develop a bomb. So next major difference that we found out between the two was how they create each browser. So Selenium connects to a browser using a session. Once your test is stopped the session is lost. So you do not connect back to the same browser. Instead QTP works at a UI level where if you have three browsers open it will be able to identify each browser one by one. So you can say connect me to browser one, connect me to browser two. So what that advantage it gives you is let's say you have a huge test case. Now it breaks on the 50th point and you have five more steps to go. In Selenium you might not be able to recover that whole scenario and you might run it. You might fix the issue, run it from the start. And in QTP you could just run it from their step and QTP will automatically connect to that existing browser. Now this may not look like a big deal. But most of the time that is actually spent in any automation project initially is in the development phase, next is in the maintenance phase. So your unit testing time for your actual script if that is reduced that means your development time your time to maintain also gets reduced. So that's why this was a very big difference that we felt was there between the two things. Now next thing is any time you, when you do use QTP you want to pay to ask what type of object you are it could say vector group. Now you go in the Selenium world you ask everyone it's just familiar. And then somebody would raise their hand, hey I'm a select. So you just have one select element which was supported by the additional BLL certificate or the additional package that we get from the library. So Selenium is so focused on providing a base for you that there is no differentiation between object type which is actually required when you build anything which is on a bigger scale. Because different types of objects will have different types of actions you can take and they would need different types of data. So this was a big difference between these two technologies and it sometimes makes you feel that maybe we are spending too much effort on the migration. So the initial challenges that we faced there is there are no functions for tables Selenium does not identify what exactly is a table. There is no easy method to check an object existence. You have to check for exceptions to check if an object exists or not. There is nothing for you to use to wait for an object. You will need a wait condition or something to use which is quite easy in QTP. And there is nothing like an object repository. Now those who are not aware of an object repository, object repository is basically a map of your application and how to identify an object within your application. Now the only thing you need to know when creating an object repository in QTP is how to use your mouse which all of us do. So it's a point and click system and what that does is it does spoil you. You do not know anything about HTML, no worries. In background QTP will start adding stuff to you. It will learn the ID, it will learn rest of the properties, it will see if there is a unique match, if not let's add more properties and suddenly you have something that works for you. But you do not understand exactly why. People in the QTP community, they do not use XPARs, they do not use CSS. So what that does is it takes away from the technology and understanding how things work rather you tend that it would work. Which is not always the situation when the application gets complex which is the case in most of the applications today. So you will not get an easy to automate application where something that was supposed to work does not work. That is where your skills come in. See the tool can work to some level but how many of you have created work arounds in your code for cases where your application was supposed to let's say handle an event or something and it did not and then you created a work around by let's say creating it first then entering the data. How many of you actually code work arounds at night to get the application working? So a lot of you. So that is the reason what we found is that tools like QTP which make it easy for you to use a lot of stuff take you away from the technology and they do not let you learn more. There are people who actually learn when working with QTP also but Selenium being at a developer level for a tool where you have to start learning it's part you have to understand CSS you have to understand HTML gets a better chance that you will be knowing the technology better compared to what you would have done when you were using So any questions still here How many of you have worked in a Python robot framework? So that's a framework which was developed in a company at Nokia or somebody developed it and they then made it open source So it is actually being used by some of the community and it's an English kind of framework I wouldn't say it's similar to Cougar or anything but it is something where you can write keywords which are more in English let's say click a button all that Now I did like that idea but somehow I was not advising them I was not pretty much into and plus our client wanted this to be on C-Shack because they were more comfortable with C-Shack So we created a complete new framework and we started with certain goals in mind What were these goals? We wanted to reduce the logging effort it takes to actually log what you are doing So you would have created your own code then you enter that statement saying we are clicking on this button the button succeeded So you would know that we spent a lot of time in actually writing code which is just supporting our automation and it's not part of the core automation So when you look at any of the code you'll see a lot of stuff is not doing anything related to automation but doing rest of the supporting stuff Let's say taking a screenshot as we saw in the review session wherein you have a method which is already added to take that screenshot Then object repository we wanted to bring that concept back from QTP to Selenium We did not want page object factors Now some of you may not agree with that kind of an approach where we keep page object patterns but my idea of killing page object pattern was that finding your identifiers, HTML locators into your code is not the right approach Your scripts are more about testing the application they are not more about defining what an application is So if you could separate these two parts out and use them in an integrated fashion where the map is external then it would have more benefit compared to when you merge them And let's say I have seen many cases where in one environment of an application we have to use one identifier in another application we have to use another identifier So you don't want to create multiple classes to work around the various which page object pattern should we load now So those are the things that we felt were not right HTML reporting we wanted to write less code Everyone's lies to write less So that it is efficient, more productive to create something Parameterization and debugging for existing failures Like I said, we did not want to scratch the whole test case again Now there are few key features that we added to this framework I will... So would you want to look at the demo first or you want to have the features first? What do you get from that? So what we did is we used C sharp and there is an open source editor for C sharp that's called sharp developer So it's a very good tool if you are developing an open source and you do not want to go for something like visual studio or anything And it is very good for C sharp So it may not be good for a VP.NET project but it is very good and it integrates well with C sharp So we wrote this whole framework from scratch in C sharp using just the Selenium Web Press APIs And you can see that I have a very small structure over here and so we have created a small demo project over here and we have few folders like data, keywords then object repository and the testing So object repository is something wherein we keep all the object definitions So in this demo I will show you how exactly we automate Google So we will not be going into very big scenarios but I will show you how our test cases look when we had actually developed it for a natural application The object repository and this is how it looks like So anything that starts with a dollar and the curly braces those are our variables What you see is we use a convention of page.i from object and the name of the object You have pg which means page Google search.txt search is equal to name equal to queue So we are defining that wherever we use this replace it with name equal to queue Now one good thing about this approach is that it is a text file very easy and fast to text it You can do versioning on this particular code point easily compared to comparing an Excel if you use an Excel or something it is difficult to compare both files but if you are using a text file it is very easy to see the description using any tool So even if you are using Git, SPN or the Microsoft visual source So that it is quite easy to compare text files and see what are the differences and maintain versioning for all of your code Next cool part is ask is not a compiled object repository which means let's say you find a bug you won't need to go and recompile your stuff you just come in here make the change and read the text So I am going to just change this btmg to btmq so that when we go for a demo we do create it So now we have got an object repository ready I tell you the first step that we have to do in this particular framework Next thing is to create a setup file which tells the framework that let's load this object repository So we have something called init.kw that is actually used to load an object repository You can have actually any code in this init file but you don't want to use a lot of the selenium functions over here The first thing that I do is set an object timeout So this we use internally for setting some timeout on when it will happen But we do have certain retry mechanism which means the object timeout will not be exactly the same because the framework does try another attempt to recover If it can recover your application from that error or your test from that error, it will recover So which means the number is not actually followed very lightly but it does reduce the time it would take for init.kw to happen The next statement is load over and you just have to get the project based path So which is object repository as Google.org And just for my reference I have printed this link and I know that this particular configuration was actually launched Now next thing that comes is let's start writing our soon Now soon could be anything, a test case, you calling another test case or you writing the whole code So when we are actually developing in this framework we want to just write code and test it directly and once we have tested it we want to move it into a test kit keyword I'll show you how keywords work but let's start with a test So the first thing that we do in any test would be to launch our browser I want to launch, you hit control space you get list of all the keywords that are available So I want to launch a browser and I want to go to Now colon has a special meaning in our framework So you just have to test the keyword It is actually used for passing parameters for any parameter without ordering So in case let's say you want to pass the last parameter, the value the rest order you want different then we can do it So, sorry, I assume that we are quite possible and next thing we want to do is we want to go to this particular So next thing that we would have to do is enter something So let's say I have, now we do not have right now something like You can just click this, click your user We will use them while searching on Google So next thing we want to do is we want to set the text box that is the Google search So I just copy So we do not have intelligence right now developed for this which would dynamically load in all the variables that you have created but we are working on that change too so that we just hit control space you would actually get whatever variables there are I want to enter my full name which is not actually there directly but it is separated into two different parameters So how would I do that? I would directly enter first name give it a space So the framework will automatically work on the concatenation This is where one thing we have seen is we write a lot of code to actually merge stuff create let's say where we have to do validation where if something that comes out would be order number, colon and that number and we do either split all that stuff So instead of doing that what we do is if you ask certain number of parameters in series they will be evaluated and the evaluation is nested What do you mean by nested? Which means let's say if I say dollar something and I put another dollar inside that So this dollar test would be evaluated first let's say it has first name then dollar first name will give So you have nested variable which means dynamic programming Next thing you want to do is click on this particular button and take a screenshot and save it like this Now one thing we found out is in most of the unit testing frameworks you have something called an SR So you want to SR in case there is something wrong then you want to let the test experience and what that does is it actually can go beyond So let's say if the logout link is saying something else there is a typo in the logout link I don't want to verify that I just want to verify that okay it is saying logout but if it is not saying still I can manage and continue my test with rest of the stuff So we built in two things with this One is verification point The other is assertion point So you can verify it which would continue the test case but fail the test at SR which would fail the test case there is So let us just run this and save it So you get a console window which basically tells you which action is being executed right now So it will launch firehooks first it will search for that name Now there would be an error We have changed that identifier in the object repository So it will try and click on between something but it would not be able to do that So let us just wait and let it wait So as I said we ignore the exception first and try and see if we can recover if not then we let it wait So right now we are in debug mode so you see some breakpoints and all but once that is done you can see that our test case actually failed Now good part about this is I can go ahead and change that to change that to the real thing and make my demo soon Now I don't want to go back to Google again So what I do is I just comment all of this Selenium does not allow us to connect to an existing browser We already lost that session So what we did is in our framework itself we went in a mechanism where you say existing you learn it the click happened the screen shot and I could resume my test case from the point where it failed Now that's a very small test case I am showing you but when you are looking at bigger test cases where it lets say you have gone down 20, 10 pages, 15 pages in the flow it really is necessary for us to be able to resume from that point and test it out So you can see that it was not very difficult for anyone to actually learn how they can write a small test case in this particular language it's more like an ASL language for Selenium and let me just show you a small test case that we had for our migration So what you can see is we have a keyword which starts with pc that means you have a test case the documentation what the test case is doing location it is not right now but we will use it in the future and you can see that we get the current type based on that generator new name So every time we add new patient we want it to be unique and the best thing is to generate that name from the time itself Then we call default login for the home screen pages will activation form save as add build button wait for busy and we click on the billing code Now you can see this wait for busy and all that stuff this is dynamically waiting for that busy item you do not have hard code as soon as the wait is over as soon as you see that image disappeared it would go ahead and update so moving back on to this we have automated reporting which means you do not have to write anything once we have created a method you can specify that how that method should document itself in the report then verify any time you want to you can just add additional that would then object repository already we saw how it is then we added built-in support for jquery and sizzle selectors also you can just start your selectors saying sizzle equal to or dollar dollar equal to and you can use those selectors you do not have to worry about whether the web page is actually using sizzle or jquery or not automatically inject jquery at one time then we added support for label selectors where applications which are actually using label can directly interact and we solved a big problem of running the test cases from between which is existing by and we saw in the demo now we added one more thing what Ajax click when you do the click the Ajax you can tell that I am looking for two Ajax calls after this click and that click will only return after I get a time out or two Ajax calls have completed so what that does is it evils ellipse you wait for objects you do not have to actually wait for an object but if there is one background operation that is going to happen the framework can judge that this URL started and the response is back now we can continue so we built it Ajax click but it cannot work on an application like Facebook which continuously monitors Ajax using Ajax they can continuously poll and get you it so this is a small demo report the framework creates it is not something that looks very neat and clean but something that does get the job done for us what you can see is we can easily filter out anything information I can search and it will filter any step that is related to that and then you get screen shots also which are smaller in size but if you over over them they just zoom in very good lower resolution so what we did is we built Excel based test runner for them what it does is it actually gives the light status of every test case that is running and the test that we have we could select how many number of browsers so you can see it gives an estimated time cut it would take 18 minutes to test the whole suit now we had intelligence as I already showed you what was the comparison when we ran the whole suit in selenium it took 19 minutes compared to 165 minutes on a single machine that used to be possible which is 88% selenium the 60% and it took us 4 months to actually develop this against the 2 months that we spent but we had to do lot of work from scratch you can see that though there is a huge difference but once we start developing over it over time we will be taking less time compared to what we would have if we were adding more and more this is something that we again made for them so it's a performance test that we wrote with 226 user simulated on a single machine using selenium so I think we are already ahead of time any questions that you have I take it offering outside the conference and I am here for 2 days so in case anyone of you want to connect I will be here in a minute so thanks everyone sorry I have to take