 My name is Narayan Raman, I am the author of an open source tool called Sahee and this was released in 2005, it was a test automation and web automation. About 3-4 years down the line I started a company called Tito Software that helps organizations with their automation process, okay. So we are a product company, we are not a services company, we are a product company, it is a product that we have called Sahee Pro and that actually we have been selling it to organizations which have faced specific problems with automation. Now what I am going to talk about today is not about the product itself, it is about some of the principles that have helped our customers to have automation which keeps pace with their product. What I mean by that is products evolve pretty fast especially nowadays, right, like you have all these processes around it, development has become much easier and the expectation of the business is that people deliver very quickly. Now when that happens you also have these cycles of releases which are quite short and when that happens you need automation also to be squeezed into those cycles. Now in fact you have, you need testing squeezed into those cycles. Automation is helping you to actually squeeze them there, okay. Now I know that a lot of you are, this is an agile conference, you may be thinking of Scrum, you may be thinking of how do I get it into this Kanban cycle or whatever that is. Now we like, my talk is not going to address that at all. What we are going to talk about is this on the basic main pretext, software exists because it enables a business, okay. Now this I kind of stress upon because we get carried away software for software sake kind of thing. But that real thing is software enables a business, okay and you need better quality of software to minimize the risk to business. So if you actually are a banking organization and you have a web application which is not great or has bugs in it, you are basically hampering your own business, you are not allowing yourself to succeed. In dynamic businesses where the business is changing fast, where both software and business change. So in the sense that there may be business changes which means that you know the newer kind of payments banking has started in India. So that was not there before. So that is a new kind of business. The other thing is you have the same kind of business let us say a bank but now you have a mobile interface through which you want everybody to actually interact. So these are different ways by which like businesses and their implementations are dynamic and software quality systems should ensure continuity of such business. Whenever something new comes in a business it should be ensured that doesn't have it doesn't add any risk. In this talk we are talking going to talk about automated tests which ensure quality of software, okay. So why am I talking about all this? Because when we talk about automation we become tool centric in our head, we become framework centric in our head to set the context that you know this is not specific to any of those things, okay. While it will have examples it is not specific to any of those. So now I will jump into the little bit more technical thing, right. So when do you use automation in your, in your let's say how many of you do web automation in your organizations right now? Okay just before I go to that slide let me ask you like what problems are you facing right now? Just to actually if you can just shout out what problem you face, what is, what do you think is the problem? Why are you unable to keep pace with evolution? How do you feel we are keeping pace with evolution? No problem at all. How many of you are fine with the pace of your automation right now? Okay so like what problems are you facing? Do you have any words that crop up in your mind quickly? Sure, okay. So most people feel that you know this is what like we've realized when talking to people. When people think about automation failure they think about flaky tests, unreliability of their giving you results that you can believe on all the time. So a lot of times the suspicion of a failure lands on hey maybe we need to add some weights there, maybe we need to tweak that particular thing there, right. So a lot of times like automation is thought of at that level, okay. So let me just take that and we'll go to the next step, okay. Can your automation help when migrating a jQuery application to AngularJS, these are JavaScript frameworks. You may be migrating from one framework to the other, okay. Can your automation scripts work when you're moving from Angular 1 to Angular 2, when I say like can automation help, it means this. To your business you had an application which was working fine with jQuery, okay. Now there are a few more requirements that people are asking for and the development team feels that it may help to actually migrate that particular user interface to let's say Angular, okay. Or they want to have parts of their overall application in Angular. Now your automation tests should ideally allow you to migrate from one to the other with minimal changes so that you can catch differences in the behavior between these two things, okay. The next thing is how about migrating from Angular 1 to Angular 2, similar problem, okay. This is a very real problem actually, Angular 1 to Angular 2, people struggled with it because there was a thing called protractor which came out with Angular 1 itself. Have you ever used protractor? So you start coding with that, you had your automation scripts with that. When Angular 2 came, it was not immediately compatible with protractor. So a lot of the automation scripts went for a toss at that time. But then afterwards, two, three versions down the line, they actually like added support for it. If you have worked with Salesforce applications, okay. There was a Salesforce classic user interface then it moved to Salesforce lightning interface that is what they are actually like trying to move people over to. Now, in this case also, there is changes in the user interface. There are changes to various things. Can your automation help? When migrating an applet or a flex application to web, this is totally different technology, right? Can your automation scripts help you in that? When extending a web application to mobile? Again, user interfaces are different, technologies are different. So this is normally the problem, rapid development is the need of the day. Automated tests, in these cases that I spoke about, it should ideally help. If it helped, think about it. Your automation scripts are working across these technologies. So how confidently can you move? It is much better, right? You have a very good harness system like a net that you can actually utilize when you are migrating these technologies. You can make changes, you can make sure that you verify that the automation scripts are passing, hey awesome, if it works, very nice, right? So it gives you a lot of confidence before the release. If whatever used to work before will continue to work within your own. But automation scripts break easily when applications will work. This is the main problem and fixing automation scripts steals time from quality testing. What I mean by this is, a lot of times people spend more time in fixing those automation scripts and maintaining them than actually finding new bugs in the system, okay? Automation efforts stagnates if you feel that suddenly the stakeholder will say, hey, you know what? I want the release now. Customers are waiting for it. You have to release it. What goes out of the, what do you throw out first thing? Hey, forget automation. Let's fix it later. Let's actually just manually test it and release it. It'll fix the automation later, right? So that is a real problem. You start doing that and then automation slowly is no longer in sync with your product. It starts stagnating. So you need to have a way by which it can, it is fast enough, okay? So what does it take to be in sync? Always like, in any task, time needed depends on the total amount of effort by the number of people who can actually solve that. So in automation tasks, okay? If you look at a normal automation thing that you do, you have all this code that you write to do things that the user interfaced it, okay? Or the user, sorry, not the user, what the user did? Now, okay, just to step back on this, okay? When we are talking about this, we are talking about UI automation test, okay? Now, UI automation is a small portion of your overall automated test cases that you'll write, okay? There are these unit level test, there are these like service level test, there are different layers you write those test cases and the more number of tests you have from the bottom up, the easier it is for you to actually deliver your product. And UI automation is just a small portion of it, okay? Your dependence on this is absolutely wrong, okay? But ignoring this may also cause you business trouble because the end user is interacting with the user interface. So you should not ignore it, but you should not overdo UI automation either, okay? Now, here, when I'm talking about your automation task here, I'm talking about UI automation. So what are the different phases of UI automation? One is creation of scripts or automation code, execution, then reporting, analysis and fixing, okay? Now, these can, some of these themselves can be automated, okay? The creation of automation, people say that recorders are bad, right? How many of you believe recorders are bad, okay? So the thing is, some recorders are bad. There are like, there are, there are tools which have good recorders also and if your application lends itself to recording, you may be able to automate part of your creation task using those automated tools, okay? So the creation process can be, if not the recorder, there is definitely a use for what is called, you know, an element spy or an object spy, okay? Or what you do with, you know, right like inspect element kind of thing. These are all like elements which, these are programmatic ways of identifying the element and giving you a way by which you can, you know, interact with them. So these things are useful, okay? Now, you need to utilize them to actually make your creation process simpler. So creation can be manual and machine, manual because you know what your test cases, you know what your business is. Only then can you actually automate that scenario. Execution is of course machine, okay? So that is when you get automation results, automation benefits. Next thing is reporting. This also can be automated. Analysis is manual. The sense that when things fail, you need to look at it manually and figure out why it is failing, what is happening, etc. And the fixing is also manual and is also iterative, okay? Why are we talking about these things? Because different, each of these things take different amounts of time. So if you look at fixing a bug, okay, in the test automation script, you go, you play it, you look for, look at an error, you go and figure out on the application, fix it. You run it again, you get to the next error, then you actually go back and fix it. So this is an iterative process, okay? And the longer your scripts are, the more time you waste in them, right? So what we want to do is reduce the effort. We want to offload to the machine as much as possible, okay? And increase people contribution. Now, increasing people contribution. The first part is of course the reducing effort, offload to machine as much as possible. We can do it through the automation tool. Increase people contribution is also something that we can do by choosing the right kind of people and the tools, okay? We want to make sure that automation is not one person's per view. It is not just the DevTest guy or the developer or the testers whole. It's not owned by that person, those people alone. We want to maximize the ability to contribute to your automation script, okay? The first thing I would say is like this is very code centric. Use version controlling. Why do we say that? Version controlling solves a lot of problems that people are still struggling with, okay? People actually copy paste their code into a central folder, shared folder, everybody takes a copy of it, works on it. This is still happening, okay? So when people actually move to version control systems, they automatically solve a lot of these problems. And even non-binary files, we suggest that you use text-based files. For example, Excel, you store it as a CSV and put it into the version control system, okay? The next thing is how do you enable everybody to contribute to this? So you go into smaller role-specific files. So this is what we're talking about is who are the different people who actually are in a team? Broadly, business analysts, quality analysts, and developers, okay? Now, these people have slightly different kind of mindsets. They have different kinds of competencies, okay? Developers, of course, are like super thrilled about writing code, okay? BAs and QAs, depending on what kind of organization it is and what kind of team you have built, may or may not be so thrilled about it. Especially in an organization which is domain knowledge specific, for example, a banking domain. Now here, the business analysts don't necessarily want to write code. They can very well interact with an Excel-like interface. They can actually formulate all their use cases. They can give you manual test cases, return an Excel that you can actually execute. But they may not be thrilled to write code there, okay? Similarly with quality analysts, should they be spending all their time writing code? Or should they be doing like finding newer bugs, exploring what is happening in the overall product, automating what they don't want to repeat all the time? That is what their overall focus should be. Now, because this, and QAs again, like testers, ideally they should be like having a mindset of figuring out problems in the application. They shouldn't have to actually write code, if possible. Of course, they will have to, but if lesser, better. Now, what we say is, break down your automation code into ways that you can enable all three of these, okay? For business analysts, you may want to expose an Excel-like user interface. Interaction code that actually performs actions on the browser, can be done by somebody else and some other things that we can do, okay? But basically what we want to do is separate out so that everybody can get involved, okay? So, and the other thing we say is minimize programming, if possible. Programming seems to be like, hey, everybody should do it. Maybe, do it for the right, right reasons, if there is something that already works without programming, use that, okay? What are the types of changes in applications? Now, mostly you can break them down into three, okay? So let me just show you a small thing before that. So, this is a small application. I'm going to, you can see this here, okay? Now, follow what I'm doing here. I'm just going, I'm logging in into the system. This is showing me a user interface, like this is a banking user interface. If anybody gets, anybody can guess which bank this looks like, yes, yes, yes. So, this, this interface, right? Like, a simple case maybe, I log in, I see that I have 25,000 available here. I go to funds transfer and transfer some money to Rajesh, let's say, 1000 rupees for, let's say, books. I click on continue, confirm and then go to the accounts and check that the total is lesser now, right? It's 24,000, 24,000, okay? This is a simplistic test case, right? So, if you were to automate it, what comes to your mind when you're automating this? What is the first thing that you may be looking, thinking about? If I said that let's automate this, start from logging in, very nice, okay? Now, if you ask this question to some subsets of people, they'll say, okay, how do you log in? You actually put in the username into that field, put in the password into that field, click on logging, okay? Then you go into that, so there are two ways of talking about this problem. One, as you said, is start with log in. The other is to say that, hey, interact with the system by putting in the username, the password and clicking on log in button, okay? Now, these are different things, okay? Now, let me actually, you've seen this interface now, right? Can you see this? It's a little hazy, right? Okay, let's see, let me just do this, okay? Now, look at this, what is written here, okay? This is log in retail user with test and secret, verify that the balance is 25,000 rupees, transfer money to Rajesh 1000 for movie tickets, verify the balance is 24,000 rupees and log in, okay? Now, this test case, does it talk about what is your user interface? Does it say that it's a web interface? No, does it say that it's a mobile interface? No, right, like all it says is this is what you're doing here, okay? Now, is this what is important to the business? Absolutely, right? This is the business scenario, what are you doing? You are a bank, you're transferring money and making sure that it is detected correctly, right? Now, if you had actually, if this test cases can be given to your business analysts, they'll be happy to maintain this and they can write more of these test cases, right? Any thoughts on this? Like, do you think like this is something that the business analysts may not want to actually work with? This is definitely better than giving all the interaction code to that person, right? So, one layer is the business layer, okay? And here, we do not have any interaction code at all, okay? So, you must have heard about Cucumber and other tools and a lot of times, while the people who actually wrote Cucumber say that, you know, don't put any steps of the interaction in it, most Cucumber implementations we see have interactions in them. They'll talk about, you know, put the username as this, put the password as this, click on login, then on the next screen, do this. But the screen specific logic should not be in the outermost layer. Now, if you have a test scripts like this, it doesn't matter whether your actual code is implemented in Angular 1 or Angular 2, whether it is implemented as, you know, a desktop application or a mobile application. This will survive across your backend changes and your user interface changes. But how do you actually make this run, right? So, the next layer that we talk about is this layer, okay? So, from an interface which look like an Excel that can be modified by business analyst, we get into a little, here we get a little more intimate with the application itself, okay? And in this, what are we doing? We're actually saying that, okay, login as a retail user, okay? Here in, you do the actual interaction with the application. You're saying set value into the text box, set value into the password, click login, submit, right? So, this is what you have broken it down to. And what are these text box user, etc., these are the elements that you're interacting with, okay? Now, these elements are very specific to your web application, okay? Now, if your code changed, let's say if you are back in, if your UI framework changed, it is likely that some of these things will change, these elements. And what are these actually? You see that this actually defines the actual element on the browser, okay? This is where it is actually coming to the nitty gritty of what it is on the browser, okay? Now, if you were to execute this whole thing, so let's say that we played back on Firefox, okay? Just kidding. So, the start URL is wrong, I'm just changing that. So, this will launch a browser and execute the test cases on that, okay? Well, it's demo time, give it a minute. Yes? So, the thing is that these test cases run very fast. So, you don't really will not have time to actually use it at that point of time. What you normally do is like you offload it to other machines to actually execute, okay? So, this is, and this again, like you're getting into the nitty gritties of how this tool is executing, but the point is not that. The point is that you have these different layers in the automation and you want to make those things execute together, okay? Every part is part of your automation code, but can be owned by different stakeholders. For example, the Excel like user interface can be owned by your business analysts. The interaction can be owned by your QA and the element identification. There is an interesting play we do on that, which is actually to say, you may be able to have a smoke test which actually passes through all those user elements and that can be run with your developers build, okay? What that allows you to do is when you're this smoke test only goes through the elements that normally are exercised in about like 70, 80% of your application flow. For example, in a banking application, you will have to log in, you will have to like go through accounts like, you know, funds transfer, etc. Most of these things involve that. So, you just have like smoke test which actually go through these and assert on these various elements once, okay? Now, when you do these assertions and you actually hook in the script along with your developer's build, the developer is making UI changes, his build fails because of this particular thing, he goes back and fixes it directly. Now, it doesn't even come to the QA before like when there are just element changes. What that helps you do is, you know, see, there are different in your overall how you set up your continuous integration, etc. There are these smaller smoke test that you want to run with each build. And there are these like larger speeds which actually run for 8 hours, 10 hours, etc., which you want to execute like, you know, overnight, okay? Now, you don't want to trigger these large tests without ensuring that most of the code flow will work correctly, okay? So, what you do is, initially, you go on and execute the smoke test. Now, this if it failed and the developer is fixing it, the developer build is not even passing over to the QA for actually executing all these nightly builds. So, what you do is make sure that they actually pass before you pass it on. And for that, if the developer owns those tests, very nice. Yes, so the thing is, so that is where, right? Like, what are the different kinds of changes, right? Like, so I'll just like, so here actually just laid back, let me do that again. Right, it's doing all of that and like, and doing, you know, finishing. And if you look at the reporting, it actually gives you something like this. So, these are the different steps that you perform. So, even when analyzing reports, okay? This should be accessible and, you know, like meaningful to all stakeholders. If a business analyst sees this, he can say that, okay, if it failed and verify balance, he knows that, okay, this is where it failed. So, he understands that, okay, is this a business logic change or is this a code level change? So, he can say that, you know, the business logic hasn't changed. You go ahead and fix your scripts accordingly. Then internally, the developer can go or the tester can go in and say, hey, this is the interaction that happened. I go in here and then check what is going to, what is happening here, okay? Then he goes on to actually like say, okay, this is where it is. So, I edit it and fix it, fix that particular script, okay? Now, see, this is all part of the tool called Sahipro, okay? But this is not specific to all what we are showing is not specific to it, actually. You could do this with cucumber, you could do it with robot. And any of these frameworks can do something similar to what we are talking about, this separation. This particular thing is a product called Sahipro and this comes, the editor comes with that, okay? Now, let's actually make a small change to our application, okay? I'm changing the application itself. Now, if I were to play this back, this is definitely going to fail, okay? Now, let me just show you the application, what has changed in it. So here, the first screen does not take the password, it just takes the user and then like you say continue and then you put in the password, okay? So there is a slight breakage in this. Now, let me ask you this, with this change, what in our layers should be affected? Do you think the elements that actually are involved in the automation, will they be affected? Yes, because there is one other element which has come in, okay? Second thing is that, which other layer will be affected? Or which other layers will be affected? Do you think it, this has changed? The logic of, so do you think anything has to be done here? No, right? So, why is it not like that? Because what has changed is a small user interface change. It is a flow change, it is not small really, it is a flow change, okay? But it does not change the business itself, okay? Now, if you actually go into that, here is where like we want to fix the particular thing, because here it says like set the value to the text box and value to the password and then submit. Now, this definitely has to change because in between we have a continue button which has come in, right? And the other thing is, you will see if we actually play this back, so now, if the person understood whoever was actually making this, they can very well actually work on this change as the product is being evolved. They can think that, hey, this is going to change, let me actually add the continue button here. This is not very tough to do. Just because you know where this overall thing is going to lie, you can easily pinpoint and do it. And there are not too many places, because you have one function which actually deals with the login of retail user. You do not have to change it in too many places. And if the element has changed, you just go and change the element itself. So, let's actually do that. I think it looks something like this, let's copy this from here, right? Now, if I play this back, okay, it's not confirmed, continue actually, right? So, if I do this, by the way, like you want to see what it looks like when it fails, so the log actually, if your reporting system is nice, it should actually show you like what is going on there. So, you see that, you know, the retail user failed, it actually went in and said, hey, I couldn't do this. It says, submit the confirm button was not available, and here it shows a screenshot because I was working on it showing me the screenshot, but it would have shown you the actual screenshot of the application at that point of time, okay? Now, this all this, like you know, if you are, when you actually build your frameworks, make sure that this whole level of reporting is available, because you never know like who is actually, who can find the real problem. If you are only to throw, let's say, a stack trace, and say that, you know, this is the problem. Now, that needs a lot of analysis as to like why the stack trace happened, what happened before that, and who is actually like really responsible for this whole thing. But if you have a full flow and say, this is the overall logic, this is the business part we wanted to do, then these are the steps that we did. It is much easier to say that, hey, you know, retail login user is failing. Why is it failing? It must be that the new button has been introduced. It's much easier to actually pinpoint there, okay? Now, yes, it is, and as I say again, it is not only this framework. There are a lot of these frameworks which actually do similar things, okay? It automatically comes with that. So, here we go again. Now, again it has failed. Let's look at that. It says verify balance failed, and here it says click accounts it failed in, okay? And it says that div accounts has changed. Now, the thing is, user element changes also cause failure, okay? Now, let's look at what it shows. You see that, you know, there is a div accounts here, there is an accounts here, but it failed to click there, okay? Now, what would you do? You can actually go to that application, and you are seeing that this is the accounts thing, right? So, you actually hover on this, okay? Now, this it shows that, you know, it is actually span accounts here, okay? This element has changed. So, what we are saying is, okay, let's look at what it is identified as in the system. If you go in there, so div accounts is actually sort of underscore div accounts. It's not span accounts, okay? So, this is what has changed. Now, what we are doing is we will, and what, you know, when this kind of a change happens, when the top navigation bar, you use a framework, let's say to actually create a navigation, okay? Now, when you create navigation using a third party tool, and you upgrade to a newer third party, you know, the newer version of that tool, what may happen is they may change how it is actually represented. So, all you care about is something that looks like this, okay? And your functionality is fine, but underlying the person who actually like developed it, you know, hey, styling is easier if I do it as a span rather than a div, so I will change it to span. Now, your automation scripts may start failing here, okay? Now, what you can do is, you just go to that central place where everything is located, okay? So, this is where all these elements are stored, okay? Now, here you can say, okay, this is where the div accounts is, let me actually change it to span, okay? And here again, I can change it here, and everywhere it is actually used, it will get changed. This is, again, a very standard concept, like if you are looking at Selenium tools, you may be looking at page object patterns, where they actually store all the elements together. If you are looking at QTP, they have their object repository, if you are looking at Sahi, it has its own access to the repository. This is not a new concept, this has just been there, only thing is that people forget to use it when they just create those scripts. Now, here we go, like we actually change this, these two changes done. Now, can this be owned by the developer himself? Yes, because he is the one who is actually making changes to the application. So, it is quite possible that if he had a small test, which actually ran through this overall thing and pointed out these errors with various elements, he may be able to fix it and before he even comes to the testers. So, what he says, element changes, one file, which can be handled by either the dev or the tester, flow changes actually comes to either the dev or the tester, depending on how your team is constructed, the actual logic changes, it can come from the business owner over at it, okay. Now, let's go back to the, so this is what we saw, there is something that we talk about in this. This expresses business intent and it is agnostic of the web application itself. In fact, it is agnostic of the application itself. It is agnostic of the testing tool. So, what you saw here, right? It doesn't have any assumption of what tool you are using. Right now, it is Sahi maybe, but if you wanted to actually get rid of Sahi and go to some other tool, look at this test case. It has nothing related to Sahi itself. These are just keywords and they make business sense. This, if you actually like sent it to your entire customer to verify something, you can very well tell you that, you know, this makes sense, it's fine, right? So, it is not tied to the tool itself. It should not be tied to the interaction code also. The third is the most commonly made mistake in automation, okay? People tied on everything in a single thing. They think that Cucumber should be actually doing all the interactions. They think that like even this Excel like user interface should view most frameworks that people build have an Excel like user interface where they say, hey, I can choose from click set value, etc. I can choose elements from here and put the value there. That actually doesn't add any value. That framework is actually useless. That layer is actually interacting with code. Let it have the full power of code. This layer which only talks about business and doesn't care about the code. Let it be devoid of code if possible. So, as I said, examples create user, approve user, log in user. These are all like business things. Well, change of business logic itself changes. By the way, can you tell me in the code that we were writing in this, in this user scenario, okay? Can you tell a business case which will lead to modification of this? Think of a business case. You do banking every day. Can you think of a business case which will ask you, which will make you modify this, generate an OTP, okay? Now, the thing is that is it the transfer money part, okay? Now, generate OTP itself may be just a detail of your transfer money, okay? The thing is it's an internal thing to your transferring of money that the OTPs. There will be other test cases associated with like OTP. So, another thing in automation, right? There are two kinds of functions you will normally be having. One is to test that functionality itself, in which case you will have lots of scenarios for it, okay? And the other is you're traversing this piece of this code because you want to get somewhere else, okay? For example, a login. A login can have a lot of use cases, test cases and assertion. But when you use it as part of a flow, do you actually do all of those? No, you just take that like login as a, as like login, go to the next step. It is just a precondition to actually going to the next step. So, in that case, right? Having the OTP may or may not affect this case, okay? It may affect it if you're testing the transfer process itself with the OTP. But if we are trying to verify that the totals are correct or the balances are correct, you may just encapsulate it inside transfer money itself, okay? So, I agree that that's a very valid case. But the thing is like, if you are in a flow, you may not actually want to make it outside. So everywhere where you do transfer money, do you want to also add a step of verify OTP? No, you may actually have that only when you're testing the OTP part. Otherwise, it just goes into this flow, okay? Any other, any other thing that you may actually come up with? No, that's okay. So, that is just a test. So, you are actually trying to, it does not change the business case. It changes who you are transferring to. Like, it doesn't change the business case really, okay? Right? So, if there is a minimum balance check, okay? Now, will this be part of this or will it be a different test case, right? So, okay, when you think about this, right, whatever you come up with, most probably will need to be added as another test case somewhere, okay? Because if you are going to do OTP, there's another test case to say that, you know, like, hey, transfer money, you actually don't, you break it down into two pieces and then like verify OTP and then go ahead. The other thing is like the one where you said like, you know, minimum balance thing, okay? That is another test case. So, if you say that, there will be transfer money and if it is greater than, like, if it is lesser than the thing, maybe like disable the transaction, right? So, it will be another use case, okay? 25,000 to 24,000 may actually just work, yes please. Right? So, then like it may be, it may reduce the overall amount. That is the, is one of the only cases where this thing changes, okay? So, what he said was if the bank wanted to introduce a transaction charge because of which your overall balance will go down, okay? Only then does this change. The others may be add on test cases for like more of those, okay? Now, sorry? Yes. Yes, yes, they will also be get, so they will get executed. For example, right? This is a bank transfer related test case, okay? So, this will have all bank transfer related thing. This will not have login related checks. So, you won't check here that, you know, hey, if I put the wrong password, it actually like fails or like, whatever it could goes in the wrong password. So, you will have another which actually individually goes to that particular place and does the various operations there. So, each, as you rightly said, it's a journey. There are different stages in the journey and you may need to actually do extensive testing of each part of this journey, okay? But you do not want to do all of this in one go, okay? There are multiple reasons for this, okay? First reason is if you were to do all of that and let's say like you want to eventually, you are facing some error in the fifth stage, okay? When you rerun the script, you are going to do all these validations and reach the fifth stage, which is not useful to you, okay? And then you will be able to reproduce and fix it. So, the better way would be when you get to the stage in the easiest possible way without assuming that everything up to that stage is working correctly, okay? So, if you want to test on the fifth stage, assume that till the fourth stage everything is working correctly. You navigate in the simplest possible way to that place and then you do all the checks at five. When you want to go to six, again assume that five is working. Go in the simplest possible way and then do the six. So, in each one of these, if there were checks in verify balance, you would do it as a verify balance related test case. If you were like doing in the log in, then it will be a log in related test cases, okay? So, if right now it is about transfer money, assume that verify and the log in work correctly, then like verify the transfer related stuff, okay? Yes? No, you don't jump. You don't jump. So, the thing is, so let's say, okay, in this case, if let's say OTP is involved in this. So, here you will go in and say that here where you actually do transfer money, you may enter OTP. You will add it here, okay? This is what will take you to that place quickly. You are not testing OTP right now. You are not testing that, you know, the OTP functionality is not being tested. It is just part of your flow. So, you just add it here and let it flow, okay? There will be other test cases that specifically test the OTP, saying that, you know, if I put the wrong OTP, what happens? If the OTP did not reach on time, what happens? Those will be a separate set of test cases. Sure, okay? You assume happy till you reach that point where you want to be sad, okay? So, you reach that and then you, so you keep going stage by stage and then like put yourself through all the trouble of, you know, testing everything there, okay? So, okay? So, business layer agnostic of these things, I'm running out of time. Again, interactions, this I have already mentioned that, you know, it can be maintained by both QA and testers and developers. Here again, this changes when the element repository layer changes when particular elements are changed via HTML or JavaScript. So, I showed you a demo of like what happens there. So, what should change now when migrating a jQuery application to AngularJS, we see that the business layer doesn't change at all, okay? Most probably some parts of the interaction layer and a good part of the element repository layer will change. Migrating from Angular 1 to Angular 2, similarly, mostly it will have a lot of changes in the, in fact, like not a lot. A little bit in the element repository, a little bit in the interaction. Salesforce, Classic to Lightning, what they have changed is basically the navigation. So, this only affects the middle layer, which is the interaction layer. Applet or flex application to web. This, the business layer doesn't change if your overall functionality is the same. But if your interactions may change, the tool that you actually used to talk to these things may change, the element identification may change. Extending a web application to mobile. Here again, all these, all these layers may change because the business also changes to actually squeeze your overall interaction to a smaller one, right? So, because on a business app, like on a mobile app, you don't do too many things. So, it's a little limited functionality. So, you'll have a subset of these things. So, it may actually affect all three layers. So, some guy on automation code is most useful when there is a lot of change planned in your application. This is actually converse to what people find nowadays. So, if you write it correctly, you can actually safeguard yourself. Its access is safety, night and the guideline. Automation code should not be thrown away when application technology changes. It should again act as a net. Building the right layers and strictly following them helps in minimal maintenance efforts and long-lived useful automation scripts. Thank you.