 Okay for the benefit of time I just want you not to talk just open a very web trolling thing in our days what is testing and I get up to this video how many have watched it loved it. In order to confirm whether the application is working according to the requirements are not and also user friendly or that is known as testing okay testing means using come on here yes using with one idea now with different possible idea all the possible ways we will use the application are you getting my point with all the maximum ideas whatever we get to we may miss one or two ideas also we are human beings no but maximum possible what is that so maximum possible ideas we will be implementing and we will be testing it is it clear no no so testing is any technology is it like some Java.net program or it's not Chapman a technology then what is it it is one process it is one come on friends so testing is a process in which we will we will first use the application and we will try to identify the defects in it and we will try to identify the defects means if anything is not working according to the customer requirement that is known as a defect I already told you so defect means deviation from the customer's requirement so whenever we are using if it is all right so hi all four voices come on yes cool so I was just wondering if you guys are checking your mobiles and checking Google traffic to your home or weather forecast is it I am thinking of skipping this talk yes no no please be honest because I was thinking all right so I just hope in this the biggest problem in this world I feel like traffic is the most problem and I feel that this there should be some scale up and scale down like AWS mechanism of roads and you just can zoom and reach your home fine anyways so let me come back to the top which I'm going to talk about today is hi by the way I didn't introduce myself I'm Pooja from Team Mo Engage let me give a brief detail what our product what it does we are into a customer re a customer re-engaging marketing platform wherein we provide a product to our customers which they can use to re-engage their customers so in simple words we do provide a solution which actually generates notifications on your utility products like snapdill, basket, swaggy or clear trip wherein you or your wife or your girlfriend makes gets addicted to shopping more and exceeding your monthly credit card bills all right so don't curse me for that but this is how this is how our business works so we help our customers get richer and so we do we do become rich more this is like very favorite thing all right so coming back to the topic which I'm going to talk about so initially in my talk at section I said about okay I'll be talking about my journey but I'm changing it slightly a bit I'm not going to talk about me alone so I'm going to talk about you you or maybe from all of you I'm going to bring your universes out while trying the scalability solution the trying test automation framework from the scratch or finding the dead end when you try to scale to the next product needs or scratching your head how to solve them all the pains all the roller coaster rides you might have had earlier or maybe going to have so I'm going to talk few of few hardest of them and then telling you what kind of work arounds work for us while working specifically working for products like e-commerce brands my earlier workplace why we go in the current workplace so we kind of have similar problems but different set different with different sets so having said that how many of you relate while with me while writing automation test solution from the scratch and scaling is a painful task raise your hand very few others geniuses alright so why do you feel so it is a painful task any answer come on I used to think only I'm the one who is shy okay let me tell you so what make you feel that is same thing which works today which you made it work today may not work tomorrow it's you call flakiness and that is why you feel that it's not it's taking too much time of me to automate automate and I'm not able to consider my quality of my product I'm just focusing complete time on scaling my product my product means the automation framework right and that's where you scratch your head and think that oh man who was that man who said that one day that automation will free me forever and I will go to Himalayas after that that man was Simon all right he created Selenium and made this okay so we will make our life easy and get your time to once you automate and use it every time but there are roller coaster rides which everybody I'm sure would have had or will have so I'm going to talk or a few of them right so starting with the ideal world situation so we we kind of heard in agile agile being agile you always focus on your unit as more than API test integration test basically and then you move on to your GI test and then you think about okay at last do manual test of the wherever the more human intelligence is required so I would see the hands up for those who strictly follow this few four five six okay keep your hands up keep your hands raised okay now put your hands down if you do it if you're not doing it for your mobile apps also okay so assume I'm not sure so you have in general nowadays each product has mobile and web form also so hardly we are we have two people clap for them who are doing it same thing for mobile also and web also they deserve clap seriously so what what about others why we are not able to approach it even if I we would we would want to do it why we're not able to approach it any guesses okay so what he says is your mobile apps can help hybrid and native nature plus the responsive web nature also so wherever in your time gets consumed in automating all these flavors and then again checking on different devices so something something like this generally happens in real world for mobile app testing and you end up doing endless iteration and figuring out when my when I'll be able to actually publish my app to the app store sort of so the people who did not raise hand I'm thinking they are they all are doing this passion still is it yes okay you have to you don't have any second opinion now because you didn't raise hands all right so this was a common problem for us also when we started thinking of automating everything and we kind of keep doing this and at the same time we had time problem okay we don't have time when do we automate so the very classic example of acts if you if you don't sharpen your axe axe and you always feel that you don't have time to do it at last you will end up doing the dead trap of keep doing this again and again so it's where we thought okay wheels wheel not only depend on my testing because of the this common problems which everybody knows now varieties of desktop web browsers varieties of m web browser types and they are make a model of different mobile handsets and then you have apps Android iOS they behave differently and the developer developers of those teams are also different they have different designs so a lot of lot of things to take care of and and that too we need to know which tool will reliably do the test not making it flaky where in you again we spend time so these are the common problems and the very very important one the last minute changes so who I'm sorry so who among us they don't hate developers giving last moment changes and feeling like killing them that moment so it's a bing in e-commerce one one very big lesson I learned is last minute changes are inevitable to come no matter whatever processes you set no matter how many hygiene processes you try to bring in they are prone to come because there's a high pace environment there's a business driven environment and you cannot just stop them coming in so what should you do you should have a safety net so my friend annan always says that automation is a safety net so you should put the safety net around it one of them is whatever we product is so it's where we also started thinking of okay last minute changes looks like okay if we have unit tiles and integration test at least we can control it so next layer we we tried doing it we had integration test in place wherein we found it yes we were finding some bugs but then over the period of time we started figuring out this one not sufficient because we kind of ending finding this way so assume that my app my app which is placed in app store from a month now and all of a sudden it breaks there's no release from the app site but backend release has been made it broke the app site also what could be the reason so few of these are the few biggest reasons we found is the API parameters which were passed to the from the backend to the app site were changed and other one the HTML templates being passed the encoding type of them was changed so so you might have seen in apps people because with iOS app specifically it takes a complete month of cycle to get your app published there so you always feel that some control which we directly want to render we will push from the backend so that is what we got into troubling now you might have question if we already had unit testing integration testing in place why would this first problem would happen API parameters get changed anybody encountered this any reason behind it you had your API test you had your integration test in place why it didn't catch catch that who said yes they have to answer okay so let me tell you in my experience what used to happen is the person who had written or changed back in API is the same person who had written the test so his expected result his actual result was according to him or her getting my point so one check here we could put as we could again put the review cycle of the test or whoever developed the test but having said that last minute change which I was talking earlier last minute changes do not give you privilege to do that so that is where you need to have safety net again and the second point estimate changes also the same same problem the including changes and your app starts rendering wrong HTML or HTML content itself you might have observed in some some apps m percent nbsp m percent amp slashes etc so why why it could be happening it was not happening when you released it and you just bang your hand why why it's happening it's because of some of the backend release we pushed the data in wrong format or wrong encoding so these are the common problems we also had and then we thought we this is the high time we need safety nets for web as completely and mobile as much as possible so that's where we thought we shall not pass in any way whatever comes in so for web surname was the obvious choice I I always love selenium and for mobile app there were they were quite quite a few choices when we started thinking of using it using them calabash was one among the one which was very stable in that nature but for our product needs so our product needs we always wanted our existing framework to scale so we already had a framework which used to do web automation get the logs published reports published screenshots everything in place by that by this time and we wanted to scale the same we do not want to have a separate framework altogether and second thing for sure we did not want to change the aot the application under test so that is where it made us choose apm so whenever you you guys choose your own tool think across what is what according your according to your existing framework you are having according to your existing resources if people are good with ruby or cucumber kind of struck style go for calabash then so you should choose accordingly so this is how we ended up choosing apm but having said that how many of you are using apm okay cool how many of you you raise your hands how many of you used to use it before two years when it started one or two yeah so remember errors like simulator crash dead boot up apm crash dead boot up and if it is it opens it again crashes or if it is successfully running in front of you and you go to go to your manager or cto and show look i made it working and it breaks again in front of him yeah happen yes so similar things it happened to us also and that's where i learned a very fundamental basics about how to work how to get work with open source tool so i'm not going to take left okay so so we also had similar biggest bigger problems like this i'm not noting all of them if you want to know all of them just go to apm slash apm issues filter by puja you will get to know all of them so i kind of started making them the apm community the great apm community piss off with all my queries all my questions there you can find there but then the evolution happened the evolution happened in a way the same problem which i was facing somehow in this struggle with my trials with the community great great people great supportive people they come up with some work around which is to work for my period up and similarly other people used to ask the question so so that way it was a health engine so that's why i said that be part of this journey so i kind of having solutions for multiple problems which i just keep getting in so one of them was apm server inspector crash or stuck so this is not specific let's not make it specific to apm it can be any tool which you try and when you start doing it from you are the one who starts it first kind of these these things are keep on happening so how do you tackle i want any one example from anyone how do you tackle tackle some such situation what is the debugging way approach you do i got inside that it's like debugging person so i also feel the same any any example mm-hmm so same thing uh when we we want to know what is going wrong we need logs right so take care so it's a one mantra first mantra figure out where the utility which you are using where it stores the log first thing second thing where are you storing your log so do have logging mechanism avoid print statements do have logging mechanism with proper log levels debug error so that you can figure out your issues quickly so these are the first one draw we learned that moment and we figured out okay now next it crashes okay it crashes no matter what what do you do next you got to know the reason reasons are uh several so what do you do that next so that is where a very important lesson which could be utilized in across your projects is know your processes the last point uh i didn't write in this page maybe know your processes so there can be several reasons behind your main setup to be crashing is maybe the other process which is interpreting or killing the process actual process figure out that some log if you figure out that find out the hook around how to restart back or how to kill the other process which is about to kill your process so that is there is two things uh very important aspects of working around such tools which which works in the virtual environment kind of stuff then choose the right locating strategy audience talked about a good strategy today and i'm not going to give more detail on it but for from the mobile context i'll give you what would what works for specifically for apm this one so so you get to know you get to know that one uh locator let's say x path you're trying for your web project it works fantastically you're trying for your mobile it it also has the same let's say same exact same locator strategy and it doesn't work then you realize okay put the lock why it's not working it throws null pointer it throws stale what kind of error it throws based on that you again make a decision right but still if your element is found the the worst case your element is found still it is not clickable what could be the reason possible okay cool so we got two points one is it's not either visible or it's not the same page also it can be a thought problem with when we were especially huh yes web we also is one one thing another thing i didn't get overlapped yeah good point i was about to come uh on this last point overlapping will come back but the main reason with specifically if you talk about ios with x path the the page source finds the item easily but the ui automation which you interact with you use any tool it internally finally talks to ui automation to uh simulate the actions on ios device or simulator anything which it's it it has to know the same passing level also and which is not good handled in for x path in ios so that is where you figure out okay this is not the right thing just just search a simple search will give you uh good examples like uh friend matthew purple noises bootstrap he has written a very good uh things on this before me trying and juggling around it so he written he has written for ruby basically so which shows that okay uh for ios specifically x path do not choose choose other strategies choose what ui automation supports one thing second thing is also know that okay uh the version compatibility thing right some things may not work in your particular version of uh the tool which you are trying and it is the issue is none already and it has been sought in next version so try thinking of upgrading that so that is also one thing you can do here third problem okay so here is a bug in my this third is missing so uh time consuming to test the test uh how many of you relate with it when you tried actually mobile automation specifically only one no others did not do or still shy to raise hands okay so uh why did you feel so yes so uh thanks so what he means is when you when you try mobile automation it has to go it has to talk via proxy and proxy and then reach to your simulator or your device so there are too many connections in between so which which takes time for server to boot up first and then your simulator to boot up first and then your test reaches there and that is too that is also a little slow in case of android it's fine in case of ios it is slow it reaches that point and then you go step by step to reach your let's say you have four pages in your application fourth page you're trying one located to test four transitions it will take time right so uh my approach any approach you took to uh reduce this time taken no no so what uh simple approach came in my mind is put the debug skip the manual steps in between all do it manually quickly reach that last location and then try your locator here a problem used to happen is which initially uh i'm not sure if it still happens is the inspector gets flaky when you reach the last location and you want to ensure that your locator is actually gonna work fine so you open the inspector it doesn't work it doesn't get you the tree xml so what uh next solution we did is uh when we know that this is the this is what is not working and we i want to try i'll put my one more statement before get the tree xml so even if it does it fails there i have my xml i can pass it in xml viewer somewhere xpath checker and i can check that or if it is other identifiers i can check them easily so it it started saving our time so initially i also had to feel i had i used to feel that i remember one day my manager comes and says that it becomes very time consuming and we are a business challenging company we cannot afford this much of our indie time let's move to some other good tool which provides on-the-fly solution like audience talk about test i am so similarly we also had took demo from them and i was like i i'm into this product and i'm constantly constantly talking to dan and emergency and everyone and i'm figuring out that this solution these problems are still going to take time to solve because everything is not in hand of apm it has to do with ui automation also so there are many things mainly layers to it i'm just wondering how they are able to solve it so i just created my list uh this list is just five points that list was very big and i'm sure they would have hated me afterwards so i i kind of uh wanted to do more on the gestures part where gestures were not uh very good supported by the tool because it is little slight difficult uh specifically e-commerce project let's say you you talk about search page search page has thousands of data or maybe billions of data to be populated and it's where your apm server gets choked while parsing when you try try to swipe down or swipe up and you you feel that i don't know what to how to solve this because it is busy in parsing those logs it is busy in keep tracking up tree xml uh to find your locator right so similar problem happened to us so i wanted this solution to happen uh to see if they are doing it magically so uh of course the the answer was no and they would have hated me but being polite and good people they just said that okay our iOS devices are busy with other clients let's let's talk next month so by the time i realized one uh unpack that uh either this is not doable or i should we should hold this project for some days and work on other qa uh practices around to solve this problem now and come back when it gets stable and i was little dream i do not know be happy or being demobilized but i was actually demobilized to see that and then uh one single thing uh which made me change my complete thought process i talked to my city and said that either which of this automation time because it's just time consuming for me or or or we do uh or we do go first or do just give to third party they let them do everything so i was like that so then then my city was says one single line which which made my complete change my complete perspective of thinking and working around such such tools is that he said that okay let's let's chuck it off this problem into sets let's target 10 solution every month you you're talking about six months it will take to be to be able to stabilize the apm tool six months you also take and by the time we reach the journey we we both are in the right phase to have completely automated system so this is what uh his line was which made me put in my mind which made me think one one uh very good thing indirectly is i just told myself if nobody does or if the moment i feel if nobody does or nobody can i should be the one who can so that is what my change my perspective i started loving its pain point i started being the part of journey is the not necessarily by fixing bugs but starting uh polling on who are uh uh asking the same questions or something and then i realized okay uh by doing that i got to know i digged into um apm's repository and figured out simulating gestures was possible if i would have taken the native approach any idea what could be that native approach okay okay what okay now the question i am asking you will be knowing the answer probably while using cinema or web driver if something doesn't work even if you know that locator is fine something doesn't work with web driver what do you do yes so that came to my mind if there is java script for web driver there should be something similar for the mobile also and that is where uh with the help of their teams figured out uh okay for android adb is the one awesome for ios ui automation we already knew but the commands and all were not in field and then that's where we thought okay if we somehow able to inject this in our code with precise calculation from from where to where i want to swipe off precise calculation my problem is solved who cares if i'm not doing it product by two product to another product if i'm knowing this is my size of this is my size of simulator and in this within this size i'm going to test my test i want to it means i want to scroll 10 products down and i have calculated it my mathematically for reaching 10th element this is what is required push the adb command push the ui automation command or ui ui automation one good thing happens is you can use the name name strategy so if you know your search results primarily in that case you can use it but in our case it was not the helpful thing because dynamic project injection was there so we cannot based on the other product but we use similar strategy to find out other name let's say let's take a typical example of app of like a swiggy and you have food items so let's say first item is uh paneer and the last item is dal makhani and you want to reach from paneer to dal makhani so one way is either you know 100% the 10th item is dal makhani then you can relate on it but it is not the true case dal makhani is gone let's say there's something else you cannot depend on that so what what do you do name so you could also utilize other other name strategy also dal makhani is fine but there might be other properties to it which are which are going to be specific that you have to you have to find out from your product understanding which i or nobody can tell so that is where many problems you don't find solutions directly on internet and that is where you find that that is where the second diagram we ended up end up doing manual and manual environment so how you can do it this is how my story is our story is and we also started with that and by this time we kind of head uh in a automation automatic test okay one point i'm missing here uh is this yeah so uh coming back to your point overlapping somebody said overlapping right so uh one component uh the the locating strategy you feel that your component is found but still it is not clickable we kind of had this problem big time and i was just wondering same thing i if i do manually or same thing if i run in debug it's it works for fantastically fine but what if i execute it uh in a parallel uh in a parameterization test data driven testing for second test it fails third test is again works for this also it works fifth test again fails so it was like random nature so i was not able to figure out what what would be going wrong this is where uh watching your tree helped me so watching the tree shown me exactly i was let's say um so there was a pop-up basically give me a little back down there was a pop-up on the screen home screen which says yes or no button so you have to click on either yes or no i wanted to click on no i was i was expecting it it should click on no but it was never clicking it sometimes it was sometimes it was not by viewing the exact XML i got to know they were overlapping two pop-ups you see so right now it is very neat to watch otherwise it will be victory XML where you cannot find it so how how uh how it happened is based on the hints i got by rerunning at least twice so hints came something looks like that so the major hint we uh we got from this is we should have for our specifically for overlapping count contents like pop-ups overview overlays for such components we should have different kind of handling just find element would not solve that why because find element finds the multiple elements and the first one is given the priority and you apply click which is not visible so it it is going to fail correct right so that is where we kind of had another uh utility function uh wrapped around find with visibility whichever is visible click otherwise iterate to uh iterate through all the elements found so this this was one big trick work and trust me now when i say give you giving you a problem in our solution it looks very easy but when you get into trouble you don't know what is going wrong so that's what happened after a lot of understanding proper logging proper screenshot we got to know what was wrong okay so uh coming to the last point uh this is also one which you can directly not find uh by telling you telling you a problem to someone until you get you're into problem so uh i'll give you a typical case so at guy before we had uh in ios we had a search flight uh option auto suggest which used to search flight which used to give you flights or across the globe across the globe so whenever i'm trying to do it manually it works but when i'm trying to automate inspector crashes first thing i cannot identify with light problem second thing is uh even if you somehow able to uh get via your txml finding your locator and you try to uh put the same in running the code it is still doesn't work so any idea people have got into such problems no no okay okay but uh i should clap for everyone then okay so we got into such problem wherein it doesn't work and by trying and realizing uh we seeing the apm logs we got to know that apm has got stuck too busy parsing all the flight details it is it was trying to uh build those nodes for all the flights so if there are millions of data in your skelite db it is trying to pass that it doesn't have it doesn't it cannot do parallel execution until it finishes everything and then come back to give you the txml so that is that was a huge problem for us and that's where a simple very simple trick helped just tell apm log to shut off say debug level sorry log level is equals to l and that's your life but sometimes we wanted debug log all to c so then took the developer's help we wanted we needed we know we knew that uh what is the test data for us so we knew that we are testing against it's a 50 data only so we created a separate separate app uh with separate uh 50 items db only so that also worked so uh yeah but to know to solution you need to get into the problem so that's the uh idea that's the concept is not about apm or any tool it's about uh the same concept or same approach or same set of problems you will be getting whatever library you use second thing is our duplicating uh thing we have already talked so i'll just move on okay so once we were done with all these activities the the main thing came to us was the uh transparency layer creating transparency layer otherwise what would happen then night's one o'clock call hey puja uh i just want to run this test and give me the feedback fast second day also it happens same and if the test fails they need to again on call only they want to know why it failed what is wrong tell me and you just feel like i have created automation oh man i have did such a brainy job nobody is recognizing that they are just thinking that i am a i'm a feedback engine all right so so so it is very very important for you to create the transparency layer so that anybody can access it access your locks access your screenshots access your test reports properly so this was in our brain and uh there are good tools but i started the moment i start use genkins i started loving it so i did not think about any other tool so uh but uh the everywhere this is a bug so using any ci tool i am assuming nowadays everybody is using it yeah genkins yeah okay so do you think uh uh once you set up your test there it's always a calm calm water lazy river you can grow always or do you feel and there is a tsunami is about to come or you feel tsunami is already come i don't know what to do release is there i need to fix it fast oh no no no run on locally and tell the feedback first happened yes so similar thing happened to us also and we want to stabilize this so we were saying that flaky test on remote machine because genkins we hosted on remote machine if if the same thing i'd work on local genkins like local genkins it works perfectly so there were a few big reasons i would tell you one of the biggest reason when we took web uh hosting web test hosting we figured out the main reason was uh the remote machine is a headless machine and you are thinking that it is it is it is to automatically invoke the prior box or chrome or any the browser and do it it cannot do because it's a headless machine so that's where the awesome tool xvb uh uh who are created thanks to them so uh awesome tool xvb a simple setting makes your life easy it removes those errors window can cannot be invoked or one more trick here actually uh it took me days to figure out this resolution setting so there are uh even if you have invoked your x you have set the xvb and it streams you it actually uh does all the graphical actions without invoking without invoking prior box and does everything and gives you a screenshot also there are chances when you you might have noticed element is not clickable at this point and that too in genkins only it works fantastically on your local the reason is it it invokes its own uh resolution you need to try for your product your window size you have to try for your product and come up with the right come up with the right resolution setting so this is one which uh works for almost all products this i tried and with trial and error another thing you can do control with your code also the dimensions class whichever language you use you can control it with that also so this requires a trial and error approach so uh this is a life saving trick though okay so uh third thing i was talking about earlier they know your processes okay how to debug we talked about locks you should have locks you should have screenshot you should know the processes also how many of you don't think think that it is devops job why should i know about what is running in my system okay people are smiling okay so i would not embarrass more so uh yes we we must know our processes what processes are running so that we can know uh how to restart or which one to restart and what time to restart back example take let's take the classic example uh your simulator crashes up so i have kept uh my simulator simulator um i don't know how to see now okay so if it crashes i need to know at very moment or maybe in 30 seconds or 40 seconds that it is crashed so that i can work it back so whenever test is running it does not hamper my actual testing and make it flaky test just because it was not involved how do you know it's very simple way actually if you look at this it's a simple way you just need to learn few Linux commands ps grep ox etc etc to figure out what processes are running and find the pattern when your test of like not running basically so for our case it used to happen whenever uh just before release uh we used to run all of them in parallel that time the randomness used to happen so we found out that pattern and kept the logs even in the processes what were the process were responding or killing others so we figured out the fine balance to restart if nothing works reboot machine works right same way so you either you either you find the culprit or you find the solution so that's the two approaches so i go for both depend on the need so there is one beautiful tool monit to help you doing it easily you just need to take you just need to tell it uh uh which one to which process to invoke so you can write shell script to start or stop and you can put in an inside demonic it takes care of it automatically uh when to start and when to stop all right so uh by this moment we were done with uh many tests now for web mobile uh mobile uh different flavors uh of mweb and uh kind of native apps hybrid apps so we we kind of had native uh cum hybrid but most of the times what artists were about native testing so here one more daunting need came is i wanted to see we wanted a single point of contact or single point where we could watch all of our test feedback easily why the reason if you have to take let's say you are you are the reason you have to take the call of go no go how would you take if you have to watch 50 tests all together it will take time uh you will see green bulb or red bulb you will be determining okay these are the failing but you have to again go inside and see what is uh what is the release version which is it is failing on right so similar problems we also had here and uh we really wanted some solution to solve this and on top of that one more problem uh which is like if by any chance so this this if happens openly if by any chance your court after all these 50 nades breaks on production what do you do so this is a simple uh clip to showcase that when you try to fix production bugs right so you you you think that like you as a team with developer think that we would be doing this then it will be solving one bug solved second bug solved so similar similar main points happen with us also and we we kind of solving one bug and introduce another bug on production bad practice right so that is where we also wanted to do something uh which used to which can tell us okay what do we do if some bug comes in the production one base keep solving there no another way is revert back immediately because our production is customer centric it's a consumer centric a one second of mistake can lead the wrong transactions and can lead uh uh malena image complete work is gone right so we wanted immediately revert back to the sorry revert back to the previous versions which were stable so how do we know in one shots one of time otherwise there should be one person who before release who goes gives no he needs a two hours of time to watch all the tests to vouch uh to to figure out the reasons and then create a report in excel or mail or somewhere and tell that this is why they should not go right and that too we have a dependency on that person so we when we talk about automation it's not about only test automation it's about automating the complete process uh which is what what we will make you someday go to himalayas okay so yeah so this is what we wanted i'm sorry so i'll just i'll just not uh skip the slides and i'll take you directly into the demo what we did to solve that so i have a set of the local Jenkins give you a little brief and local Jenkins and i have created a test jobs around a particular application calculator basic application i took again uh credits to uh the people who developed it so android ios win uh and web applications i took from the of course those community and i uh kind of written test around it so you see five tests so similarly we had almost 50 tests uh uh in real time there so it's seeing all of them the big list you have to scroll down and keep seeing and and keep going inside and keep coming back to see what made it wrong uh was a difficult thing so we came up with some simple solution like this you see quickly here i have opened it here does it look better than that it gives you complete clarity of all the jobs which you have run it gives you the version number on which you had your course core is run it gives you the environment detail where it has run so that it's not that uh okay uh you you report some bug it might have happened right you you report some bug and then developers keep uh comes back this is a wrong bug my code is not in that machine you're trained you you're not testing my code at all right so so this information was very necessary for us and we wanted to know the responsibles also uh because of whose code it is broke so this you can take care right now i have named no mention myself because i was the only one local Jenkins otherwise we can control this how to feed the data in the inside it so whoever who else builds commit commits are there that comes up commit commit authors we can pick it and put it here so kind of and it gives you direct links detail reports screenshots console everything so let's let's see uh so so green shows the past test so both are past these are failed so let me run you run you through how these these are done so the idea is to tell you you can do similar for you your work also so i created a simple deployment job this is a mock job this is not actually doing any deployment but this can be a simple script to deploy the code on some particular machine so let me see how to build it so it takes two parameters one is your test environment where you want to deploy the code and what is the code version you want to so i'm just saying rebuilding the last so that i do not write too much okay i'm smart okay so so i wanted to do i wanted to do i want to have a code job basically the deployment job which deploys and in at the end at the end of deployment it triggers your test job that's what ci makes makes it happen right country's integration means you deploy some code it in turn creates some it starts some test and reports back and if it is fine then it merges to the next pipeline so similar thing we want you to try and we did it so let me just roughly say that it is 2.6 i'm doing earlier we had 2.5 right so i'm saying 2.5 2.6 the build version i say rebuild so this is supposed to invoke my other test so let's go to the mainstream and you see all the four tests are invoked and they are running this might take a little while because all right so let's let we can see the view here actually so the moment we invoke that you see the progress bar tests are running now it becomes the transparency layer you can actually make yourself feel full and put it on your some dashboard on some monitor somewhere anybody can monitor these tests are running it means these are these releases on okay so things happen okay so this was intentionally done i wanted to show you three examples one is flaky test second is your intentional test failure or genuine test failure you can say in this case and the other one is a fast test so i invoked i made flaky test for in a firefox okay so you can see the tests are running okay i just minimize your size okay so the tests are running on ios and right you can see the speed difference because of the architecture problems i told you in before so the tests are running and keep putting the uh so there is a three thirty second time now the refresh time mode you can change it and it keeps feeding the data in so my android test is done it is passed again it shows that 2.6 i had run on this machine and last stable is also the same in this case it says that 2.6 is the one last it was working on 2.4 not on 2.5 also probably right so it gives the better clarity to the person who is giving bono go or even the devops or even you sit there can look and figure out okay these are the these are the release versions they can either revert back quickly right and you can go into detail also so let's go to the flaky test first uh firefox right so i can go to detail reports so here i have linked all my reports linked very clear reports it shows that okay this directly shows something configuration failed test has not run you can say ignore test all tests are ignored you can see the exact errors basically clicking on it shows you exact error and you know that firefox has not been launched properly clearly right so it makes for anybody it do not do you don't need to force anybody to use uh your tool you and you motivate them automatically to look at themselves so kind of thing so let's have a look at the other test which we intentionally failed so there was uh there was a uh chrome i had a last test probably uh seven divided by two it should turn out to 3.5 but side gives me three right so that's it's a it's a failure right so it gives me right failure reason exactly i can watch other tests also these are the past need clear anybody can understand right and whatever the failure reason also they can see and are logging also this is this is useful for us when something goes wrong without your understanding you you have logs in place you can watch them properly right so this is not one thing which is there there are multiple things which are connected and everything is open source so i did not use anything which is property here or and will kill me for that okay so uh this is how uh this is how i came up with solution now anybody have any query if they want to do it similar thing uh i can i'm happy to answer for that before i go to the next slide yeah screenshots awesome so we do i do i did have a linked screenshot also you can just go in screenshot it shows you three there was three just was like this was wrong 3.5 was supposed to come right so if there were multiple failures it will direct links will take you all the failures uh screenshots here so how how it is there is a uh there is a very awesome plugin image library which we we added so we keep put pulling from our existing screenshots uh path to this path the job job path basically and linked it in the plugin how how i how we did is uh there is a except plugin it's extreme feedback plugin i just tell you take you to uh the library from slide how we started the journey of this so this is the actual plugin because this this was the actual plugin which used to give bare minimal bare minimal things required for our product was only the feedback pass or fail this used to give this this screenshot so when we are when we are trying to get all the information there was no solution for that and then i came up with the bare minimum things which was i want to get the jobs detail pass or fail at least so this was this was to give only this so we made code around it and added whatever was needed to do a one one place solution for it so which is what you see this looks this doesn't look very neat i took help of java javascript and made it with the relevant permissions required so so this is very important for you to uh there is one word called sell your bugs so this is one way to sell your bugs you you have automated you have put your efforts on publish them so that they become specific how you can do is is it it's easy if you open your brain for the open source communities right so that's what is in mind so i'm done with this you have any query towards for it also you can write me down also i would love to help uh the last but not least is one point which everybody focuses on test code to be production quality how many of you do it okay how do you ensure that that is our production quality okay okay so these are the process exercises uh uh uh how how from the uh tech exercises he said that uh from the review uh we should do just like developers we should do code review also we should uh evolve that culture so that uh if other finds if you find bugs in others code it's like makes it uh more useful for future uh but also from the tech's perspective also how how while writing itself before it goes to uh review so how do you say that standardization okay cool nice um so uh uh you take care of uh this by uh following coding standards wherein i follow uh another approach sorry disruptive approach where some sometimes i break those coding standard rules also if they are not fitting into my product needs my automation needs or my scalable needs so if somebody forcibly says that use the design pattern because it is good for their product i would not do i would try again even if it is written in google metrics what's this this design pattern versus this design pattern even if it is written there i would not directly jump to those conclusions i would try for my product what is the best pattern would work for us for now and for the scaling perspective also that is where we would we were able to scale our uh test code from uh web automation to mobile automation uh uh with ease and that's where we thought of next plan to publish them all right so these these are the two uh points uh second point is very interesting actually i still see people using it so i just want to know how many of you use static methods static variables uh very often no oh damn good yes yes yeah so if you talk about from the great knowledge in great world we get to know that but even before that if you know the coding principles properly you knew it already so i had seen one i kind of get uh sometimes offline request to check their code so i got into such problem where i couldn't i did not see what kind of variable uh uh she has defined uh on top i just saw uh the the project she was running it was not expect it was not giving the exact results what she was supposed to so uh i tried to debug started debugging and my mistake i did not see in the first place what kind of variable she was using so what what what her exercise was to uh invoke test in parallel so she was not using great it was a test engine test uh it does the same thing very easily uh so she was doing that and she was the moment the firefox launches and chrome launches whoever gets the last one takes the control of the first one it so i was like how come it is happening and then suddenly i see oh oh so so she has defined her web driver object is a static object what do you suppose what do you suppose static uh the definition of static is it means that it is supposed to be shared among all the objects so no matter how many parallel threads you run all threads are going to use the same so whatever's the last state it will use that so uh for the people who yeah without intention start using static i would say no say no but yeah so whenever you say static unintentionally you use a static you end up finding into such troubles for which you do not get the solution you do not understand why it's happening so do not do it uh proper logging proper design pattern we already talked about take screenshots as much as possible because taking screenshots helped us that pop-up solution which i was talking about uh figuring out yes it was there but it was the code was not able to find before click and then when you automate everything think about publishing it all right that's it so any i'm open for questions i don't know on time uh which we talked about either publishing uh the plain passion okay i'll take you directly there either plain passion over here any ci tool or or think of uh advanced level which uh this plugin by the way you can i hope i have uh contributed the repository back so you can find it uh either there or on my page also how to use it pdp00j4 you can find it with description how can you utilize or where where all you can make code changes to uh help you make it for yourself so you can fork on my repo and uh start contributing yes that's it's not a framework so whatever exercise i did pardon uh so his question is uh uh so the first question or second question the first question was uh about the publishing the results so i went up talking on uh you either you public in some ci can see i server or you publish in some more effective way like this which i talked about one in short uh quick links to everything second question he has is not question basically uh if i have uploaded the uh framework or demo framework a sample framework so yes i did have so whatever test i had shown here running uh these tests i have published under my uh report self actually you can find ujah puja um i'm sorry github account you can go and find out it's there with scalable framework demo under conference 16 so you can find it there yeah pdp00 i did not write so it's puja p00j4 all right any more fairies because if that happened you can switch to mitra worries want to use a static like you struggled in terms of finding the static method right you'll pay some because of uh they're not qualified for instance so how much i understood the question because of the interruptions so your question is either follow the coding standards or uh when you're said disrupt no i notice some uh contradict between your conversation usually you say like in such condition you'll be bypass using the coding standard okay you will you'll not follow the coding standard in some no i did not say bypass coding in some condition right so and then you i said no i did not say bypass coding standard i said do not jump to any design pattern just because somebody said okay and you choose at your own product and or own existing resources you available and next we say like someone has been mistakenly used a static method right so they might have thinking this way and they never follow the coding standard and they use the static so coding standard and choosing the design pattern are two different aspects not only design pattern everything to show the coding standard true true you you need to follow coding standards but you don't have to uh just follow one particular design pattern just because somebody said so that is what i hope i'm making clear or we can take it offline again all right thank you so with last line uh repeating uh my cto's word uh with you is whenever you feel that this is not something i can do or this is not something anybody does always repeat yourself because nobody is doing be the one who can all right thank you thanks for having me