 Welcome everyone. I like to sleep a lot and I see few people sleeping because They had a very good lunch. I See few people are about to collapse a few more people coming in. I'll start soon in that time I know it's difficult to be attentive after lunch, but I'll find the most Attentive person in the crowd and I'll offer him free credits of our application and you will be you will be Interested because if I tell where I work You will you wouldn't want to miss it. I work at a dating app And I see lot of people suddenly became more interested and more attentive Yeah, I did this one guy who says yeah, I want the credits. Okay one more Thing to wake you up. I'll tell you a very interesting story from my childhood, which will be interesting enough to Put some energy and wake you up so I was born in 80s and as usual any 80s kid I used to play this game a lot How many of you have played this game in their life? Almost 90 percent How many of you have not played this game? Yeah, I see few hands and these people are looking very young Yeah, but as usual with any games this was one of my favorite game and The reason was I was undisputed champion in this game amongst my friends so My friends were never winning against me we used to compete we used to compete For money like two rupees per game and whoever wins get to two rupees During that time and I made a fortune out of it because I was champion I used to I used to defeat everyone so my friends were very scared to play against me and The reason I used to win Was not really because I was very skilled in playing this game It was not because I practiced a lot. I played a lot and I became very Like very professional in this game and I used to win. No, that was not the case The reason why I would win was because I Unlocked a very interesting sec secret of this game And I became champion overnight and That secret was this Up up down down left right left right. Can anyone guess what this is? What cheat code yeah cheetahs everywhere Everyone has used it so I became a cheetah to win the game so This was very interesting key combination as soon as I key in these combinations on this screen I Get 30 lives that was like amazing. I was like It was like a superpower right and 30 lives are enough to win any any level Like kill the dragon and still you remain with plenty of lives and by any chance if you die unfortunately in the game You can key key in these combination again here 30 30 more lives, which is really I it was tremendous power in my hand right and That was really amazing. So these were this was my secret of winning this game Now coming today can anyone guess why did the game developers? Put these key combinations in the game at the first place Anyone like why would they put cheat codes in their game? Does it make the game more popular? So why would they do it? Take a random guess. Yeah Testing testing testing testing all of them are saying testing and you are right They put this cheat code for testing not for people to use it Not for the players right and the reason they put it for testing is let's say you are a tester Testing this game and you want to test level five of this game Would you start with level one play all the way till level five and test the game on level five? And what if you die in the level five you start the whole buddy process again, right? So these key combinations these cheat codes are meant to be used by testers or developer to set some state of your game where you can start testing from right and There are more such example in our industry where we use cheat codes in In our case we call them back doors one of the more popular example is how many of you are Android testers or developers in this room? Lot of you right so if you know you you you go to settings and tap this build number like five six times you unlock a secret settings menu called developer options and Inside the developer options you can see a lot of options one of them is like don't keep activities If you check that tick box you basically are putting a back door in your Android operating system To disable the activity life cycle to alter the act activity life cycle so that the operating system does not store the activities on the stack and That's basically a back door which helps you test your application under stress of memory right So what you are essentially doing is cheating you are calling a cheat code there There are many such examples if you are Android then probably so many apps many app developers put Some some secret debug option in their own apps in the debug flavor of the app To change the API URLs. So if you want to test against the QA environment versus Production or staging you can switch those environments easily using hidden debug menu options created by developers Right. I think most of you have this So this is also kind of a back door what you are doing essentially is you are invoking a back door in your application which calls some Some routine or some method in your application which sets the application state so that you can start testing Right now. That's really good for manual testing What if you are doing automation a p.m. Calabash or any other framework you are using espresso? What what if you are doing automation? How how you can solve this problem? How can you take? How can you utilize these back doors in your automation and that's the agenda of today's talk? And I like I'll be explaining that part how to invoke back door using a p.m. In your automation code and I'll be talking only about Android part. I Will there in my talk? There's nothing about iOS, but I'm happy to take questions After the talk or maybe outside if there is limited time Good So before starting let me introduce myself. I am Rajdeep. I have worked in automation from for nearly more than nine years now and Currently I'm working at a company called Badu in London. We have two offices in London and Moscow and I work at London My responsibility there is to take care of the automation of all of our Application for Android part so we have lots of applications not just but do but we have an area of application But do hot or not blender for different market. We have lots of applications and I take care of Android application For but do specially we we are using a tool called Calabash by the way anyone here have used Calabash or Know about it Know about a lot of people know about it. So we are using Calabash for Badoo and Calabash has this Capability out of the box to call back doors and Recently for one of our app. I started using a p.m. And I hit the roadblocker there because a p.m. Doesn't have anything called back door and that's where I started working on back doors in a p.m. and that was my recent work and That will be that I'll be explaining how I achieved it in later part of my talk so The most the challenges were in terms of automation with this new app which I started them There were two biggest challenges one is I mean everyone probably here knows of these two challenges in automation The first is reliability and second is obviously the speed, right? These are the biggest challenges. So when I talk about Reliability, I'll I'll expense I'll tell some of the complex issues which we faced and Some of the issues which were which were not solvable using any other approach and The solution which we took so talking about the reliability. I'll use the f word here The flaky app so imagine you are testing application and Suddenly a pop-up shows up on the screen. Please give us feedback During the automation now this pop-up is very random It can appear any point in time The logic can be like it should be third session of the user and fifth start of the application It should show up so it can appear anytime in your during your automation and will fail your test block your test And they will fail. How will you solve this problem? Has anyone solved it by doing something? Okay, it's it's something very common problem Which all of us face there can be multiple ways to solve it But what if your automation code could tell your application that hey, I'm testing you currently So please stop showing these pop-ups Another example the problem number two is driver limitations. So most of the application these days have Functionality like if you shake the device something happens for a popular example is in the morning the alarm Rings and you flip the phone to snooze the alarm, right? How will you automate this case? Can APM flip the phone upside down? Not really there is no no way in Android to do it currently how will you test such case? How will you automate this? now imagine No, there's one way some in last time I talked about this This talk there was one guy who said I'll use raspberry pi with a robotic arm clip it and continue my testing anyways, yeah, but What if your test code? Could tell your app code that I am testing you I need to shake the device right now Can you please assume the device has been shaken and move on? What if we could do this another example is take example is of this app. It's a weather app and the business logic here is the weather keeps refreshing every 30 minutes Right, and you need to test that after 30 minutes Weather is changed Right, how will you test this case? Will you wait 30 minutes for the weather to refresh? not really So This is yet another example of impossible Case now what what if your automation code could tell your application code that I'm testing you Temporarily, can you please change this 30-minute schedule? to three seconds That will be lovely right if we could do that if your automation code could tell your app code this and That brings me to the concept of oh, sorry. There's one more problem our test are slow So I talked about stability, and this is about the speed so slow because firstly in our case We have got 1,500 test don't ask me why we have so much, but it's it's a big app So we have got lots of tests, but let's say 1,500 test if I have to test The app I need to log in now the login initially takes 30 second interning username password capture Accepting terms and condition blah blah it takes 30 seconds And then another 30 seconds in dismissing some what's new screens some tool tips and some others some other crap So it's like just one minute one minute to log in So 1,500 minutes just wasted on login which I really don't want to test in all 1,500 tests Which is a pain So what if your app sorry your test code could tell your app that I'm testing you This is the session ID. Just log in and Don't show any any random pop-ups or any what's new or any tool tip just dismiss them just disable them temporarily That would be awesome, and your life will be life will be good. So that is the concept of Backdoor where in your app when your test code could talk to your app code to do something. So imagine you have Android activity let's say login activity here in your Android code base And imagine you could define a method called foo Which you could use to set some application state like seeding some database values or disabling them random pop-ups or or mocking the shake gesture in your in this method and What if you could call this method from your automation code? By simply saying so this is a let's say a Ruby code a Ruby automation code and this snippet in the purple box So here what if I could say backdoor and the name of the method which is foo And what if I do this the method gets invoked magically that would be again very lovely, right? So let's do a short demo. I'll show you how we do it for this purpose. I have a sample app Can anyone can can everyone see it from the back raise your hand if you cannot I'll increase the font size Yeah, perfect. Thank you. So short description of this app there is a text box It's just one text box and then there are some buttons to alter the text box So this button is called make bold if I press this button the fonts will become bold If I press this button it will become italic and I can increase the size of this font For example, if I click this button the size has increased it will keep increasing if I keep pressing the button it will blow away so That's the app and here is the code of the application This is the name of activity which I just showed you it's called create message activity and The methods which gets go get get called on tapping the buttons are these for example On bold this method will make the text bold by setting the typeface to bold Don't worry if you don't understand code if you're not Android developer This is just a short description. What does nothing to nothing really fancy here So here these these four methods if you see are used by our application But there are these two methods these two methods raised toast and this one called message you if you see They are in gray color. The reason why they are in gray color is because they are not being used in our application in our project of our application project and the reason why they exist here is because I'll be using these method From to I'll be calling these methods from my test automation code my apm code, which is in ruby So let's see this first method here called raised toast so this method takes one parameter called string parameter called message and with this parameter it makes a toast message on the application and Let's Call this method from our automation code So I'll explain what is the automation code. It's very simple. I have got I Required all the apm libraries. I Then create a driver and then start the driver and then fall into a debugger. That's pretty much it So let's run this test Ruby It's a demo. It's just a plain script. There's no framework and all I'll just call this Let's start the app. There's nothing fancy here. Everyone probably here has seen a apm an apm test running here so there we go it installs the app and By the way, I have a apm server running in a separate terminal So we are there Now let's call this method which is called raise toast From ruby code. So I have a method called backdoor. I'll show the implementation of this method in a while but just assume that there is this method and I call this method say name of the method using a String string name of the method and this method takes arguments. I pass the arguments as like Bangalore And if you see there's a toast message on screen here. Can everyone see here? I'll do it again Look here. So what I just did was I invoke this method called raise toast defined in my Android app project from the Ruby code now That looks fancy, but not really helpful in automation. So let's see another example of why I want to do it and The reason why I want to do it if you have used apm There are some limit some limitations like you cannot verify whether the text is bold you cannot verify whether the text text is italic or What is the text size has the text size increased or not in ideal world? You would not want to use apm here You would probably write these cases at your unit level or integration layer, but just for the sake of example of Why what if you want to do it and driver doesn't support it? How can you utilize backdoors here? So for this purpose what I've done is I have exposed this text box This text box using a backdoor called message view now This message view does nothing but returns me an instance of that text box an Android instance of that text box Now let's see how it returns so I say backdoor and If I say name This message view it returns me something like an object. It's a string representation of the object It says Android support v7 widget app compact edit text, which is the text view right Now the cool part is I can change the methods on this So I can invoke one more method on the output of the previous method and so on so I can keep calling methods and methods on the object so this app compact edit text is a Android view and there are some methods defined on this view and I can invoke any public method defined on this view So let's see what are the public method defined on this view. So let's refer the official Android docs and if you see this text view This text view class has a method called get text size Can I is the font okay for everyone? So get text size Defined on app compact edit text view so we can call this method in a chain. So let's try this I Say message view and I chain another method with name get text size So that fits in one line and I do this in returns me the text size is 54 Now let's verify if this works. I increase the text size it's big enough now and I call it again and the text size has become one zero six and this is one of the ways to verify the text size has increased a Similar way is to verify the text is bold or not. So I make the text italic and on the same class text view class there is another method called Get typeface and this get typeface object So this is the method called get typeface typeface and the get typeface we have a method called is bold right, so We can call this method by chaining it to the previous method Let's do it. I say get typeface Instead of get text size It gives error because there's nothing no method called typeface. It should be small f right So if you see the return type is an object of typeface That's a object of Android operating and Android OS and on the get type in the typeface object. We called is bold here. So I chain another method called comma Name is bold and says false Now let's make it bold. I click on This button this button I click and let's see it has become bold and The return type is now true. I think So that's basically a small Example of how we can call application methods from our automation code It can be in any language by the way, not just Ruby you can define this kind of method in your preferable client bindings So now let's get back to the slides and let me explain the secret, but first let me explain the ways we use Backdoors the power of backdoors at the do the first is Changing the back-end URLs. So we built only one artifact from our CI and use the same APK for testing against our staging integration and production back-end So we have exposed a backdoor and We call that backdoor before each test give the parameter as the back-end URL and That's how we change the back-end URL in our app we have our application in some somewhere around 40 odd languages and We want to run the test some tests against all the languages and the way we change the language is Using backdoor. We don't change the device language because it is very time-consuming and We need to probably restart the device in some cases after changing the locale So what we do is we have exposed a backdoor which mocks the system language for the application. So I say change locale Are you so it just tells the application imagine your location is Russian your locale is Russian and just start Application in that locale Getting session ID back from app now most of you who have used selenium here use execute JavaScript and Get some session ID parameters back from the dome and then use it for communicating to your server What we do is we do exactly similar thing We get this session ID using backdoor from app by the way backdoors can return you parameters as well and you can So you can tell application to do something or you can get back some data from application so we get session ID back from Application and when then we give it to our server that can you give me the capture for this particular session ID and server gives us the capture value and we test the capture in such a way and Disabling this annoying what's new tooltap stuff at the start of the app for every test We don't we don't call this backdoor if you really want to test the what's new itself More example disabling client side AB test, but do We do a lot of AB testing Almost each feature has three two to three variants of AB test now the variants are decided by server It's easy to call server API and decide the variant But what if the variants are decided by client in a random way and that will be a pain to Test because if it's not a desired way desired variant the test will fail So we for those cases we have exposed backdoor We tell client using backdoor that set this particular variant for this feature and just move on Faking some sim card parameters for example If I want to test payment options in India with a Vodafone sim card I can market using backdoor and last but not least getting analytics data back from app We do a lot of analytics testing and if you want to say if you want to check what data did our app send to? Analytics server we ask via backdoor so the question is how does it work and To know to know how it works. We need to know a bit of architecture of the APM UI automator to driver now this architecture is Not complete by any means, but it has components which are of our interest here so the On the left hand side. We have a APM client, which is which can be any language binding can be Ruby or Yawa We then have a Jason wire protocol like HTTP interface which talks to the Android device, which is on the right hand side and In the device we have three APKs two of them, which are in orange are supplied by APM UI automator to driver and The third one is obviously our app under test in all cases, but do app So let's talk about these two orange apps The first one is called UI automator to server APK. This is the app which have all the handlings of Webdriver endpoints So whenever you say if this is the one which initiates the web driver session responds to click or Get text or stuff like this So this is a HTTP server built in netty inside it now this server cannot start by itself To start the server APM provides one more APK and that's this APK called Android test APK The whole purpose of existence of this APK in the world is to start the second APK and Then there's no purpose of it now To boot this APK what AP what we what APM does is The APM driver issues a ADB shell AM instrument command which starts the test which is Defined inside Android test APK as soon as the test start Basically Because this Android test APK is instrumenting UI automator to server APK it can they can communicate So this APK can get this Context of server APK and it can start the server. So that's how it basically starts the server Android driver issues ADB shell AM instrument test name class name Which starts the test and then test starts the server? Once the server is booted it initiate instantiate the UI automator to driver Which then drives our application under test? So so far so good. This is the as is architecture this is how the APM works and And one of the thing which limits which is probably a bit Limiting is this app called Android test APK as I said the whole existence of this APS app is to start the second app now What what if we could alter this architecture a bit something like this? We merge these two APK in one APK Let's say we call it Android test APK Now if we merge them in one the same APK will have the test and the server both in one So we got rid of one APK now we have a empty slot there we can fill it up by extending our instrumentation boundary With our application under test so we merge both APKs and using this this merged APK We instrument our application under test Now there's a way I'll tell how we can instrument it In later part of slide, but if we can instrument our application under test that means our application and This merged server APK they will both run under the same process of Android And if they run in the same process This merged APK can access the context of our application under test And if it can access the context or our application under test it can invoke methods of our application using a backdoor Now that's fine app APK this merged APK can invoke Methods in our app using backdoor, but how does the client invokes the method client sends the method name to this merged APK Very simple. Here is our client Over STTP it will pass on the method name to Android test APK and This APK then will invoke this method using backdoor inside Now that's the theory part. Let's see how it How we do how we move how we merge these APKs and instrument it So as you don't worry too much about it. We don't understand it but on a high level that's how the folder structure of APM UI automators to server APK looks like it might have changed in the recent master branch, but It's like probably couple of months ago That's how it it looks So if there are two highlighted part two highlighted folders The first one is called Android test server which has just single file as I told only purpose of this file is to start the server That's the first APK second APK is the main folder which has all the packages to handle the web driver and points and now We merge we merge both of them in a single folder. I Took all the packages from the main folder to Android test server folder and what remains in main folder is just one manifest file and Android test server which is on the upper upper half Has everything it has the test instrumented test and the code for the HTTP server both Once we do it We can build our app Using Gradle, but this merged code is already present in this GitHub In my GitHub repo here and this has a special endpoint For backdoor handling so along with other web driver endpoints it also has a backdoor handler which looks something like this so let's say along with WD hub session or Clash element find element find element blah blah blah. There is one more endpoint called backdoor and The code of this endpoint is something like this We first get that activity using a get activity method which has a definition of its own And then from this activity we get the application after we get application we parse the all the operations which we have got from the backdoor endpoints and We apply these operations one by one So here is how we build the application using Gradle the merged APK After we will we build it we get a very Application with a very long name. I don't know why on earth. It's such a big name doesn't fit the slides But it's called UI automator to server debug and write test APK which has all the code But this app has two problems The first problem is it's not instrumenting our application under test right now And the second problem is it doesn't have the same signing certificate as our application under test And we need to solve these problems. So the first problem is it's not instrumenting our app under test and The reason is this is how the instrumentation tag looks like in the manifest file of this app The part to look at is the bottom part which says target packages I o APM UI automator to server We want this not to be this but this Like our app This is the first problem. We need to solve second problem we need to solve is the signing and To solve this problem. We have created a Ruby utility called APM instrumenter. What it does is It resigns your application. It resigns the merged APK which we have generated from APM with the same signing certificate of our application under test and also modifies the instrumentation tag to point to our application test But with this gem, it's very easy to one click command you just say APM instrumenter instrument my APK And that's pretty much it you need to do and will generate a new APK in a separate folder Most of the code for this gem is taken from Calabash Android gem. So thanks to those guys and Once you get the APK you can install it And on the client side, so that's pretty much it from server side you generate APK Instrument it using the gem. You don't have to do a lot of you don't have to do it manually generate APK instrument it install it On the client side you can take advantage of this backdoor endpoint which we inserted on the server All you need to do is post your method name to the server You can use curl and all you need to know is the port which is forwarded by APM Which is usually a constant port unless you are doing some parallel testing and changing it Or you can create a helpful method utility in your client binding client bindings like this the backdoor and It it will post to basically the URL in backdoor. So that's the method which I showed in my demo backdoor That's the that's that's what the implementation is And that's how we call it as we as we saw in the demo backdoor name of the method arguments to the method and We can write this method in any client you can write your own Java method called backdoor which should work Now that's very cool. We solved a big problem. We are happy life is lovely But there's a danger Backdoor is very very powerful feature and with great power comes great responsibility right for example You can write a backdoor to destroy your app and call it from automation code You can write any secret of application and expose it to the whole world Like a you can write a method to crash at 10 times start crash start You can you can basically do anything like you have the ultimate power So One thing we need to take care of is We need to expose back doors, but when we release the app our things like Proguard or off-pacation mapper. We need to cut off the back doors in our release variants So there are special there are various method to do is you can use the pro guard. I think it's Defacto standard these are everyone probably uses the pro guard So you can tell pro guard Annotate those methods which are backdoor methods and you can cut off those method in your release variant of apk So that's one thing you need to do in which is which is usually very easy That's what we do and second thing. We need to take care of it When we expose backdoor we need to be very mindful of what kind of backdoors we need to expose and To explain this I have a quiz for you guys Very quick one. So imagine this example, which I talked previously There's a weather app which refreshes whether every 30 minutes Now you can expose back doors in two ways here The first way is you act add a backdoor method to change the refresh interval and you call the method as Change refresh interval which takes time as parameter which can change your refresh interval So you can start the app call the backdoor change the refresh interval with the time as three seconds and Wait for three seconds and assert that the weather is changed Another way to do this is you can add a backdoor which refreshes the weather So you can start the app refresh weather Assert that the weather is refreshed Right. There are two ways to do it. How many of you will go with the first way and How many of you will go with the second way? Okay, almost similar numbers There's a tie because some people did not raise hands So if those you are using the first way are cheaters, but ethical cheaters And Those you are who are using second way are the real cheaters The reason is if you are using the first way You are telling your application to schedule To change your scheduler to To change the schedule of the refresh interval, which is fine because you are not bypassing the application logic But with the second way you are bypassing the application logic completely. You are not testing what you want to test So that was it and that was my talk. Thank you for listening to me. These are some nice references please give feedback if you have any we have two minutes for question and If there are more questions we can talk outside So the question was if we instrument our Instrument the APK using this APM instrument or gem does it create single APK or two APKs? Answer it it creates two APKs But we need to use only one because it has the code for server and the test both We can just delete the second one and install the only one. There's one more question if There at the back There is a way to do so the question was is this applicable only for Android the answer is no There's a way to do this in iOS also But we have not done it in the APM currently What we have done it is only in Android. There's a way to do this in iOS For iOS we are using Calabash We have a server running inside our app itself to call the back doors Yeah