 Hello everyone, good afternoon. Welcome to yet another awesome session that is coming up on UI Automata 2 and Espresso drivers by Dimpy and Rashmi. Really looking forward to this session, I hope everyone enjoys it as well. So without further ado, Rashmi and Dimpy, I hand over to you. Hi, good afternoon everyone. Thanks for joining the session. This is Rashmi. I have around 30 plus years of experience in this industry. Primarily, I work as a desktop architect in Hopkins Science. My areas of expertise are on the big data front. Having said that, I feel proud to be part of testing fraternity. Hey everyone, I'm Dimpy. I'm having around 16 years of experience in testing and I worked in Hopkins Science Technologies. Rashmi is my colleague. So looking forward for this session. Okay. So today's session is about how UI Automata and Espresso drivers helps all the automation issues and serves as a perfect viewer for MQTT and IoT based applications. Now let's see the agenda. First, we'll go through what are the key concepts of MQTT and what is our use case? How these two drivers helped us solve the automation issues that we encountered. A quick demo with the different apps that we have. Finally, we will be doing a code work through as well. Okay. Now let's get started with MQTT key concepts. Well, MQTT is the main one of the main protocol that is used in IoT based applications. In Internet of Things or the IoT world, everything is connected. Be it your smart home or your smart parking lots, your temperature or thermostat sensors, your devices, your connected cars, anything which emits data. They are called as publishers. Ideally, in this IoT world, you have a few basic concepts like publishers, traffic subscriber and MQTT broker. Now the one which transmits the data are called the publishers. You should have someone who is ready to consume this data and make sense of this data, analyze, build dashboards, etc. for your analytical purposes. The ones that consume this data are called the subscribers. You can see publishers are here, subscribers are here. They are totally decoupled. Now it is the broker which enables this transmission of data from the publishers to the subscribers. So the MQTT broker that we are using is High MQ, which is a public broker. That is the centralized broker which acts as an intermediate between the publisher and the subscriber. And one of the key concepts we have to understand is the topics. Topic is a simple string, but whatever data is published onto a particular topic, only that data will be sent to the subscriber who has registered or subscribed to this particular topic. And who does that? That is done by the MQTT broker. Now in this slide you can see we have three different topics, humane motion sensor, moisture. So now when data flows from the first sensor into the humane, only the very first subscriber will be able to consume that data. The one that is available in motion sensor will not be sent to the subscriber who has subscribed to humane. So this is very important in understanding how topics play an important role. So in a nutshell, publisher publishes data onto a particular topic. Subscriber who has subscribed to a particular topic receives that messages via the MQTT broker. IoT itself is a very vast concept just to get the context what our use case is. In a nutshell, this is how the MQTT workflow is all about. Now let's get into our actual use case. So what is our use case? We have kept it very minimal and simple. Now we have considered it as a smart home where we have smart devices, LED switches and all. Now in our use case what we are trying to do is we have a switch on and switch off evens that gets generated. So we call this on the left hand side as a publisher client. So this app where you can see switch on, when a user performs a switch on, it goes to the high end queue broker and the broker has two subscribers. You can see on the right hand side, one is a dashboard app, another one is your alert app. So these two are subscribed to a topic called Hall Switch. And when an action is performed on any of these topics, the payload or the data that it transmits will be sent to the corresponding subscribed clients via the high end queue. Now in this diagram you can see the switch on is enabled. So you can see the Hall Switch is turned on and the LED is glowing. Coming to the alerts part, since it's a demo we have kept it minimal, what is our alert condition? Simple, when a switch is turned on, we want to send an alert. In a realistic world, consider a thermostat sensors wherein you have kept a threshold like 100 or 200 degrees. If it breaches your threshold, you want an alert and modification and perform an action according to the alert that you have received. So in such use case, these alerts and notifications play a key role. So we wanted to automate this entire workflow and we found few of the challenges when we chose a single driver. Coming to the challenges that we encountered. Now you saw we have three different apps. One is a dashboard app where actually LED glows. One is a publisher which sends the data or simulates the sensor data or the payload and the notifier which actually generates the notification alert. So we are using multiple apps. Switching between apps is possible using the UI Automator tool. The other challenges like we have to validate the style attributes, the bulb, the switch colors and all that. That is not possible using the UI Automator. So we have to go for Espresso notifications. Now we talked about the notifications that we are going to receive. So in that notification bar, you want to go click that notification. All that is possible in the UI Automator. So individually, if you go and see, these are the few use cases where we are considering UI Automator and few use cases where we are considering Espresso. And together, we are making it as a complete end-to-end scenario and using both the drivers, we have achieved automating the complete workflow. So the first one what I talked about is the notifications. This is on the app type called release. So we are using the release app unlike the debugger. And the driver that we used is UI Automator tool. And UI Automator has a feature to perform a click, pass through the notifications tray and view the notifications, click, go to that notification app. All this is possible and we were able to do this. The other use case is on the battery. Now you saw we have multiple apps. If we had to run our tests, we wanted to ensure that we have sufficient battery before our tests get executed. As part of this UI Automator tool, we have a battery info method which enables us to see what is the battery level that we have. In our use case, we are considering anything above 50%, it's fine to run our automation tool. If it is less than 50%, we are not interested to run our execution. So we are performing the preliminary check how the battery condition of the device that is being used is in. It also has different states, but we are more interested in the levels, whether it is zero stands for no battery or it's completely drained. One is fully charged, which is 100%. I just quickly want to show you the apps that we were talking about. So here you can see this is the publisher app. So this is our publisher app, which is connected to HivemQ. This is our broker. It is emitting data on a topic called Hall Switch. So when you perform an action call switch on, you can see an alert got generated and all we wanted to do is go to that alert. Click it and see what is the payload that is here. As we said, it is zero and one just to turn on, turn off. In other sensor devices, maybe you want to make sense out of it, like what was the actual data that you received. So you can perform whatever validations or checks you want to perform. So you want to go to your alert app. This is how the alert gets generated. And similarly, you can actually do a switch off. If you perform a switch off, the alert should not be enabled and there should not be any alert in your modification. So this is automation is done using the UI Automator tool. Let me quickly walk you through the code. It is a pretty straightforward and easily doable thing. Here we are initializing the driver UI Automated Driver using this automation name. Then once the driver gets initiated, we are performing the battery information check. And for this, you can see, we are just getting the battery info and state. So here you can see the battery info is one, which means it is 100%. And it is charging because I connected my device to my laptop. So it is in a charging state. Once this is done, then what we're trying we want to publish to the topic. So we have to go to the publisher app. And in the publisher app, we are identifying the switch on button. So the switch on panel basically. So with that panel, we're performing a fake operation. Once this is done, you're actually publishing your data to this switch topic with the payload as well. Once we have published, we want to check whether did we decide any notifications. So you can go to the notifications and scan through all the notifications here. I don't have much of a notification, but we received the notification. And the MQTT alerts switch on was detected. So this is the payload one was detected. And this is one of the use case. Another use case was to perform a switch off. In a similar way, you establish a connection to your driver. You create a button using that driver. You identify your switch off panel, click it up, and then you go and check your notifications. And you expect there is no notification into that notification. So this is pretty much on the UI automated. There are other use cases wherein we were unable to achieve the validations that needs to be performed on the style attributes and the colors. So all this is done using the Espresso, which I want Jimmy to take it forward. And Jimmy will be presenting on the Espresso use cases. Over to you, Jimmy. Thank you. And stop sharing my screen. Yeah. Let me know when my sharing is visible. It's visible. Thank you. Yeah. Thanks, Rashmi for elaborating the use cases with the two applications like publisher application as well as the modifier application. And luckily we have our UI automated to driver to overcome all the challenges we faced. Next is our main dashboard application. You can see here that I'm sharing my phone screen. So this is my dashboard application. This is again a simple application and we have added two panels here. One is a switch panel and one is RGB LED panel. Now, what will happen when we will start to switch on operation or switch off operation in the publisher, right? This switch or LED is actually subscribed to the same topic. So as soon as we received some event, this, you can see like, you know, there will be some color change here. Okay. So we need to basically verify what is the color change. And then we have to add our validation in our test scripts. So how we will be doing that? I'll be walking you through that process. Since we are using Espresso driver as we are already like, no, we need a debug app here. So we have the complete source code available here. If you see any Android app, right? So there are different ways know how we can actually set the colors. So in this application itself, you can see one text called network, right? And next to that, we have a text called MQTT. And there is a difference between color for both the text, right? So first I will show you how we can actually verify the color of the network text because that is straightforward. So that is actually coming from Android main activity XML layout XML itself. So how we will actually find out, right? First we need the ID of that particular element. So the ID is this one like the text view underscore text underscore connection. That is the ID and you can see here that is the element. And if you look into the attributes of that text view, right? You could see the text color is actually directly set in this XML file. So whatever text color is you are seeing the application, you can see the same text color here. So after that, now we know like, okay, we have a text color available. So how we can actually call this particular thing in our test. Let me go to the test. So first thing is we have to find the element like regular find element. So once we have the web element ready, we have to pass the web element to our back door. So I'm using mobile back door here and I'm passing the web element and I'm passing one method called get current text color. So this method is actually associated with the text view and since the color is also direct. So we are able to get, you know, that is actually able to get that color back to be in the script. So once I got that color back in my script, so I can do the validation as I want. Like what is the expected and what is the actual color. So this is the first scenario. Next scenario is like Hall switch, right? So if you see this switch, this is the switch element here. So the attributes you can see like ID is there, layout, weight, layout, height, gravity, visibility, right? But you cannot see, you are not going to see any color, you know, defined use, right? So then from where the color is coming, right? So we have to actually find out exactly from where the color is coming. Now if I go to my adapter class, I have adapter class available here. So in the adapter class also, I could not see like, no, this is not a user defined color, right? So next option I have is like, okay, color is coming from the theme. So as we know, there is a file called styles.xml in our source code, which is, you know, where we can define the colors which you want in our team. So basically if you see this style area, right? So here you can see our three elements added, right? Three items added. There is something called color primary, there is something called color primary dark and there is something called color ascent. So color primary dark, right? So color primary dark, you can see the status area of this app. The color is little darker shade, right? So this color is actually the color primary dark, which has been set by the theme. And you can see color primary, so which is the action bar, like we have this linear MQTT dashboard panel, right? So where you can see the shade is little lighter than that. And then we have color ascent, which is, which will be applied to any views, which are not like, you know, custom views, which are like, you know, coming from the default views, like switch, it can be, you know, any other views. So in our case, it is switch. Now to access that color value, right? So what we have done, we have added a custom method here in our main activity class. Since we are not having any, you know, handle to the theme. So what we have done here, like we have added a method where first we are searching the element with the ID of the switch. And then we are checking also the status of the switch. Like this switch has a function called ease check. Again, this function will not be available outside the source code. So we are checking that. And then based on the switch status, we are returning the color. Now, if I go to my test, you can see here, this time to the back door, I am giving the method name to get switch color. And I am giving activity name here because this time I am not passing the Web element. Since I have added a custom method, so I am giving activity since this is present in my main activity itself. So I could, you know, I could send that directly, like the method name. So back door, we return me the method name. And then I am also verifying the switch status. So I have showed you, like, you know, there is a method called ease check. So I am also calling that method. But when, while calling that method, since that method is part of the switch, you know, element, so I am passing the Web element. And I am getting the state of the switch, whether it is on or off. In this way, the type of panels, which are actually, you know, set by the team, right? Colors are set by the team, we are able to verify using the back door. So two types of validation are just what you drew. The next is custom view. Now, switch is again a default coming as a default view, like switch and text. But we have this LED. This is not a default view. This is a custom view. Let me show you the LED view. This is actually a custom LED view. So which is where we have different methods for, you know, for example, I want to know what is the color of the light, right? So what is the current color of the light? So I can call this is color light method from this view. Then if the LED is on or LED is off, even we can do set operation also. Like we can set the LED on and off using back door here. And next thing is color picker. Now, if you see this LED, right? So I'll go to the edit view now. If you see here, I have an option to pick the color, right? So I have a small icon here, which is showing whatever color I have selected. If I click on that, I have a color picker here. Now this particular color picker, right? Suppose I want to verify what is the color currently selected. And if I change any color, right? Which has to be updated, right? I want to do that validation. Now if I inspect this, this full picker is coming as one element like this coming as a color picker. So again, UI Automator will not be able to help me in this case. So I have to use back door. If I go to my, see, this is the color picker XML actually. If you see here, we don't have much, you know, again attributes other than we tried and I do. Let's go to color picker. This is the color picker class or color picker view, which is a custom view again. And luckily we have method called set color and get color. Right? So this method helped me actually perform operation and validation on this custom view. Suppose now I want to get the color. What is the color? Next selected color. I can use this get color and then I want to perform some action. There is another API, which we can call. And then I can come to the picker view and I can verify what is the current color selected. So this is the color picker and we have a color implementation class also. If you see here, you can see, you know, all the colors you have seen in the color picker, right? Which corresponding color codes are actually added here. And again, if you want to, you know, use this particular class in our test, we can actually use and we can even get the color by index. And the index are maintained like, you know, 20 colors are there and it was the codes are added as per the sequence. So these are the two main classes for my, for me to, you know, get the color from the view. Since if you see here in the adapter class, if I show you the RGB LED, so here also we are setting the custom color. So this is the set color light matter. The code is using and we'll be using get color in our test. So you can see here in the photo LED, I'm using this color light, right? So that will give me the selected color. And if you are in the color picker screen, then you can use get color method. I'll be showing you that as well. Okay, let me walk you through the test I have added. So first test is the color picker test, right? So they are, yeah, first step is like, you know, I'm getting the espresso driver and then I'm clicking on the edit play button and I'm going to the edit menu. And I'm clicking on the, I'll show you here. I'm clicking on the image and I'm in the color picker page, right? So one, I have that color picker web element, I'm passing that web element and I'm calling them get color. So espresso will return the color and I'll verify. And then I'm selecting a different color. For example, I'm selecting purple. Then again, I need to go back to the color picker to verify it because in the previous screen, actually, this color picker was not available, right? And the method I'm calling is associated only with the color picker. So we need to make sure if we are calling some, you know, methods from any view, so that view has to be present in the screen. Then only we will get the results. So this is the color picker validation. And next is I have a validation for the amputee dashboard panel, right? So that is my main application. So first thing is like I'll be opening the dashboard app, right? And then using the espresso driver and first device and I have connected another device where I'll be opening the other two application, like the publisher application I'll be opening and the notifier application. So from the publisher application, I will be, you know, starting the switch on operation and then it will switch on the switch and LED also will be on. And I'll verify the status in the dashboard. So once I verify the status, then again, I'll be going back to my publisher app and I'll be doing a switch off operation. And once the LED is switched off, then I'll check notify allowed is not triggered this time. And then I'll also check the panel dashboard panel so that my to see are very, very fine that colors are reset to default. So these are the simple steps I have, we have automated it. And yeah, so only key thing here is like how we are actually calling the back door. So to execute the back door, you may need to do something like this. Suppose, you know, I want to I want to call back door, which is specific element ID and the method name. So you just create a map and pass that map and the target should be as far whatever we are passing. Suppose we are passing element ID, so target should be element. And if we are passing activity, then element ID is not needed or if you're passing application element ID is not needed. Then we can pass the method name. Sometimes a method may have parameters, right? So that is also possible. And we can pass parameters like that. Like what is the type of the parameter like string or integer, all primitive types are supported. And what is the value of that parameter we can pass even multiple parameters we can pass users have to construct a map here and then give it to back door. Now once I have that, then driver.executeScript will execute that and give me back the results. Even we have done a click here. So in the color picker for selecting color, we have done mobile click action. So I passed that element as well as where exactly I want to click. I want to click in the top left. So it clicked and it selected the desired color. Let me quickly play the demo. First it will execute the color picker test. So you can see like, you know, I have changed the color. I clicked on green color from the default color. Then we have verified that. And next is on the main scenario. So the first application, the dashboard application, this is open by the espresso driver. And you can see the main activity. You can see the network text color, then the switch color before and switch state before. Now let me open the other application. So this is the publisher application which resume has explained. So I have two panels here, switch on and switch off, which will publish to a specific payload hall switch topic. You can see here, as soon as I clicked on the hall switch, I could see a notification alert in the status tray, as well as the hall switch is turned on and the hall LED is turned on. So that means notification is also triggered as well as the application, dashboard application also got driven. And it is doing the functionality as per expected. You can see here it is going to alert app in the first application. Now again I'm opening the publisher app and this time I want to off the switch. Switch is off and you can see immediate response in the dashboard app. Switch is off as well as LED is also off. You can see here the switch color and default to zero. After operation also all the things are verified and now we are closing both the application. Yeah, so using this two drivers we could actually achieve the complete end-to-end flow of this scenario, like since we were having three applications and two release app and one debug app and luckily the dashboard app was debug app. So we could use backdoor effectively and in the release app we could use your automated driver effectively and we could automate the entire thing. So Expressive Driver is very powerful in terms of getting things inside from your application. The only thing you need is debugable app and you should also know which method to call. So as I showed you different classes of my application, some knowledge of the internal of your application is also needed here. Rashmi, you want to add something here? No, like you mentioned, so we can use both the drivers and serve the required need like anything related to UI. You can go ahead with the UI automated and the nitty gritty is where you want to use the backdoor. You can rely on the Espresso and I think some question was asked on that front. We have used both in one test case to perform an end-to-end using both the drivers and it works perfectly fine. Thanks. Just to add one that Espresso does not need any weights and it is smart enough to ensure the weight and all is not required. It knows when to get triggered and when to perform a validation. Yeah, that is actually a new stuff like news because we have not used any weight in our code except for the demo purpose. We just added some thread.slate. That's all but without any weights we could achieve the entire thing very effectively. So these are few of the references like we referred while creating this demo and the sample code of the source for the demo application is also available. We took it from internet only and then our slides and our demo code it is available in a GitHub. This is our social media handles. That's all I think we have for today. So we are ready to take up the questions. I think few questions are there. Yes, thanks Dimpy and Rashmi. This was a great demo. Live demos are very interesting. I'm glad you were able to show code and walk everyone through the intricacies of how you actually did this. So thanks for that. So there are a few questions over here. I'll just start reading them out one by one. Rashmi, thanks for answering the first one already. So the first question is why are we using Android Studio rather than APM Inspector? So we have used APM Inspector to inspect it but Android Studio we are using to see the internal methods since we are using backdoor. So we need to know what are the internal methods available. So that's why Android Studio used. Great. That is good. The next question is so we can run tests with different drivers in a single APM automated test run. Is that right? Yeah, we have used two devices actually for this demo but we have used single APM server. Okay. I hope that clarifies and provides an answer to your question. The next question Sangam is asking is did you face any challenge while launching the app using Espresso? Okay. No, we have not faced any challenge while launching the app using Espresso. One challenge I could see while accessing the notification actually although we are able to get the notification. We are able to open the notification trade but we are not able to access the notification content using Espresso. Since Espresso is actually a context will be only to the app. It is specific to the app which we are building. So that's why I like for notification app and clicking on the notification could not do using Espresso. Otherwise launching closing app there was no issues. Okay. Thank you. The next question is any specific reason to use Espresso driver for part of the use case? Okay. The second use case where we explained right so that was a dashboard app and there was color changes happening based on incoming events. Since the color codes are not available outside the app so we need to actually look into inside the app and see where from where the color is getting set. And then call the corresponding methods using Espresso. That is the major reason why you use Espresso. Makes sense. The next one Shamal is asking is can we automate as headless automation for mobile app? I mean without showing UI in devices. Okay. At least I am not aware of it. We haven't tried all that. Yeah we have not tried. At least we have done headless hooking on to browser stack and all but yeah mobile apps we haven't. Shamal I think this calls for a feature request and a pull request coming from your side as well. Yes. Okay. The next one is could you paste the links here in chat? Shamal in fact the slides Tim P and Rashmi have already uploaded on the proposal page itself and the resources the links who are already available over there. Yes. So that would be better otherwise this chat would be lost once you move out from here. The links are already available in the PDF link to the slides on the proposal page. Sorry I answered that on your behalf. No problem. Thanks. So the next is congrats to both of you interested to learn Espresso after this session any reference we can refer to. We actually started with the page so all the references whatever we followed I have added in the slide link. Okay. If I can add on to the same question from my personal side right what was your journey of learning Espresso and getting to the stage where you are in. Okay. The first one was the primary source for us all the methods the read need of was very helpful using that we have explored quite a bit and we at least we both could do it. Okay. And one more thing is like you know since this is internal right so when we call methods which are available internal to the application. Knowledge of Android application like internals also helps. Otherwise like you may you may a little bit no struggle initially like you know from where to you know get the method how to search the method. Things custom views. Yeah, all those kind of things. As we said, knowledge of those kind of things help. Makes sense. Thank you. The next question any specific functions or capability or features Espresso has over UI Automator 2. Okay. Backdoor is one factor is the main one which is the main thing. Okay. Well, great I think that's it for all the questions that are there. There was one comment. Could you please share the source code link. I believe that was also there in your references. Yeah. You had asked that in the chat. So the link to the source code is also available in the slides. You can take a look at that from the proposal page. Even I have created the debug app and the debug app also is available in my GitHub. I uploaded that. Excellent. I believe we are done with the questions now that are there. And with this, in that case, we will stop this session. So again, thank you very much, Rashmi and Dimpy for this amazing session. Looking at the interaction and the feedback on chat. Everyone really enjoyed it. Thanks for joining the session. It was great interacting with everyone.