 Thank you. So today we will speak about basics, so why basics? Because we already had like several meetups about IPM itself, but I saw that the community is not so big. So we decided to make one basic presentation about IPM itself, how to set up it, how to work with that, to increase the size of our community. So we hope that you will also join to our community and we'll be contributing or we will try to help you if you will have any problems. So let me speak about what we'll do today. At first it's our first meetup and also it's our first presentation in those circles, live brand. We will do much more presentation, it will not be only about test automation, it will be about completely different topics, so please follow us, maybe you will also find something interesting. Soon we will have much more presentation under this brand. Okay, so about me. My name is Pavel Strongin, I have a little bit more than 10 years experience in information technologies. I worked as a system engineer, I worked as a developer, I worked as a system administrator, I worked with such companies like ABM, IPM, Yandex, Lazada, and now I'm working here in Circles Life as a QA lead. Initially I'm from Belarus, not so many people know where is it, that's why I put a map here. Yeah, this is Russia, this is Poland, and Ukraine downstairs. So just a few words about Belarus itself. So we have beautiful castles, we have beautiful girls, we have last bisons in Europe or Asia, and we have the last dictatorship in Europe. Okay, so tools that we will use today. Of course it will be APIOM, because here and in Lazada I used Ruby, so mostly I will use RubyMine and Ruby itself. I will tell you from the beginning that it doesn't matter in which language I will show it today, because it completely same on all languages and all the realizations of APIOM libraries. Also we will speak a little bit about allure reporting, about simulation with GenieMotion, and about simulation with GenieMotion on Google cloud. And of course I'm using Mac for everything. Okay, so let's speak about APIOM itself. So before APIOM we had of course many different frameworks to test mobile device, mobile applications, but mostly you had to add the SDK inside your application. So every time you want to test or you want to change something, you need to recompile the application. So APIOM was made to follow the next philosophy. So apps shouldn't require including any SDK. So you should be able just to get any application and start your testing, as you're doing with everything else, like with browsers in Selenium or command line on API, so you don't need to change anything on developer side. You can just take it and test it. So you don't need to recompile your application. Also you should be able to use your test practice frameworks and tools. So you can choose any language you like. You can use any tools you like and use it for testing. So you shouldn't be limited. For example, like with Espresso you can use only Java and you should use only IDEs that provided by Google. Or if you're testing iOS, you're limited to use Mac OS and Xcode only. So APIOM gives you a freedom in that. So you can choose anything you want. Also APIOM is an open source and free project with a very big community right now that contributes a lot. So if you check the APIOM repository like three years ago and right now on GitHub, there is a huge amount of small project already added to APIOM. So APIOM right now it's not only one server. It's a lot of small projects, a lot of models, a lot of plugins that helps you to do usual stuff very quickly, very easy. So everything already invented. You just need to take it and use. Okay. So let's speak about main features of APIOM. So first it supports both platforms Android and iOS. Also it's not limited to it. So there are also better to test Windows some way. You can find a special driver to do that. Also APIOM supports all the types of application right now. It's native, hybrid and web application. So you can test native application if your application have web view. You also can do it. If you need to test on browser on mobile, you also can do it with APIOM. It supports any types of device. So you can do real devices, you can do simulators, you can use clouds, anything you want. Because for APIOM doesn't matter which type of device connected to your machine. It just should be connected. You can use any programming language. So for all the main programming languages, you already have the libraries that supports communication with APIOM server. If you're using some kind of not popular language, it's not so difficult to implement it because APIOM itself is just HCCP server that follows web driver protocol. It has the same protocol as used by Selenium. So if you're somewhere trying to use unpopular language for test automation, never do that. You still can use APIOM. Because it's following the same protocol as Selenium, you can use Selenium Grid. Same way as you used it in Selenium. Who knows what is Selenium Grid? Okay. So I will note that Selenium Grid is a tool to parallelize your test. So if you want to run your test not one by one, but in parallel, Selenium Grid is one of the tools that you can use. Also you can use any other tools. You're free to do anything with APIOM. And yeah, I noticed that it's quick start and you will see today why it's really quick. You don't need a lot of knowledge to start testing with APIOM. It's very easy. It's basic and we will see it today. So our requirements. You need to put this server into your office. Now I'm joking, of course. Requirements once again. Any programming language. Java, Ruby, Python, anything you want. Any operation system. APIOM works in macOS, in Linux, in Windows. Currently I'm using only Mac because I'm doing testing for both Android and iOS and with iOS you couldn't move from Mac anywhere. So you always must have your Xcode. So if you want to test iOS, you should use Xcode. If you want to test Android, APIOM supports Android starting from SDK version 16. Before that, they had a different driver called Selendroid. But probably you don't need it anymore. It's too old. 16 version is below Android 4. And you need to install Node.js because APIOM implemented itself in Node. So you need Node.js just to run the APIOM server. Okay. So how does it work? As I told, APIOM is just a HTTP server, but it doesn't work by itself. It also needs something. So between your tests and real mobile application, there is two main things. APIOM server that accepts your client request, web driver protocol here. So then depends on the type of automation you use, Android, iOS, or type of the driver, because for Android, we will discuss it a little bit later, we can use different type of driver. It will choose a special plugin to use with this driver and install a special server that works only like a proxy to use internal API of platform. So in case of Android, it probably will be UI Automator or UI Automator 2. In iOS, it will be XT UI test. So there is two types of APIOM servers right now. Actually, it's one, but one is UI with additional features. So UI server you can download in APIOM, there's a download link. So it will be UI version with inspector possibilities. Or you can just install it in your Mac. If you use Mac, you can just run these three commands and you're getting APIOM server running. How does it look like? Okay, I need to close it for a second. Okay, so this is APIOM server UI. So it's just a small tool. You press one button and it starts the server. That's all. And same you can do it just in command line. Just run APIOM. And it also starts in APIOM in command line. So probably for writing the test, you will always use the UI one for CI integration. So for continuous integration or running on Jenkins, you will use the command line one because it's easy to script. There's no difference. So it's still UI version just running the command line version. Just has additional features. We'll speak a little bit later about that. But one more notice you should know. If you're using UI version, check the version of APIOM server inside because they release UI version a little bit later than real APIOM. So if you need any new features, you should use command line first because UI will be released like several weeks after. Okay. Of course, if you have server, we need a client. APIOM clients can be found here. I put QR code. So if you want to see the full list right now, you can do it. Oh, sorry. So currently it's implemented for next languages once again. So if you're using something not from this list, some why you still can use it. Just HTTP request and it will work. Okay. Next thing we will speak, it's about how to set up APIOM and how to work with that. So that's why we will need desired capabilities. It's something that called APIOM config. This is a way to tell APIOM what are we testing, where we're testing and how we're testing. It can be set up on both server side or client side. Depends on use or how you will set up your environment. But there is several, there are capabilities that should be sent always to set up your session to the server. It's a platform name where you specify iOS or Android. Platform version, device name, where you specify how to find your device. Automation name, this is the name of the driver. So for iOS, it will be almost always exterior test and application password to get your application. Everything else will be done just by your APIOM server. So it will find your device, it will find your application and will install your application on the device and will start it. There's a lot of possible capabilities. So you can find all in the documentation. There's a huge list. Probably you can set up everything you need and even, I don't know, I don't use half of these capabilities, but maybe if you will need it, if you'll find it helpful, you can use it. So some of them universal, some of them only works with special operation system like your DID. For example, if you want to work with real device on iOS, you should specify your DID to connect to it. Okay. So one of the third capabilities is automation name. So possible variants for now, it's possible drivers that you can use for automation of your application. For Android, it's APIOM. It's previously called UI Automator. It's one of the first drivers for Android that were used by APIOM. The only problem with that is that it's outdated. It will not work with Android 8. So if you need to test newer devices with Android 8 or API26, you should use UI Automator too. Also, express driver available, but it's still in beta. So if you want to use it some way, be careful because nobody guarantees that it will be stable right now. Maybe in the near future, they will release the full version. For iOS, currently, the XE UI test available for all the applications that you will use, you are engine for applications that develop it with UI engine. Actually, I never heard about that before. I just found it in the possibilities of APIOM. Okay. So to set up your capabilities, you need to get your device ID or your device name. I will show you how to do that. If you're using Android, you just need to execute ADB devices minus L, and you will see your connected device. My device name is this AP address because I'm using simulator right now. The not real device connected. If your real device will be connected, it should show your name of the device. So if you want to use APIOM with your device, you're just copying this part and setting as device name. For iOS, there is two variants. If you're using real device, you should use iTunes to get your UDID. So you just open iTunes and it shows your device UDID. If you want to use simulators, you need to execute the command XIRAN, sim-ctl-list, and it will show all possible simulators that exist in your system. So you can choose any device you want and any operation system that install it currently. You can manage it by Xcode or by this command line tool. It's very powerful. You can create any combination of device and operation system you need. Also, you can create custom devices with custom names, but it's a separate topic because there are a lot of possibilities. So right now, what all you need is just this name and this UDID, and you will be able to connect to this simulator. APIOM by itself will start the simulator, will install the application, and if you need it, you also can kill the simulator after the test execution. So let's try. Just for trying, I will use the APIOM Ruby console. It's ARC. It's just a very simple implementation of APIOM client. So basically, it starts the Ruby command line and imparts the APIOM library just for me to show you how to execute commands and just to try anything new because usually if you have a huge framework and you want to try something simple, you just need to isolate your code or split a breakpoint or something, ARC or any implementation like that can help you to try your new features very fast. So I'll bring it back. So I'm starting APIOM server. It started. I have already the device connected. So then I need to set up my capabilities. This is all capabilities I need. I'm using Android platform. I use device name this one. I'm using applications that are located in my folder and I'm using the UI Automator 2 driver. So let's try. ARC automatically set up the connection to APIOM, starts the driver. So right now APIOM installed all applications that it needs like proxy server, APIOM driver on my device, and then installed application itself and started it. So now we see that our application started and we have a command line here so we can just try to send several comments. So let's start from something basic. Of course, when we're working with UI, we want to get some kind of IDs or any locators that we can use for our elements identifications. So APIOM also can show your UI as XML documents. So it will use the same pattern as you used in Selenium. So I can use source and I will see the all XML files that presents my current screen. Yeah, it's very big and uncomfortable to use. So there is some helpful stuff like page and it will show me all the elements that has ID on this page. So if your developers made a good job and assigned IDs to elements, it will make your life very simple. So for example, I want to click on the get started button and I already can see here that I have element that has text get started and has ID get started layout button. So all I need is just to say ID, copy this ID and I'm getting Selenium element. So as you can see here, element is just Selenium WebDriver elements the same way as you worked in Selenium. There is completely no difference. It uses the same Selenium library as you used in your Selenium. So it means that you can do any same actions as Selenium can do. So you can get its text. You can click on it if you want and it will click. Also Appium have a lot of helpful methods that you can check here. So there is huge amount of methods available. So you can work with alerts directly so you don't need to find the ID of alert. It will automatically will find the text that it's alert and will do any action you need. You can get any information about your application. For example, if you're using localizations in your application, so you work for different markets, so you have different languages. So you can use application strings, application strings, actually just a file of keys and values. So it's some kind of dict that use it inside your application for localization purposes. So instead of using real text for searching an element, you can use its ID or key that will never be changed. So even if your text will be translated into another language or some why designers will ask you to change the text, you don't need to fix your test because you can use the key of this text that's also used by developers inside the application. Also you can collect data about performance. For example, I will copy this. So you can get the current information about your performance of your application. So you're using package name and information you need so you can use it. The user, usage currently is 0.3, currently 0.1. So you can monitor the performance of your application during test execution. And you don't need any other tools for that. So you can use APM for that also as well. If somewhere you forgot that which properties you set for your driver, you always can get it back. If you don't remember which platform you're using, you also always can check your platform. So it can be helpful if you have one framework for both application for Android and for iOS. Because for now I write only one test case for both platforms. I'm not copy-pasting my tests everywhere. So it helped me just to have one library for test cases and just to separate libraries with locators how to find these elements on Android or iOS device. Okay, let's move. Okay, so let's speak about APM Inspector. It's a tool also implemented by APM. It's already included in APM UI server and it helped us to much easier find the elements. So how does it look like? So here on the server you can see the start inspector session button. You're just clicking this button. Yeah, did I click it? Ah, okay, it's opened another screen. Okay, so you'll see this screen. It allows you to set up the same session as I did with ARC but with UI. So it's much easier for you. So you can see, you can edit your capabilities anytime. I already have a preset capabilities. So it's completely the same that I used in ARC. So it's also Android, same device, same application, automation name, but also here I set the new command timeout because default timeout for APM is about 60 seconds. If you're not sending anything in 60 seconds, it will close the session. So probably if you're doing something long between your tests, it makes sense to set up this timeout. So APM will not stop the session. Okay, let's try to start the session. It will do completely the same thing. So it will install all the support applications. So like UI Automator 2 proxy and my application on the device and will edit the session. So here we can see completely the same screen from here but also now it's attached to our source. So I can click on the element and find its ID, XFAST and all the settings of these elements. So it's very fast way to get your elements, especially if it has ID. Sometimes, especially with Android, if your developers using fragments instead of separate layouts, so it may show that you have two screens one under another. So in this case, you will not be able to click directly here because it will think that the screens on the background is actually on the top. It's some kind of Android problem. You cannot do anything with that. So in this case, you will still have to use your XML just to find your element. Also, if you found your element, you always can send the tap action. Wow, interesting. And it will do the same. Also, it allows you to check your XFAST. So if you're trying to not use this long XFAST and you want to make it short, you can use the search element, choose the strategy XFAST, for example, and put your XFAST here and search. So if it will be able to find your element, it will show all available elements by this XFAST. So you can use APM-Expector just for checking your XFAST. Is it right or not? So you don't need to put it in your framework first, execute, get the fail result, and try again. Okay, let's continue. So the other way is how to get your identifiers. Of course, you can use APM-Expector. It's like most universal tool right now, so you can get its elements from Android, iOS. But if you like other tools some way, you can also use any other tools like XCode accessibility inspector. It's part of XCode IDE. You can use Android Studio Inspector. It's also part of Android SDK. You can use Macaca. It's Alibaba tool for the automation, but it also can find and record all your actions. So maybe it will be helpful for you. Also, you can use source code if you know how to read Java code or Object C code for iOS. You can use source code to get your locators or set your locators. So if you will want to set your identifiers by yourself or you will need to ask your developers to put some kind of identifiers for your application, you need to set next things. For Android, you can use resource ID or content description. Because in some cases, developers cannot set resource ID for an element. There may be some kind of limitations in their current implementation of software. So in this case, they always can't set content description. APM still will be able to find it. For iOS, you have the three types of accessibility identifier, value and hint. All three of them can be found by APM. And in APM, you can get text from all three of them. So you can get values from all three elements. So you can use it as some kind of help for your automation. So for example, APM has a very big problem with iOS when you need to get the text label text. Sometimes it just cannot get text from text label. It seems like basic operation, but implementation issue on iOS side that they don't want to fix. So in this case, in Lazada, we implemented things that we put the text from any text label inside one of these accessibility parameters. And APM was able to get it from there. So you still can test, but not directly. Okay. If we want to test hybrid application. So then we need to know how to use WebU. For Android, if you're using a real device, you will need to ask your developers to allow access to the debug functions of your Chrome. Because the Android like to do protection everywhere. So if you're using simulators, you don't need to do that. If you're using a real device, you need to unlock the debug functions of your Chrome browser. For iOS, you don't need to do anything. You'll be able to use Safari and WebU from the beginning. So APM see WebU content as another context. So if you just will come to WebU page and will try to get a source of this page, all you see is just one element that calls WebU and nothing inside. To see the real content of WebU, you need to switch a context. So when you come in there, in your test, you just need to get available context to check that you already have WebU context and just switch to WebU context. That's all. But sometimes it makes problems, especially at the beginning, when you're trying to get element, you know the ID, but somehow you cannot get it. Localization. APM can get default strings XML for you. So what is strings XML? Any application uses for setting the strings for all the elements or for all the UI elements in your application. So developers use this to keep all the strings in one place. To not put it everywhere in the code, just hardcoded somewhere. So all the strings of Android and iOS applications are located in strings files. So APM can get the strings file from the application for Android, for iOS, it cannot be done. Because iOS compiles the application, so it's transformed all the files into bytecodes. So if you have already built application, you cannot get strings from that for iOS. So if you need to test localization for iOS, you need to copy the directories with strings files before it's compiled. So from the source code. Okay, let's speak about, yeah, hacks or workarounds. Because it's not real hacks. Sometimes APM works ideally, but some things you still want to do better or faster. Before version 1.8, APM, the main problem of APM was it was very slow. So all other frameworks was much faster. And every time you want to discuss with other developers who use other frameworks, they will always use this argument that APM is too slow. From version 1.8 is the current version, APM becomes super fast, but still you can make it faster. So first, you can use ADB. So ADB is Android bridge protocol that allows you to do anything with Android device. So it's allow your access to the shell or for main function. So you can switch to flight mode. You can choose the language on your device. You can upload and download files. For example, you want to upload some photos if your application is using camera or something. Or you can download logs if your application writes some kind of logs. Also you can do screenshots because sometimes APM can do it very slow or make, can make it like with some kind of problems. So if you face it for like unpopular devices, you can implement your own method for getting screenshots using ADB. It may be more stable and faster. Permission. So if you ever tried to use Android starting from version 6, you saw that during usage of application first time, you see the alerts that ask you, do you want to allow this permission for this application or not? And of course it may be a real problem for tests because it's asked only first time. So you should put a lot of ifs in your framework or in your test that if alert is shown, then agreed. If alert not shown, then they do other actions. Of course we don't want to do that. So in this case, you can, okay, I'm telling about another slide. Yeah, I'm telling about, so you can allow your permissions before tests really started. So you can use ADB command just to allow all permissions before you start your test. So Android will not ask you about any other permissions during your test execution. So your test can go very good. And if you need to test your permission, that if you need to test the functionality that your application uses permission, you just disable the turning off the permissions and you get all the permissions back. Also, there may be a problem that I faced with one of my projects that application asked for permission when it starts. And this is the blocker for APIOM. Because when APIOM start application, it allows you to manipulate this application only when it gets the starting activity. So any Android application defines the starting activity. So some kind of first screen that should be shown. And APIOM always wait for that. So if you will see, if Android at this time will show your notification about permission, APIOM will not be able to see the activity behind this permission. So APIOM will not be able to start. So it will give you just time out that your activity not started. So that's why they implemented the functionality to set non-default application activity and package. So you can wait for your permission request, for example, at the beginning if you know that application first shows the permission request and only after that the first activity. So you can ask APIOM to wait for this activity and only then wait for real first activity of the application. Okay, multiple executions on one machine. Of course, you don't want to have one machine with one device. So if we're speaking about our device farm locally, so of course you want to attach as many devices as you want to your local computer that will be used as main hub for connection. So in this case, you need to specify as separate APIOM execution. At first, basically one APIOM server can work only with one device. So you need to start the same amount of APIOM servers as many devices you have or you want to run in parallel. And every instance of APIOM should have own APIOM port, bootstrap port or system port capability, and from driver port if you're using for hybrid application or web testing. In other case, it will all go to same port and you will have problems because one test may affect another one by sending comments to several APIOMs at the same time. Another problem you may face is hiding of keyboard. So sometimes when you enter text into a text field, your device showing the keyboard that hides the half of the screen. But for example, you need to check something under the keyboard. So you need to hide this keyboard. If you're using simulators, you can disable keyboard easily. So it will not be shown at all. So you will use just send keys to send any text and everything will be good. For IS, it's impossible to do at all. So you cannot just hide the keyboard. APIOM has methods that called hide keyboard, but basically just click enter button. So if your application accepts enter button as go to next screen, you will go to next screen. So you will not be able just to hide the keyboard. So on real Android devices, you can cheat a little bit. You can use ADB to disable all the input methods. So Android will not be able to show any keyboard if it doesn't have any keyboard. Okay, alerts. APIOM has several methods that you can use for alert, but they miss the one alert displayed. So they don't really have this method to check same moments that alert really displayed. In all the cases, if you ask APIOM just to deal with alert, it will wait that implicit timeout. So it will wait like 10 or 15 seconds. Even if you don't have this alert, APIOM still during this 15 seconds will try to find it. So maybe you will be interested to implement new methods that checks the alert just same moment. Do I have alerts right now or not without waiting anything? Yes, text label, that's what I already told. So APIOM sometimes can get text labels, text for IS. So then you just ask your developers to create a new category that puts label text into accessibility attribute. Anyone. Hint, value, ID, anyone you want to use, you can put it there. WebView. So for WebView, if you're trying to use APIOM just with usual methods, send keys, maybe you will notice that it works very slow. Because it sent one symbol, then check the length of the string. Is it full or not? Then send another symbol. And if your machine is not really powerful, it takes very long time to input long string. So we just implemented it with JavaScript. APIOM has the possibility to send any JavaScript for the current screen. So you can use any JavaScript code inside the execute script method and send it like to set value or just click on checkbox or anything else. So same as you did with your Selenium test, you can still do with APIOM. Okay, let's speak about GeniMotion a little bit. At first, GeniMotion itself, it's Android simulators. This is the one of them. It gives you some additional possibility like it has like a little bit more functional analysis than usual Android studio simulator. It's more powerful. It's much closer to real device because it's a full virtual machine with simulator running inside. Also, if you will buy the full version, you will be able to modify GPS coordinates, ID of device, the connections, the network and so on. So right now I just downloaded the free version for demo purposes because I'm using GeniMotion in cloud. But if some way your application require usage of GPS or something like that, maybe it makes sense to buy GeniMotion simulator because it will be much easier to manipulate than the real device. Okay, so let's see the GeniMotion implementation in cloud. So they provide you three types of simulators. First one is a local one that you can run on your machine just for test purposes or just for development purposes doesn't matter. Also, they provide you a virtual machines for cloud. They have both for Google cloud and for AVG for Amazon cloud. I tried the Google one because it's a little bit easier to use and it gives you some benefits. This is a Google cloud console. Here you create just usual virtual machine with their template of GeniMotion and you can access it the same way as you access it on your local machine. So they provide you a special port for ADB connection so you can just connect it to local ADB and ADB will sync or APUM will sync that this device connected locally to your machine. So for APUM it will be completely the same as the simulator running on your machine but actually it will be running on the cloud so you're not using the resources. It works much faster and you can use any amounts and types of the machine prepared also. Google, if you will try it on Google, Google will give you 300 dollars and one year trial for free. So for every new account if you're just joining Google cloud you're getting 300 dollars to try all their possibilities. This simulator costs only half dollar per hour of usage. So if you will turn it off right after your test probably you will not spend a lot of money. So right now I played with this 300 dollars for like three weeks and I spent only two or three dollars only. Benefits also have some disadvantages like it takes some time to set up at first time. It's not really obvious so you will have to read a little bit of documentation about that. But when you set up at first time it will work very good. They provide you a command line interface on your machine that can work with cloud machines so you don't need to implement any API tools or anything. You just use this command line tool to send the command to start machine, to stop machine or do anything you want with this machine. Another disadvantage is that you don't have the possibility to modify genie motion features from command line. So if you want to set another GPS coordinates or another ID for device you should use UI only to set this for simulators. But you can create any amount of simulators so if you want with any setup and then use it in your test when you need it. So you don't need to change your settings all the time. You can have like million machines at the same time. If they don't work they don't use your money. You have shared access so you can create a lot of devices same in cloud and everybody can use them. It's fast. It's really fast. I even faced the problem with APELM with your UI Automator 2 driver. It cannot work so fast sometimes. So yeah. The problem I already posted in our Slack channel that if your tests work so fast with UI Automator 2 the proxy server that they install inside the device sometimes can't stop working because it cannot work with so many requests per second. So in this case I'm looking for a good solution right now. I definitely don't want to make my tests slow. And another one small hack about gcloud. If you will just press the stop button on your virtual machine it will take about two minutes to shut it down. But if you will send adb command to shut down the machine from the inside it will take five or ten seconds. So it will save you a huge amount of time if you run tests from different instances with different indices. Another topic I want to speak today is just a small advertisement of ALU reports. Maybe some of you already seen that. Maybe not. It's just one of the implementation of test reports. It was implemented by Yandex company, Russian guys. It's very simple and very nice because it provides you all the features you need. So it helps you to shorten common defect life cycle because when you get any year with ALU report you can get everything you need from the beginning. So you don't need to go to logs. You don't need to debug anything. It can collect all the information for you. You have failure separation into bugs and broken tests so you can easily understand where's the problem. Is it your tests or it's a bug? It supports all types of attachments. So during your test execution you can attach any type of the file. So you can attach skin shots, you can attach log files, you can attach APM output for example. You can, it supports BDD framework so if you have cucumber you will just put one library APM cucumber and it will work from the beginning. So you don't need to implement anything. You even don't need to mark steps. It will automatically get all your steps and put in the report. It has integration with CMS and bug tracking. So if you mark some kind of test that you already have a bug in this test it will be shown in report and it will not show the red status anymore because you already have a bug about that. It has timelines and it supports parallel execution. I will show you right now. It's on an example. I'm just telling about all the features. So you can use it for parallel executions. You can use it with execution history because if you're using it on Jenkins or you're using it with SimCity it will support the history of execution. So it will know all the history of the test execution so you will be able to see how flaky your test or how unstable your test is. So let's try a demo. If you want you can try it on your phone. I will also show it here. This is a demo provided by developers of Allure. So they have all the types of tests here right now. So this is the main screen of Allure reports. So it shows that we have 21 test cases. This is a presentation of Passet tests. This is our history trend. So every execution it shows that we have like Mowgreens here. We have less here. It shows you your suits if you're using not BDD framework or if you use BDD framework it will be shown like features and storage here. So you can see everything separated into your features and storage. Also you can add any environment information here. You can use links. You can use information about your device that were used for execution. So you can provide all the information needed for developer to debug a problem. So let's take a look closer to our bugs or our steps. It's a simple scenario. The green one. So we have steps here. So we're doing some... Can you see how I need to make it bigger? Yeah, it's like better. So you have steps here. If you have any attachments you can also see here. So you can put any attachment here inside the step. So you'll be able to track or reproduce anything you want to follow the steps to reproduce your bug. For example, you see that on the third steps you got some kind of failure. You can put all information how to reproduce these steps here in attachments. So developer will be able to reproduce your bug by himself. You don't need to come and show to him. You can provide all the information here. Also you can mark your test, skip it. Yeah. So if you want to skip some tests, some why, it will not be red. So it will be gray. So you don't need to pay attention to that. Also it supports unknown status. So some why you're working right now on the test and it's got into the test execution. So you mark test in the progress like unknown or skip it. You can customize these names by yourself. So it also will not be red. So you don't need to spend time on that. So you know that your problems is the red test case. So purple is something you can do later. Also, as I told, it supports the integration with any bug tracking or CMS. So if you attach any JIRA ticket or like any CMS ticket to your test case, it will be shown here. So you will know that why it's failing. Yeah. They don't have JIRA attached to this report. So it's nothing right now. Same with this case. So if you want to see the real test case, where did you get this test from or your manual team doesn't like how this test case look and they want to use their test case in any CMS system, you can attach the link to CMS. For a failure test, it will provide you all the output you need. So it will be a stack trace here, the error message, and all the outputs that you attach it. So you will be able to see why it's happened or what's in your log files. Yeah. And yeah, skip to all of this one. And broken test, yeah. If you mark some kind of type of errors as not product errors but framework errors, this test will be marked by different colors. So you will also know that it's not a bug that's problem in your framework. So you don't need to send it to developers for investigation. Allure also multiplatforms. So they have implementation of their library for almost all the languages. Yeah. So you can find it also on Kamiata.io Allure. You can just google by Allure reports keyword. You will find all the documentation. It's all in English. It's very simple, very simple installation for Jenkins and SimCity. They have their plugins, so you also don't need to set up anything by yourself. All you need to specify the folder where do you put your Allure results. That's all. Everything will be beautiful from the scratch as you want it. Okay. I think I finished on that. Any questions? Guys, do you have any questions? Okay. So I'm not using Java, but usually it's just one library. So if you're using any BDD system like you can bring anything, you're just adding this library into your project and that's all. Everything will be done automatically. If you're not using BDD system, then you need to add a special call to Allure library like start step and stop step. So then Allure will know all about your steps. If you need to add attachments, you're telling Allure attach and you get the contents that you want to attach. So it will put all this information into XML files in some folder and then plugin for Jenkins will be able to build the HTML reports from this folder. So from you, you just need to say where to start, what to put inside and where to stop. That's all. If you're using BDD, you don't need to do anything. I never work with Serenity, but I just know that it's the simple one and it can do everything I need because I know that I can send this report to my developers and they will be able to find bugs without me. So I don't need to participate in this process. So it's simply why it's a process. I can send it to my managers and they will get all the information. I will show you all the information about, so all the statistics about the problems right now and also timelines. So how long does it take to execute which test? So you can see here that several tests were executed parallel, some was executed earlier. So if you have only one execution at a time, it just will be one line separating to test and you will know which test takes which time or how much time did it take. Also, if you want to contact me, you can scan this bar code or QR code or you can use any of these links, LinkedIn, Facebook, Instagram, anywhere you want. Feel free to contact me to ask the questions. Also, we have Slack, so if you have any problems with Apium, you have any questions, feel free to join our Slack by this link or by this QR code. You can send an invitation to yourself into Slack, in our Slack channel. You can ask any questions there and our small Singaporean Apium community will try to help you to solve this problem. Okay, so thank you very much. I think that's all. We have to list all the contacts first and then after that we have to choose which contact you want to go. But what happened after that? We don't see the real HTML. Because we run tests on Android and iOS. In Android, we can see the full HTML page. But in iOS, after we switched to the context, we didn't see anything just HTML and HTML. Are you sure you switched to the right context? Because sometimes on iOS, I thought that it creates two WebU contexts. So maybe you just switched to the wrong WebU context. First, you need to get the full list of the context. Check to which one you're switching. If you're switching to the right context, then you just need to investigate it with your developers. Because for me, it always worked with a hybrid application. Because the other application was a hybrid. So we had a native part. We had the WebU part. So switch context always worked for both Android and iOS. So you have multiple contacts sometimes? Yes, yes. So maybe it's an issue made by your developers. Maybe it's some kind of iOS issue. With iOS, we have a lot of problems. But you just need to investigate a little bit more. So also you can like post your problem in Slack. We can try to help you investigate. Okay. Yes. He was first. Sorry. Yeah. What's your think on testing mobile device? Okay, in Lazada, we mostly use it only simulators. So because our application was pretty stable, we had less than 0.1% of crashes for our users. So we didn't care about the models of the devices because it's a very small problem of crashes. So we were able to use simulators always. Let us see if I want to start doing what we should testing on mobile browsers. It doesn't matter. For you, if you use a mobile browser, it will be completely the same for simulators and real devices. And of course, you cannot have like 10 different devices in real. It's too expensive. So for mobile browsers, I believe it will be much easier to use simulators. Sorry, your question. How do you run the... I was now... Sorry, how do I run Android and iOS? Okay. So it's just different instances of test framework. So I'm not running this really like in one test execution. So one test execution is for Android, one test execution for Android. But I have like initial parameters that when I'm starting IPM, my framework knows which platform I will work. So it just loads the page objects for Android or for iOS. And everything else is the same. So all the... I'm using Cucumber. So all the feature files, all the step definitions, completely same for Android and iOS. It just loads the page objects for Android or for iOS at the start. That's all. Can you set up the local device form? As I told, we use it simulators. So we had just several servers where we run simulators. And it was more than enough for our purposes. How do you handle the dynamic... So with simulators, you don't have this problem because every time you start your test, you start the simulator. When you stop the test, you stop the simulator. So it's restarted all the time. So there are no problems like that. With real device, of course, you will have it, especially with Android. ADB likes to hang up after some time. So you need to restart your device or reattach your device. So it brings a lot of problems. So if you're thinking about local device farm, maybe you can also think about real device farm, but in cloud. So maybe for you, it may be cheaper in like time spending or resources and so on. Okay. Any other questions? Okay. Thank you very much. If you learned today anything interesting, I'm glad. If not, also okay. So join us on other events. I hope we will soon have events not only about APM, but about test automation in general. So we will try to switch the context. If you have anything to tell, if you have any good experience, if you have anything to share, just reach us in Slack. So we will be happy to give you a chance to present something by yourself. If your company wants to or can handle such meetups in your office, also feel free to offer to Sam. Yeah, he's main organizer. Okay. So thank you very much. We still have 30 minutes for networking. If you want, if you have any questions privately, you can come and I will try to ask answer. If you want to drink coffee or water, feel free.