 Okay, are you good to go? Okay, so we start with the second part, which is about visual and layout testing, again with Appium. This is a bit different than the first part. This is less about presenting how we do things at Carousel. This is more a look into this field of testing with some ideas. I'm going to present some frameworks that can be utilized for that, so it's more of presenting ideas and giving you some thoughts to further look into. All working demos, but it's not as full-fledged as the framework you've seen before. So basically it will be three parts. First, something, yeah, some very simple use case. So basically instead of finding an element by very complicated x-path expression or CSSID, I want to put it simply, I want to do find that picture. Because sometimes we don't really want to care about locator, we also don't care about how big is this image. We just want it to be on our screen. We also don't care where is it and whether it's a pixel perfect match. So for example, some banner, some third-party logo that we need to show is it there, something like that. There are a couple of use cases where this could be interesting. And there is a testing framework, probably some of you have heard about, Sikuli, that does something like this, like a visual aid for test automation. And I looked into that. There is an Appium integration called Sikupium. It's like some GitHub project. I thought I want to share it. And then I looked into it and tried to make it run. And it's like two years, two years old when the last change was done. I can share the project. It's basically integration of Sikuli that does this image recognition test automation in Appium. And it all sounds nice and well. The problem is it doesn't really work or I spent the better amount of a weekend trying to get it to work. It's not been maintained. So it doesn't really work with the newer versions of Appium. It uses OpenCV for image detection. And there is probably just one combination of versions of Appium and OpenCV that would make this work. So I thought, actually, what do we want to have? We want to have a lot of possibility to find images on our screen, whether it's a phone or browser. And since Sikuli uses OpenCV, I thought, why not integrate that directly? So the test case, maybe this time we start from the feature file. The reason why this is only Android is because only Android has this little icon. So what we want to do is ensure that on the Stack Overflow app, this icon, which is icon to ask a question, is visible. So we have feature file for this. Given I am on the home page, I can see they ask a question icon. So then this links to, oh, sorry. And I actually done a little bit more just to demo that it's not like matching the picture pixel by pixel. I have the icon itself. I have a blurred version. I have one that's like slightly rotated and one where the image is distorted. And then just to double check, there's one icon that should not occur. There's an icon of Facebook. So I'll show a bit more code here. So we can also define a threshold. So for example, I think the default is I put 95%. So 95% match. For the ones that are slightly more distorted, you need to lower that. Otherwise, it wouldn't match. Actually, I can show you the images so you get an idea. So that's the original one. It's tilted by 10 degrees. This is, I think it's some lens filter applied. And this is some Gaussian blur. So getting back to the code. It's a very simple, short piece of code that uses OpenCV. OpenCV is, I think, a C library. But there is a Java version that uses JNI, a Java native interface to be able to call OpenCV. That's basically just a main dependency you need to add and then you can use it. There's not a lot of code that you need to detect an image. Some very simple, cheap trick to detect it in multiple sizes. We could do something more sophisticated, but this works surprisingly well. So you look for the image in the original size. If you don't find it, you scale it up by a factor, search again, scale up more, and then you do the same scaling down. And this is a very naive approach, but it works perfectly well. So in many, many cases, you don't have to come up with anything more sophisticated. There are other frameworks you could use. There's something called Image Magic that can do similar things. And there are some frameworks who can do a size invariant template matching images on your screen out of the box. So this is more of a proof of concept. So let's run this. We actually run it before because the stack overflow features for Android ran all the tests. Let's see on this from here. So this is the scenario. Cucumball has a very nice log output. Given I'm on the home page, I'm looking for this icon and you can already see found with match quality 0.95 something, 9288, and even the very distorted image, still a match of 75%. So it's probably safe to assume that this is a match. But in almost all real life scenarios, you will have matches well above 90%. And it should also say, yeah, the Facebook icon is not on that page. So basically how it works, like Selenium web driver can take a screenshot of whatever is on your screen. And then I use OpenCV to do image comparison. This is not a very big feature, but I think it's something in certain scenarios that can come in handy. You just have this one element you cannot match any other way and you need to make sure it's present or you need to can also click on it afterwards or interact with it. And instead of using a different framework to achieve this, just a little bit of code you can easily integrate it in your existing Selenium or Appium solution. Do we have questions? Yeah. Sorry, sorry. One and then two. Okay. Do you have any APIs for like OCR support OCR? Not sure about OpenCV. If they support OCR, there are other frameworks that can do it. I don't have a demo for that. But it's, it would be a logical extension of this. Just haven't done it for this demo. I am just going to ask that. This guy can't recognize. Sorry, we are recording, so we need to use the mic for the questions. I got scolded during the break. Is this can recognize the color? It's also configurable. I think by default it's using grayscale. So the question was what I can show. No, I'm sorry. I didn't prepare this part. But the framework is very, very strong. So there are a lot of features. So I looked at the documentation as possible to do. So again, this is more about showing options, showing what could be done. But no, there's no demo for that. Sorry. Next time. Any, any more? Actually, how does the image matching works? Is it like it will match in any point of any part of the screen? Or do we have a control where and say like I want to match the image at this particular position on the screen? Is it possible to add it as a session? Yes, yes to both. So OpenCV tells you exactly on which, where it found the image. So it such as the whole, the whole screen, the whole screen that you give it. It gets a bit tricky when you do the scaling bit because then you have to scale back the coordinates. It returns you, but it's, it's very well possible. So it returns the match, the quality of the match and the coordinates where it was found. And then you can do assertions also on the, on the coordinates and say, I expect this image to be there. It needs to be extended a little bit, but yes, it's possible. So I just got it that it will check that it will resize the image. It will distort the image and they'll compare it with the original screenshot to see that where it is. Okay. So if I want to check the image is not distorted, can I check it like using this technique? Because internally it's resizing 10%, 20% and then distorting and then comparing it. Yeah. We're not resizing the, the image itself, but the screenshot. Okay. So the original image stays the same. Yeah. If I want to see that the UI is not distorted, the icon in the UI is not distorted. Yeah. Can I do it? Then you would need to make sure that you have the icon in the same size as the screen resolution. Yeah. Yeah. So it's possible. It is possible because we, we think it a little bit. Again, the, the algorithm to do the scaling is more for demo purposes than anything else. So in your case, what you need to ensure is you need to control the size, the screen size, and then you have the image in a size that matches that. Then you can check for 100% match. Yeah. Okay. Thank you. Anyone else? Okay. Let's continue. So in another use case, it's very common and I know a lot of companies use, it's called visual regression testing. The idea is to have a baseline. So I defined a carousel login screen on, I think that's Android. Yeah. Looks like this. And every time I encounter the screen, I compare the two images and see where it differs. And you can already see here that is a bit tricky because sometimes it's only a few pixel off and it's already marked as a difference where there shouldn't be a difference. Obviously there is a username. There's a password. Okay. That's, that's clear. But the other differences are very subtle. And then you have to take care of that. You have to have a threshold, how much, if it's just one pixel off, we're still accepted. And then the things you wouldn't consider at first. So for example, obviously you have to ignore the time, the battery and the signal strength of your network. And for some reason the keyboard is different. That's all like these small things that make it a little bit trickier. So for this one, what I want to show, this is the only not pure open source solution. It's called aptly tools. That's a paid third party service for visual regression testing. The reason I chose this one is because it's very simple to integrate, easy to show and give you an idea of what this concept is about. But there are open source tools that do basically the same thing. You can also integrate with the Appium. So let's have a look at this. Martin, one question here. So instead of taking the whole page, can we take the screenshot of username and password session and just come here? Yes, this is also possible. In that way you can get rid of the battery, signal and everything. Yeah. Actually, I can show you right now. This is one approach. The other one is to ignore certain parts of the image. So for example, for this, this is actually the splash screen of the app. I say, okay, this part of the screen I'm not interested in. Please ignore. And aptly tools called aptly tools, eyes has a lot of other ways to do things. I can ignore region is what I did here. You can have define a region where you only want to evaluate if the layout matches and not all the pictures. Some ways to fine tune this. And to be honest, I'm not an expert with aptly tools. It's just again to bring across the general idea. If anyone wants to try it out, I'll let you play with it some more. But the basic idea is you have a baseline and every run you check against that baseline. And when there's differences that you don't expect, you fail the test. So for this one, what I've done, I've been talking about the page object pattern we use for our test framework. And in our case, every page extends a base page. And that's the location where we put this functionality. So this is the template measure we saw in the example before. And then this object is aptly tools, eyes. And so the base page just gets a method that's called check window. And it does the check and then returns itself. And the reason why it does that is because then you can do something like check. Yeah, for the login. So I just changed the step definition. I call on the home page, page object, I call check window. Then the aptly tools check runs. Then go to login, check again. We come to that in a moment. Then I perform the actual login. And then I check the third time. Once maybe let's run it. And then we see the result. And this is again just a simple change in the configuration file. I want to enable this. Let's just double check. Yeah, and write. So the test should look about the same. Might be small. The only small difference is it takes a little bit longer for the framework to take the screenshot and send to to eyes, aptly tools eyes. Another thing you need to configure is the you get an API for the for the framework. I'm not going to show you that, but it's also just a configuration. So if we look at the log, it should, oh, it's obviously not going to work because it's the wrong example. Let's go again because it's a carousel after all. So we wouldn't be too happy if I don't show any demo with our own app. Sorry, wrong one. So you can see in the log file already, this feature is enabled. So performing check on that class. That's also the key which aptly tools then uses to aggregate the screenshots. What you need to do before we start a feature. There is a cucumber hook. There's some initialization and basically says if aptly tools eyes is enabled, then you have to call open on that object. And as a identifier, I give application name and the platform. So once the test is done, I can just go to my dashboard. Okay, we have a new run for Android green, everything passed. That was the baseline and that's the result we got. So for example, if we, let's remove some of these ignores, then we'll see that this part doesn't match. And here the same. And for this one, we have to, we have to ignore, for example, this banner part because that's some other, that's changing the promotions or different different ads. So the same again, this is demo for Android. I have a demo for web, but I'll spare you guys. This one works again across all, all platforms. And don't get turned off too much by the thought that this is a tool you have to pay for. There are open source alternatives that work the same way. The nice thing is you have this dashboard that already comes out of the box. Otherwise you would need to, to build something like that. Okay, any questions about this before we move to the, to the last bit? Almost there. So we already mentioned before, some use cases, this works very well, but there are some limitations, especially if you look at the last example, the last two, like on carousel, you have all, all listings. I think Amazon search results or stack overflow results. If you search for, for something, it's a bit hard to do a visual testing here because what can you really test because you have to exclude all the images because they're different. You have to exclude all the descriptions because they're different. So in this example, you could probably just check the header, which is not very useful. Okay. So maybe there's something, something better we can do. Well, let's consider a different approach. It's called layout-based testing. Well, that's how I would call it. And there is one framework I just learned about very recently. It's called Galen, which happens to have Java version. You can, you guys can look it up. And very nicely integrates with Selenium and Appium. Last bit I'm going to show. So if you have Galen, you have a specification of your test case. We'll need to use an additive syntax highlighting. So it's a bit similar to how we defined the, um, locators previously for the, for the test framework. Galen, they're called objects. So I said, this is my username field is my password field. And I want to identify it by an X path. Maybe now you get an idea why it's a bit slow on Android. And then you can define how you expect the layer to look like. Again, very simple. I want the username field to be above the password field. I want it to be vertically aligned the same as the password field. I want it to be the same size as the password field. I want it to be the same size as the login button. So for example, let's look at the, at the app once more. And this is Carousel. I'm already logged in. Let's log out. So very simple, this text field, this text field, this login button. And this is the description of how I expect the layer to look like. And if we have a quick look at Galen framework, this is probably the guy it's named after. That's the language guide where you can see what you, and just give me a quick overview. You can have a look at that yourself later on. Basically, you can define that certain elements have to be near to each other. They, you can define that by pixel. You can define that percentage below, above, left, right, inside. Like a button has to be inside a specific container, has to be centered or positioned a certain way inside a container. This is all things that couldn't be easily achieved with other forms of visual testing. And you can also do some things that, here we have the alignment example. You can also do some things that you would usually use Selenium for where you check the text. So there's a lot to automate with Galen alone, even if you don't want to use other features of Selenium. So let me show you how it integrates. And again, pretty simple. So the base page gets another method. This one is called check layout. I could probably be refactored a little bit to be even nicer. But basically you find the spec file, which by naming convention, I put in the same package with the same name as the Java class dot spec. And then call Galen check layout on that. And Galen just needs the web driver and the spec file. And after the feature is completed, I'll aggregate all the results and put them into report. So let's run this one. Disable the other one. It doesn't take so long. Don't worry about the exceptions known back about taking screenshots, which I probably have to file with the guys developers of Galen or maybe find a workaround. But everything else works. So test again is successful. And now we should have a report somewhere as nice HTML report tells us exactly which checks have been performed on which fields. And let's try to to break something. So you see that as well. And let's run it again. So now we say username field should be should be visible below the password field, which obviously doesn't make sense. And we say the width should be 90%. And we change the text here as well. Let's run it one last time. And let's hope we get some errors. As we see three of the tests failed. So the only thing that's not working yet, as I said, I have to figure that out, usually would have a screenshot that tells you exactly things don't don't match. And even this report is quite helpful. And you can do a lot of tests that are hard to do with other frameworks. And functional tests, what usually do with Selenium and Appium only go so far. If your layout is screwed up, can make your app or your web application unusable for for human. And Selenium might still be able to access it. Even though it looks crazy because all the elements are there. It's also important that they're at the right place. This is why where Galen comes in handy. Now we have tested with only login page spec for one device, right? It's the same format, it will not be same for the another device, if the resolution may change. So is it possible to manage the different spec for the different devices execution? Like in case of parallel, is it possible to manage? So the question was different specs for different platforms or for different. Yeah, it's possible. Again, for demo purposes, we don't don't have that. You need to, I think Galen supports that out of the box within one spec file even. Wouldn't need to look up the details. But it's a very common use case. So they should cover that. Again, for different platforms. Now is a simple approach, just different files and different packages. But it's the same. We have the question with the JSON file for the locators. Same here. You can combine it into a single file. I'm pretty sure Galen would support that. To be honest, I need to look that up myself. Martin went question here. So let's say we baseline the image with iPhone 5s. And if you're running our test against iPhone X, since there will be a difference in the screen resolution, will the test fail or pass? Depends what specs you defined. Specs that check the distance between objects by pixel, it will probably fail. So you need to have different specs for different devices. Just simple test like we have here that one element is above or below another one, or they all the same size or they all aligned the same. That should not depend on the screen resolution. I see actually we specify the object identification for Galen. So can you go to Galen spec? Yeah. So should we specify the objects specifically for Galen or can we reuse the ones we defined for the page objects? It's also a good question. If we decide to build that in to our framework, we need to discuss that here at Carousel. Then this is definitely something I want to look into how we can reuse it because now you're defining it twice. One way could be that you define them in one place and then you have your framework generate the Galen specs from a JSON or from property files or from whatever we use. There is no out of the box solution because the way we do the locators in our framework is our own solution. So Galen obviously doesn't support that out of the box, but it's easy to build something that can generate the Galen specs from what we have. It's also for for later stage. If we would want to make this production ready, it's definitely something we need to do. Yeah, I have one more question. So here Galen, what information will take from APM comments? I mean, what we need to ask the parameters actually, apart from the spec internally, how it will. So what do you need to pass from Galen to APM? Yeah. So basically the Galen comment the spec, right? How it will depend on the APM? Galen, you pass Galen the web driver that you use with APM and you pass the spec. These are the two things it needs. It just uses the same web driver used for the rest of your tests. And this way, the web driver has the capability sets or which device to test against and all these things. And that's that's apparently all that Galen needs. And the spec you pass the location to the spec. Okay. Same information we can see from the XML when we do inspect the elements as per the device, right? In the XMLs, we can see the all the spec information. So I thought maybe Galen also depending on the same XML file. When we try to capture the object properties from the Android or iOS, they have their own APM also have the inspector tool. So when we do inspect the element, we can also see the same properties, how much the width it has. But we don't see the information internally clearly the how much width it is, the gap between the one component to another component. So they just wanted to see maybe offline session we can discuss on this. Okay. I'm not sure I understand the question completely. So is it about where to get the locators from? Correct. So how this Galen will talk to the each and every object property and get the values or information from that object? Galen talks using the web driver. Where we can we can discuss the details afterwards. Any other questions? Just building on top of this. Is there any way of integrating the results that give chance out into your normal test? Because if you see your test actually passed, I'm pretty sure you was backing on the login test and running this side by side. So these failed and that basically passed. So is there any way of integrating the test reports together? Yes, there is. Actually, the Galen demo makes the test fail if the layout doesn't match and just disable that. If you just want to pass or fail, that's quite straightforward. If you want to integrate the results, that might be a little bit more difficult if you want to know within the JUnit test what exactly failed. But if it's just what a layout is good or not, that's a standard functionality. Okay. So I think, so maybe just two links to share in the end. So this one everything I've shown. I put a little proof of concept demo on GitHub. Very stupid name. Yet another Selenium wrapper, as if there are not enough. So it's mainly for demo. But if anyone wants to try to use it, contribute to it, we're probably going to use that internally for Carousel. If anyone else wants to contribute, please feel free, have a look at the demos, see if there's anything that could help you in any way. So it's all on GitHub. The first part of the presentation that's already on Master Branch and released, you can just integrate it with Maven. It's on Maven Central. The other bits with the visual testing that's on the branch, you should also be able to find it. Because it's not that state. If you have any questions about it, just have my name cards with me. Just contact me. I'm glad to help. And finally, I already mentioned before the break, Jam has set up a Slack channel for our Appium Meetup where we will announce the next meetups. And we can also use this for other discussion to stay in touch with each other and really try to build a QA and testing community in Singapore. So thanks all for coming. I hope it was, you've heard something that is useful for you. Thank you. Hey, thanks Martin. I think I have learned a lot of new things today. I think you guys also learned. I hope so, right? How are you? Cool. So a special thanks to Carousel for sponsoring if the food done when you. And Satish, thanks for recording the video. You know what? The first part of the session is already on YouTube. And I will share in the Slack channel the link to that one. So it's been uploaded to engineers.sg. So you can have a visit and you can find the video there. So just let us know if you would like to, if you want to take a presentation or if you want to share about your experience with APM or mobile testing. So we can sponsor you the venue to talk about that. And if you want to conduct the meetup in your office space, then that also possible. Just let us know whether your organization is okay to go with medium at your place. So once again, thanks everyone. So just one final thought before I forget. I also want to say thanks for, yeah, for Carousel for having us here and for everyone who organized it, Charlotte, Shyam for the help and Satish for all the technical setup and support and get to know to have someone where you don't have to worry about tech failing on you. So appreciate that everything went so smoothly. Thanks a lot. And a special thanks to Rifa. Rifa just, Rifa and her team flew from Jakarta to Singapore to just attend this meetup. She's the organizer of Indonesia Software Quality Assurance Forum. So that's quite active forums in Indonesia. Thank you, Rifa, for joining us. Yeah. Then maybe.