 As I was already introduced, my name is Patrick Höhlen. I'm presenting here today as a member of the Navit project. And we mostly do geo-spatial, more in application case for GPS navigation. And today, I will talk about how we do our compilation and testing at the Navit project. For the outline, at first I will give a brief intro about what Navit is, in case people do not know. And then I will dive into more details, build infrastructure, continuous integration, and how we do our testing using a device farm and Appium. And at the end, I will give some brief outlook on future work and conclude my presentation. For the introduction, at first, what is Navit? Navit is an open source GPS app. So you can think of it a bit like an open source Google Maps. But unlike Google Maps, which is such just an Android app, it runs on many different platforms. And I will also show you quite a couple of those in one of the future slides. We are already pretty long in the market. Our first commit dates more than 10 years ago. And we use mostly open street map data. We could also use some other map sources. We still have that in the code. But the default case is just using open street map data. For the user, from the user perspective, the advantage is we keep the privacy because it's just offline navigation. We support many platforms. We don't have many or very high system requirements. And what for me also a real benefit is it's highly customizable Navit. You can change how your map is drawn on the screen. You can also change also different screen elements. And you can also define your own routing profiles. And if you have any other requirement or need, you can come to us and together we can also modify the code in order to fulfill specific needs which we as developers did not think about yet. And for the text to speech part, we also support different backends depending on the platform. For example, in Android, we, for example, use a standard Android text to speech system. But we could also use Festival or eSpeak. And it's also highly international. So we translate it to many different languages. Here is a picture of quite a few different devices where we run on. We run on ordinary desktop PCs or laptops which run Linux, Windows, or MacOS, but also a lot of more handheld devices where we currently support Android, Salesforce, those two ports I'm also actively using. And also some people use it still on Windows CE, despite that is, as you know, rather old. And on the embedded part, there we have an active development for TomTom, which is currently broken. And there's also a port for the Raspberry Pi which you see here in the middle with the touch screen. Now, how does it look like when you're on the road? Here you see our default, still up to date default, OSD layout, which is more or less everything here on the outside, where you can add different items and here's just one example. But as I said, you can modify that as you want with also different OSD on-screen display items, which we have and which I also did for my own and for the map layout. Here you can change, for example, that blue color to maybe to be more compliant with RoboSat Pink to a nice pink. And that's more or less how you would see it. If you now would touch it here, then you would get this menu in which you can do all the settings. So I think from the user experience, it's pretty OK. So now that you know a bit what it is doing, I want to talk a bit about our built infrastructure. For that, I think that's usually a problem for any kind of open source project. Do I host it myself? Do I go for something in the cloud? And when one thinks a bit about it, usually developers don't want to make that much about the infrastructure. They want to code and make new features, resolve bugs, but the infrastructure is something that at least at our project, people are not that much curious about or don't want to put that much time in. And that's why also one can use their hosted services, which we are also doing. If you are not an open source project, usually that's rather tricky because those services also cost you money. In our case, as an open source project and probably most projects here, the situation is a bit better because there are also free tiers for different online continuous integration systems, for example, Travis CI or Circle CI, and quite a few more, which I probably cannot remember right now, also offer for free and open source software projects there are some free services. And with these, you can not only do the building of the artifacts, but also compilation tests to see if any new commit broke your code base, you can run platform specific tests, which we also partially do already, and then you see if trust that built for a specific platform was broken. We recently had it said a new change was trust breaking the Windows 3 platform, and then one had to look into it and resolve it. But we also run as a static code analysis using Coverti in order to also get our coding style consistent and also don't have any potential problems. And of course, one can also chain all together and that's what we are actually doing. And here I already have some screenshots. In our case, we use Circle CI, and those screenshots are also from Circle CI. And the nice thing about the system is like I would say most continuous integration systems, you can also run jobs in parallel, depending on how many workers are available. And then you see, like for example, here the Android platform for some reason failed, but TomTom, Windows 32, and Linux are still running perfectly fine to build at least. So one needs to see what is their platform specific which causes some problem. Or what one can also do in such a built pipeline. One can just say that specific jobs are only run in under certain conditions. For example, here, our run-oxygen shop would just run if we commit to trunk because it would not make much sense on a development branch just to run-oxygen. And you can also chain it up nicely that you put different jobs sequentially. And then also this kind of CI helps that you see, OK, where actually was something broken. And what do you need to fix? And here, just for the notation, those are jobs which are still running. Those were the red ones as a failed ones, and the green ones as a one which were running perfectly. So you can also make it in a way, change them, not just put them in parallel, but also sequentially. And then depending on the platform, you get different kind of artifacts. The artifact which I have shown here would be an artifact where you have an image which you can then install on a Raspberry Pi and run Navit directly from that image. But of course, for different platforms, you have different artifacts. If you have a Linux-based system, like also Sailfish or FF, then probably you have an RPM package file or in Windows and XE executable. And for other systems, also other files. And we can use the same process, not only for building, but also for signing and for an automatic uploading, which we are also doing uploading, I think, to the beta channel of the Play Store is going automatically. For the continuous integration as such, as I already mentioned, we use CircleCI, which also nicely integrates with GitHub. And what is beneficial for us and I think also for us is that it also allows not only testing of the main branch, but also full requests. So just by that, you can work on your pull request, improve it, see what it breaks, and also then do an easy reviewing and see, OK, my pull request would not break anything when I merged into trunk. And the other thing, which is nice about it, you can also then, if you want to develop a new feature, you just do a branch for that specific feature. And develop it there, and then merge it and start off with a new branch. So I think it makes developing a bit more in a clean procedure. For when I talk now about what I talk now about everything, it's just more on does it start, does it build, but nothing about the user experience as such. And nothing that much device specific. And this is worth a point. Device Farm comes into play. At Device Farm, you can imagine as a computer with many different mobile phones connected to it. And there you can do manual or automatic testing. And that's for us a pretty large problem because as an Android app among us, of course, we have a large number of devices our app is deployed to. And depending on which device, there could be some issues. And the second thing is also we, on Android, we have a pretty large fragmentation of versions. So also we somehow need to see how we can test an app and all those different versions. And maybe also do a check up if some user comes to us and says, I found a bug. It's not working on my device. So we need a way how to help him and also do the testing. And if one wants to also make a part for iOS, which we don't have currently, then development and testing is pretty much changed to having a Mac or a MacBook. And right now, I think amongst the developers, we don't have any, but also make our life a bit more difficult in that sense. For the device farms, there are quite a few device farms. Such services are already out there in the cloud. And as I said, it provides you an online interface where you can select the phone and also say which tests should be run on them. So you see, as I mentioned, if it's broken on a specific phone, which we have in our test set, but you can also run GUI-based tests both in manual way and also in automated way. About the automated way, I will talk a little bit later about. And what we are not doing yet, but what we are planning for the future, we can also evaluate the performance of, for example, our routing algorithm on different phones. Because with the routing algorithm, also it's a bit dependent on how much memory we have in a phone. So that could also give the user some hints. Look, you have a limited memory, so maybe you should expect some performance degradation. For the one device farm that I have been briefly looking into, I have listed here a few. The ones with which we did our initial tests was the Amazon device farm, AWS web services. And what we also looked a bit into was a Google test and for Xamarin Cloud, but we didn't do anything detailed. And out of those seven, one device farm is a bit outstanding. I really would like to mention it here. That's the OpenSTS device farm, because that's also an open source project, which you can just run on your own server based on Node.js and gives you the option to also connect different phones. It has a web interface as well, so you can also let others test on the phones connected to your computer from somewhere outside. And that's also something we look into for the future. And when we do that GI testing, I know there are different testing frameworks out there. For my purposes, I looked at Appium for you as a brief intro. This is you can do GI testing on a number of different platforms, Android, iOS, Windows Mobile. I know it's a bit outdated. I think they also support Windows Phone, but they need to check out. And due to its heritage from Selenium, it also allows to do web-based tests of web-based apps. What was pretty attractive for me was also that you can write the clients in many different, or your code for the testing, your tests in many different languages. I just named here a few. And I will send you into sample code. I will show you Python. As I said, it's open source and compatible with the AWS device form. And there are also some, there's also already some sample code for OpenSTF, but I think that's a pretty initial stage. Anyhow, we would look into that a bit later in our project. Here, I did an initial implementation on the AWS device form, because we had some three minutes. And I also tested in a local device. And here, as a first sample test, I show the startup and initialization of the app after it was just installed on that particular phone. I think that was a Samsung S5, if I'm correct. Because the Samsung also wanted that one selects the text-to-speech system for the app. And here, I also show the initial steps to download a map. Here, this is how my code looks in Python, where I at first start my app with this first command. Then I select the Google text-to-speech system here. I click that element. And then, again, I am going through the menu of Navit and select the Europe. And basically, then I go down and select the maps of the other islands, because that's a pretty small area. And for testing, I did not want to create such a huge traffic. Now, I would like to show you the video of how it looks like. You need to go out as a message. Thank you. So if we start here, apparently that phone was not up to date. So some update message for some reason came. Here, our app is starting. We select the Google text-to-speech system. We close that initial messages. Now we open the menu and select our map we want to download. And then here, in such example, it stops already before it's completely downloading, and then just leaves the app. And if the AWS Cloud wasn't also possible to take the videos, and maybe there could be also some interesting question about going more to machine learning or OpenCV to actually detect if something was not working out perfectly here as we intended it. So that future development, we have not reached that point yet. But if any one of you has experience there, we are also very welcome to code contributions. And we would also love if people help us improve the app. So you can also discuss with me after that talk. Just feel free. And just to conclude my talk and talk a little bit about future work, as I showed you, we were building an Android APK package and also a Raspberry Pi image using CircleCI. We distributed via Google Play and Asteroid. We did some testing of the compile phase. I showed you a brief example of GI-based testing using Appium, which was run on the AWS device farm. And now we would like to work towards our own device farm, which will be a little bit on top of what OpenSDF does because OpenSDF is just one instance. And we probably would have at every developer side a couple of devices for testing. So we will need to add a layer on top of it. And that will be, I think, also a pretty interesting adventure. In one slide, we have reached our destination. And now here you find all our different contact details. But as I said, feel also free to talk to me after the presentation. And with that, I would like to thank for the attention. And open the floor for questions.