 Welcome everybody to this session. I'm Victor from DSDK, and the Android SDK team, and my colleagues, Luis and Marcos, that we'll present later. So, yeah, in this session, we are going to talk about the Android SDK. So we will start with an overview about the Android SDK to know what the SDK is and what it does. Then the presentation will become more technical and we will take a look at the code with some code samples and then some demos. Finally, we will see what is next, what is going to be next in the SDK. And we would like to have like 10 or 15 minutes at the end just for questions. So if you have any questions during the presentation, just write in the community channel or here. Okay, so let's start with what the SDK is. So the SDK is a common good from the University of Oslo. It's a library that facilitates the development of the Android application. It targets all the functionality that is common for the Android applications and mainly the capability to work offline and to communicate with the DSD2 instances abstracting all that complexity. So if we take a look at the box diagram, here we have the DSD2 API, the server. The SDK will be here. So the SDK interacts with the web API and with the Android with the DSD2 server. And the Android application, in this case the official Android application will be on top of the SDK. So the application only interacts with the SDK and is the SDK who interacts with the API. All these things are a common good from the University of Oslo and it's available for you to use it. So you can have your own custom applications on top of the SDK and the same level that the official application if you want. So let's see the timeline of the Android SDK. The Android SDK is quite a recent product. The first release was in December last year and we have had two more major versions and in the meanwhile several back fixing versions. So the current one is 1.2.1. The next one is planned for early October aligned with the next version of the Android official application and with the release of the 235 because this new SDK will be compatible with that. So let's go to what it does. From the very beginning one of the things that the SDK tries to solve was the capability to work offline. So the first thing that it does is the metadata synchronization. So to have the metadata in the device to be fully operational when offline. In the same way the SDK does the data synchronization. So the SDK downloads the data to the device to be available to work offline and when the end users, the data collectors collect the information the SDK puts all the data back to the server. So these downloads can be parameterized with different parameters and now this is New DGR and we will talk about that later in the presentation. We have a new Android settings web app in the server that is available in the web to define some of these parameters. So we have all the information in the device available for us to work fully offline. So the next layer that the SDK offers is a data access layer. It's a way to present to the developer the information in a very easy and handy way like this one. Also the SDK, one of the most important things that it does is to support and handle the complexity of dealing with different DS2 versions. So the SDK at least guarantees the compatibility with the current version of the two previous ones but we will try to do our best to keep supporting all versions if possible. So here you can see that for example the 1.2 is compatible with the latest six versions that support. Because of the nature of working offline and having a device that is not in constant sync with the server it could happen that there are changes in the server that causes some errors in our data when we are not loading or posting. So the SDK also handles the error management to give clues about how to solve the errors that are produced in the synchronization of the data. Integrate the check because we don't load everything online search. We have said that we mainly work offline but in some cases it is interesting to have a view of for example of the TI's in the server or any other thing. So it is interesting to be able to search online. Also we are working fully offline and there is this property called unique attributes and the SDK guarantees that these attributes are unique and can be assigned. And also handling with the changes in the model in DHS2 and also in the SDK will change with different versions. So the SDK handles the complexity of moving from one model to the other. So this is all this functionality was included in the first version of the SDK. So for those of you who attended this same session in the last year we have introduced new features like SMS synchronization and was also very demanded by the community and now there is support for synchronizing target data, aggregated data and it is very interesting that it uses a compression library which is shared with the backend. So the number of SMS required to synchronize the data and the size of the SMS are optimized. The Android settings app is new this year and Luis will do a demo for this later. Encryption as well, the capability to encrypt the local database in the device as well. This is very interesting the now the SDK uses a shared expression parser and is shared with the backend. So now there is a separate product a separate library that is used by the backend and by the mobile SDK to parse the expression for the program indicators, validation rules and also program rules. So this is really great because now we will expect the same result when executing a program indicator or a validation rule in the backend and in the mobile because in the past there were some inconsistencies between those two words so now this is a really cool thing that we will continue in this version. And also utility classes this is an ongoing thing and there are operations that are pure VSS2 logic like for example decide if a user can have an event in a particular enrollment so you have to take care of the access permissions and all that are implemented in the SDK this is an ongoing thing. So as a summary the SDK is something to facilitate the development of the Android application is a product for Android developers not for end users is written in Java although we are moving to Kotlin Kotlin you know that is the first class language in Android but anyway it's compatible with Java and Kotlin the SDK is compatible with Kotlin the SDK has an internal database and SQL right and the SDK is the result also of group effort from the SDK team and the backend team so together we will try to optimize optimize the calls so everything is working as efficient as possible so this is the overview about what the SDK is and what it does now we are becoming more technical and Marcos are you there? here? I will stop presenting okay can you see it right? okay hello everyone this is Marcos from the DHS to Android SDK team and now I'm going to introduce you a bit more on the SDK let's try to understand what will be a typical workflow using the SDK we have to do is instantiate the D2 which is the entry point from the D2 instance we will access all the functionality that the SDK offers so once we have instantiated the D2 we can take all the functionality we can log in and then we can sync the metadata we can sync the data we can start working we can collect all the data that we need and after that we can even upload the data and so the flow could be repeated if you need to sync the metadata and so on so okay then how is the SDK composed now I'm going to present you the SDK it provides a bunch of classes that uses the access and operation with the database but also with the API so it will hide the developer and the complexity of accessing and syncing and so on synchronizing with the API and the developer will only work with the D2 instance, the modules which is the public part that you can see here so what's the first thing that a developer will do well he will instantiate D2 because D2 is the entry point to interact with the SDK and for this instantiation we only are going to need the Android context but we also can add some optional configuration parameters as the app name or the app version okay directly from D2 it's possible to access the modules and they are used as a wrapper for all the related functionality that we have and it contains the repositories and there is one repository for its entity type and they go from the run repository to the track entity instance repository and so on but we also offer some services as the period helper for example here you can find a list of some of the different modules the SDK provides so you can see here the enrollment module, event, program relationship and so many more and inside these modules we find the repositories if you ask yourself what's a repository well it's a half a shade for the database so you can use it to read metadata from the database or to read data from the database you also can write some data thanks to the repositories and the repositories offer you a builder composition with compile time validation what is this builder composition well if you want to access the database if you write an SQL sentence and make a mistake in a fail 9 for example you probably won't know until run time but with this kind of composition it is less likely to make a mistake and you can detect it when you are writing the code and it has also a similar syntax to the Wave API so you can find some filters, nested fields and so on if you still prefer to perform a SQL query the SDK will let you do it here you can find an example of the TrackEntity module inside the TrackEntity module you have a lot of repositories like the TrackEntity types, TrackEntity instances but also some services at the end you can find the reserved value manager so that's all wrap in this TrackEntity module now we are going to get into the code with some samples here first of all we need to configure the SDK and the only thing we need to do is to pass the Android context pass some optional parameters as I said before you can check them and the documentation but with that you will create the D2 configuration and after that the only thing you have to do is to call the instantiate method in the D2 manager so with that you have the D2 instance and you can start working with the SDK as you can see here in the blocking way if you do it like that you are going to stop your thread and I would recommend you to use a directive way the SDK provides both but I would recommend this second one because you can subscribe to the instantiation of the D2 and once it's finished you can take the D2 instance and start working with the SDK after the instantiation you can follow the typical workflow I saw you before so we could continue by logging in if you want to log in you can go to the user module and then you can find a method which is logging in and you can pass the username, the password and the server you are at after logging you maybe want to download all your metadata so you can go to the metadata module and just download it with the method blocking the load and after that you want to download your data and for that we have the repositories so in this example we are going to work with the event module and the event repository so once you have the instance you can just call the event module enter in the event repository and if you want to fetch all the data that is in the database you will just call the get method and you will retrieve all the events but also you can just wanted them page and you can call the get page method and you will retrieve all the data that is in the database and it will show you the data inside a recycler view so it will be easier for you to show the data also you can count the events and you can also add some filtering so in this sample we are getting the events so we are getting all the events in your database so if you want to filter them you can use for example this filter by the organization unit or even say something like give me all the events with the event date after this date also for filtering we develop different types of operators as you can see here we have different operators like generic equal not equal in non-in it's not null booleans, strings numbers like smaller than or greater than dates like before after in period which is an operator which accepts the period scheme that we provide on the period repository in order to filter dates inside this period and order if you want to order for example by event date you can pass them an enum with the ascending code you can change the order also only fields storing the entity table are written by default so the events won't have any track entity data value but if you want to have nested some of the track entity data values you can add it with this method and also the repositories offer upload capabilities so after filtering you can upload just the data you want okay and now data creation in this example we use the event module again and we add a new event that we just created and we pass it in this projection and enrollment the program states the order you need and the attribute combo UIDs and with that we create an event in the database if we want to make some changes in this event we can access again in the event repository called the UID method and then set the status to complete and what about values yeah it's just something like that so we can enter the repository of the track entity data values and go to the value entity UID data element ID and you can set the value or you can get, you can retrieve it delete or as the SDK if it exists okay that's all from me and now my colleague Luis is going to carry on with the presentation okay hello everyone this is Luis from the Android SDK team I will start sharing my screen as well can you see it I guess yes yeah so I'm going to introduce you the Skeleton app the Skeleton app is an Android app which uses the SDK and it has mainly two goals the first one is to surface an entry point for developers to develop an Android app from scratch and the second one is to show how the SDK is used and all its features it's available on GitHub and it has two relevant branches the branch master which is a basic project showing how to configure and instantiate the SDK on the screen and emit the data and the data synchronization buttons and also the use cases branch which is a more advanced project with programs very simple data entry forums, data sets and more examples of how the SDK can be used if you open it on GitHub you can see the list of features for each branch okay so I'm going to do a short demo this is how the Skeleton app looks like I've already logged in and synchronized the metadata and data so as you can see we have a sync metadata and sync data buttons on the bottom and at the top we can see a short summary of the metadata and data if I click here on the left menu I can see for example the list of programs you can see them so I'm going to show you how we can make some small changes on the SDK or on the Skeleton app using the SDK to change what we can see on the screen so I'm going to show you this is Andrew's studio this is the program activities of the Java class that controls what we see in that screen that I just showed you and if I go here to the get the programs method you can see that we are extracting the programs using the SDK using the data access layer that Marco just introduced so you can see here how we get the D2 instance the program module using the programs module we get to the programs repository and what we're actually using to use the programs is getting the programs that belong to the Skeleton organization units and order them by name in Ascendant order we also use pagination to make sure that we don't overflow the page with too many results and memory problems as we are going to apply one of those filters that Marco introduced so I'm going to also apply a by name filter with the text G so if I run the application again so launch it again and you can see that it was filtered by name so I will not see the rest of the programs only the child program and I can also comment this filter to show you how we can also filter by program type as you can see it's really comfortable to do this using the SDK in other studio because it will always show me all the possibilities that we have all the methods and all the possibilities so for example here I have a program type which is an enum so it will be able to autocomplete without registration I lined again the application and you will see that only the programs without registration will be shown I can also change for example the order so instead of Ascendant I will set the standard until again and you see that we were starting with the A and now it will be starting with X so also I also want to show you here on the menu how we search for tracking the instances so we have also a screen where we can search for tracking the instances by attribute by program I by entering a text so I will select as an attribute first name program I will select the child program and as a text I will enter fra so as you can see it was really fast because it was showing us offline results it only showed us a targeted instance and what I want to show you is how we get that from the SDK so if I go to the tracking the instance search activity which is the Java class that controls the screen and you can see that using the SDK going through the tracking activity module we can access the track entity instance query which is the object of reuse to perform this kind of queries we can have very different filters and what we have here is the offline only property so if I change this by offline first for example and I launch the application again I have to enter the second filters so first name child program fra you will see that it took a while it was not as fast as the other one but it also searched for any other tracking the instances in the server and it shows us two different results the first one is synced because it's offline and the second one it was downloaded from the from the API and to conclude before I go to the next topic I want to show you here some aggregated data so I will go to the dataset instances screen as you can see we have here all the list of dataset instances I can click one see all the data and some filters in the dataset instance activity which is again the Java class that controls the screen here is the place where we get to the data so we access SDK, D2 object dataset module, dataset instances and I'm not applying any filters now but if I change for example I add a filter by dataset UED I can start the application again and you will see that the filter is applied and I can also apply this to filters combined and say okay give me by this dataset UED and by this period that's the application again and as you can see there's only one dataset instance alright so I'm going to go back to the presentation and the next topic I want to talk about is the settings web app that Victor already introduced so it's not an Android app but a web app that you can access from the DHS2 web and this is how it looks like and the settings web app defines at the instance level how Android clients should behave and you can control things like encryption, SMS settings, data syncs basically how much data is synchronized and how often and we also have implemented their single load simulator which can tell how much data would be synced for a given user so we can also help to track problems in the server. Okay so I'm going to talk a little bit about data encryption right now so as we have told you the SDK creates a local database in the Android device and how can users access this database there are basically two methods the first one is programmatically so for that the SDK manages authentication and authorization so data can only be accessed in a secure way but there's other way which is using the file system and for that Android prevents accessing the database except in root devices so why do we need encryption? To prevent unauthorized users extract the database and get the sensitive data from root devices and this was a feature that was really divided by the community so we have implemented the solution which was first introduced in the SDK 1.1 which was used in the app version 210 it's completely transparent from the user and it uses the open source project SQL cipher so if it's encrypted or not it can be set at the DHS instance level by the server admin using the Android settings app as we have introduced so basically either all or none devices from a given instance will be encrypted it's unencrypted by default so in case nobody has configured that in the settings app it's unencrypted and once it's changed in the settings app it will be changed in the devices in the next login or in the next metadata sync without any data loss so it will be completely transparent from the user the database is encrypted with an encryption key just securely generated in the SDK and it's securely stored in the device as well and without performance are there any penalties so in size there's a maximum 10% increase that we have observed which matches the specifications of the SQL cipher and in time there's normally no more than 5 to 10% penalty which also matches more or less the specifications from the open source tool some queries in the app were a bit slower but we could optimize them so in the end there's no penalty observed in a regular use in the app okay so I'm going to go back to the skeleton app and do a demo if I come here you see I'm on the main screen of the skeleton app I'm going to open the settings app so I'm just here in the DHS instance I go here to the web apps I click on settings and the settings and you can see that the database is not encrypted we can also see many other things here on the settings app so basically you have a general panel and some panels to control the program and data sync and the user sync test we've already talked about I'm not going to go through too much into detail but you can see here how you can configure how much data, how often the data is synchronized SMS configuration how many reserve values are downloaded and finally the encryption which is what I'm going to present now so if I go here to the Android file system so basically I'm right now in the file system and you can see here that I'm inside the package of the skeleton app in the directory databases so if I show here what I have you will see that I only have here one database these are temporary files so actually there's only one database which isn't encrypted so I can go here and say OK, SQLite, type the name of the database and say for example and enter an SQL query like select everything from the table user and you see that I can see the data from the from the user just reading the database so I come back here to the setting app I say that I want to increase the database, save it go back to the to the skeleton app and perform metadata sync so you will see that it takes a while but it's not because it's migrating the database but also it's also downloading all the metadata so it takes a minute so basically every time that we once we change this parameter in the settings app in the next synchronization or in the next login the database will be encrypted or unencrypted and it will be completely transparent for the user so the synchronization is done you can see that we have exactly the same number of data data and data values here and if I come back here to the file system I can show the list of databases you see that instead of having an unencrypted database I have an encrypted database so if I try to do the same thing that I did before let's kill it 3 enter the name of the database and I say select everything everything from user the tool will tell me ok this is not the database because it's encrypted so I'm not able to write all the data using the file system ok so I think this was everything from my site and Victor will continue with the presentation right now thank you so hi so we are going to see now two more topics about the Android SDK the first one is the compatibility we are going to talk about compatibility that the SDK is compatible with the latest 3 version but now we are going to see what it means for an application and for a developer so yeah the SDK is compatible with the latest 3 version at least I mean we give full support for the latest 3 but we have basic compatibility the SDK is now still compatible with 229 because there is not a major breaking change so but if a new feature is coming in the SDK it's not compatible with those versions we will probably not introduce them so the SDK also blocks the connection to non-supported versions, prevent errors are the good thing about using the SDK and the changes in the web API is that the changes in the SDK can be detected at compile time instead of at runtime so it can be fixed following the upgrade nodes they can be updated so just now a few examples about what it means for the code for the developer there is a minor change in 229 for example we have the capture coordinates property in the program in class in 230 we have feature type that is an extension of capture coordinates in capture coordinates we have true false now we have a bunch of polygons, type of polygons so here between them there is no mapping so the result of all this is that in the SDK we keep both properties capture coordinates and feature type so the developer can decide which one to use we mark the capture coordinates has depicted and it will be removed in the next or next version of the next one we recommend to use feature type but it's not a breaking change so we try to not break the code of the application when possible this other example is an example of a breaking change if you know in the 230 version the model of the relationship type and relationship change completely so the approach in this case was to force the application to use the new model and the SDK handles the compatibility under the code so let's suppose we have an application and an SDK 1.0 compatible with 229 and 228 230 appears and the SDK 1.1 that is compatible with 230 so in this case we force the application to do some changes in the code to adapt to this new model and they will still be compatible with 229 and 228 so using the new model the SDK guarantees the compatibility with the 3D expression and these links with this slide how to upgrade the applications in the field so let's suppose we have an application 1.0 application in the field with 1000 of users and we want to update the server that is 229 we want to update it to 230 so in this case the first step is to have an application, a new version of the application compatible with 230 and we need the users to upgrade in first place it doesn't matter if this process takes a few weeks or a few months because application is compatible with the previous version so they can still use the application and once all the users in the field have upgraded the application of the app we can move the server to 230 and continue working without interruption and the last topic is about error management because we are working in a fully offline environment we need some extra checks about the data so in the regarding metadata we check that all the metadata makes sense, that there are no inconsistencies in the metadata and what the SDK does is try to continue and store the error for future inspection by administrator and try to continue but keeping metadata integrity always this is regarding metadata about data because we are also in an offline environment the metadata in the server can change and can cause errors in the data what the SDK does is it tries to identify as many details as possible about the conflicts so the application can give some feedback to the user to solve the errors so in aggregated we try to identify the conflicting data values in Tracker the SDK tries to identify if possible down to the Tracker identity attribute or data element level so in some cases this is a screen shot from the official capture app if we have all that information we can give the user some feedback about this is the failing data element this is the cause of the conflict ok so this is all about functionalities what we have in the SDK right now we are talking about the future for its coming what we talk about the utility classes pure VH2 logic about aspiration, validation and all that stuff that is common to VH2 applications we want to increase that capability of this SDK to do those evaluations take the glass support if you are working with protective or closed programs and you are in the official application if you have tried to access data in the search scope maybe you have noticed that there is no support for that something that we should add more features about analysis to have a simple local analytics module to do some calculation about the number of TI's average of specific data element things like that that are very helpful for the users and also the evaluation of the indicators that are embedded in the data entry screen working list from an end user from a community worker perspective is very useful to filter the event or tracking entity instances that are assigned to that user or scheduled for today then support for multi-user, multi-server I think I have not mentioned that but the SDK only supports one user and one server at a time but in some cases the same device is used by several users or maybe the same user needs to use several instances like for example one instance for tracker data another one for aggregated data we want to give support for that more granularity in the metadata synchronization everything that is that is related to the user and offline support because we know that in some cases in environments where the devices are used indoor you know that there is a good connectivity maybe it makes sense to work online and to save and access the data from the server directly and finally we did an UI component for example this organization Dinec3 that is always a nightmare to build these three in the application or value type form so all that things could be structured in a new model that is common for all the applications about learning resources the academies are going to be online of course and we plan to have an Android developer academy in the first quarter of 2021 to be online and we have not the dates yet but the plan is to have it in the first quarter and this is the list of resources that is available for you who want to take a look the source code of the SDK documentation about how to use it from the developer perspective is more or less what we have seen here but much more extended the skeleton app to use it has a starting point for your application Jira to report a bug or a new feature and the community so just to sum up the session if you need to build a new application because the official application does not fit your needs and you need the application to work offline to be compatible with different versions to manage all that complexity of dealing with errors, having good performance and upgrading new versions and much more things that will come in the SDK so I think it is interesting to try the SDK for that kind of application so yeah the recommendation is to if you are in that position just give it a try so I think that's all that's all we have five minutes for questions it seems that we have two questions okay I see one question in the panel and I don't know if there is any question in the community yeah so okay the question in the panel is from Monjur, hi Monjur which chat library was used in Android Captured app or DS2 related Android app which chat library I don't know if we have any chat library in the Android Captured app I don't know if Pablo is there or Jose that could help on this but that's something that is coming but it's not there yet and one is about to speak we just had Pablo and Jose's panelists so they can respond so hi Monjur I think we don't have any chat in the Android app right now so I don't know what you're referring to if you're there maybe you can add some clarification in your question while you're waiting for that there is another question in the community of practice that asks is it possible to use the Android SDK within footer slash dart okay Pablo correct me if I'm wrong but I think that this current is not possible do you use it in Flutter dart actually you can use it in Flutter you can use it in Flutter in Kotlin multi-platform but it will only be compatible with the Android part of the multi-platform project so you can have a Flutter project with iOS and use the SDK for the iOS part of the project you can use it for the Android the SDK is moving also to Kotlin the code so eventually the Kotlin multi-platform evolves and the SDK is within native Kotlin so eventually at some point it could be used but right now only for the Android part of the okay and there's one last question which is added to the community of practice which is how much is the database access speed affected with encryption Luis do you want to yeah so the last of the open source tool that we're using is that it should be or it could be around 10% slower and this was mainly what we saw in our experiments we saw that there were some queries that were a bit slower but we could not optimize them so basically I could say 10% slower and it doesn't really notice it in your user database so I think you don't have more questions right yep that's it okay so yeah it's time so thank you all for attending this session and don't hesitate to tweet to us if you have any doubts or do you want to try the SDK so thanks