 Welcome everyone, thank you for joining this session which is about reaching the last offline mile. It was open to different technologies, we have gotten mostly mobile implementations. And we are going to have, it's a long session, it's two hours. And so, and we don't have many abstracts we wanted to have time for each one of them so we are going to have four presenters. The first three are projects that actually have implemented mobile solutions for different use cases we have a client level, electronic data management project will be the first one. Then we will have an offline data collection system in a field humanitarian hospital in South Sudan will be the second and the third one will be a system for helping in child vaccination uptake in German with mobile solutions. And then we will close with a presentation from to share the experience from training to introduce the presenters now but from Chase. He has been supporting many implementations many mobile implementations, and we thought it was a good wrap up to share key takeaways or challenges and open the discussion for the audience. So, so every session will take 15 to every abstract 15 to 20 minutes and then we will open for questions for each right after. So I think we can go straight with the first one that our presenter is Rachel limo. Let me know if that's not correct Rachel, send your technical advisor and FHI 360. So Rachel, I think I'm going to pass it over to you. I don't know if the screen that we're seeing is already hairs max. Yes. So if you want to search your screen or put in presentation mode and the floor is yours. Thank you. Thank you. Good afternoon, everyone. I hope also is treating everyone so well so far. As I was introduced, my name is Rachel limo. I FHI 360 Liberia, and I am the senior technical advisor for strategic information working specifically with the epic project. So, while I'll be presenting offline, I just want to quickly acknowledge the presence of my FHI colleagues who are attending the conference physically and who are the leaders of the FHI 360 S I work strategic information work who are all behind the work that I'm going to present today. I mean, God lip, who is our side technical director from FHI 360 Washington DC, Kyla, who is also a co-author for this abstract, and also Gordon Parola, who is our side lead for data systems design and development. And we also have in attendance physically a colleague from the Ministry of Health Liberia, Mr Patrick Conlow, and others whom we are also working on the ground to implement the DHS to work. So Rachel one sec. Are there any of these colleagues in the room. Yes. Hello. Maybe they have not joined yet. I'm asking the audience. Are you here. No, okay. We will ask later during the questions. We are open in the session so maybe they come a bit later. Okay, thank you Rachel you can continue. Thank you. I am so honored to be part of this year's DHS to annual conference, and I will be speaking about our DHS to work in the HIV programming here in Liberia. And specifically will be presenting on an abstract titled first time successful implementation of a client level electronic data management system in Liberia HIV program using the DHS to trucker. And I will also be sharing how this implementation is helping us as a country in reaching the last miles of the HIV treatment escape. So, Rachel, can I interrupt your second, do you want to change your view we are seeing your presenter mouths. So if you go to the top bar of the screen or the display settings in your own screen. Yeah. I'm actually just playing. Is it not showing presentation mode yet. It was, but we were seeing your, your, your screen your notes. Do you want to go to this presentation mode. Again, because it's down on the bottom right corner. Yeah, I'm actually doing that. Sorry. Yes, the showing now. Maybe we have a bit of delay. I think it's loading. Yeah, because now I start so. Okay, and now if you go to the top. It says display settings. Okay, just do it. Just present as it comes. I'm using two screens here and both are showing presentation mode. What about that. Not yet. Hello, I think it's loading based on the little delay icon. We are seeing a screen with a slide that says Liberia country profile. Yes, that's what you can take it from there I think it's fine. All right. So, yes, I am currently presenting on the Liberia country profile and for those of you who do not know Liberia is located on the West Coast of the African continent. The population size of about four million six thousand and fifty six hundred and seventy six individuals. We have 15 counties, 94 health districts and 994 facilities. The epic project that is implementing these ph is to trucker walk is a five year US ID and for funded global cooperative agreement, which is dedicated to complement the efforts of the government of Liberia in achieving and maintaining HIV epidemic control. The epic project started in October 2020, and it is being implemented in 21 per supported facilities and Rachel please go ahead but we are going to share our screen, your screen is frozen for some reason. Oh, just just tell me when you want to go to the next slide, and I'll do that for you here. All right. But we need you to stop sharing. Okay. Thank you and sorry about that. No problem. Okay. You can continue Rachel. Thank you. Thank you. So, I was saying that the epic project is being implemented in 21 for supported facilities. And these health facilities, the 21 facilities represent about 70% of the total of HIV who are current on treatment in Liberia. And epic project is also implementing COVID clinical care and vaccine response programs, along with the HIV program. Next slide. So, the diagram that you're seeing represent key data sources that are feeding into the Envision Liberia health information system. And if you look at the bottom side, the last but one is the electronic medical record, which is the data source that the FHR 360 epic project is feeding into with the work that we are doing with the DHS to next slide. So, as indicated earlier, I am going to present an abstract which is titled first time successful implementation of a client level electronic data management system in Liberia HIV program using the DHS to trucker. Next slide. So, by the way of background, it is estimated that 35,000 individuals are living with HIV in Liberia, and only 23,000 are on antiretroviral therapy. This is about 66% of the second 95 per target, which means as a country we are lagging behind for about 29% reaching the second 95. So, in order to close this treatment gap and be able to reach the last mile, we need quality data that will be used to inform decision making and improve service delivery and also improve performance. Before this implementation, before rolling out the DHI to trucker, the 21 health facilities that FHI 360 is supporting, we're still relying on the fragmented paper bus system to collect and report the HIV program data. And these fragmented paper systems were characterized by poor documentation, poor filing and storage systems, which made retrieval clients charts difficult, and at times providers were using random playing papers, notebooks and other documents to prevent client services. And this fragmented paper bus system also led to uncoordinated provider, and sometimes PR led ERT dispensing to clients homes without documentation, or without being registered. And these practices led to poor quality of data, and it was difficult to do any granular data analysis that can inform the program and also help improve the services that were being provided. So with these challenges in September 2021, USID through funding from PEPFAR to FHI 360 EPFIC project in collaboration with the National AIDS Control Program and the Ministry of Health decided to then implement the district health information software, which is the DHI to truck up client level data system to be able to record and report the HIV program data. Next slide. So in our implementation, we use the APIS framework, which is the exploratory preparatory implementation and sustainment framework. During the exploratory stage, we conducted needs assessment, which enabled us to determine the availability of electricity, internet coverage and connectivity, availability of data entry staff, estimating the client load and also doing an assessment of the filing and storage systems. During that stage, we also conducted various engagement meetings with stakeholders from the Ministry of Health here in Liberia. We also did meetings with the National HIV Programme and also with the heads of these 21 health facilities that we were supporting so that we can build awareness on the data quality challenges and brainstorm solutions together and show this buy in support collaboration and ownership. And also at the exploratory stage, we conducted data quality assessment, especially at the fall of the high volume facilities that we were supporting so that we can determine the documentation gaps that will inform our next steps. Next, we moved to the preparatory stage where we recruited data entry clerks, procured generators, solar panels, client files, cabinets for storage of files, we procured desktop computers, Android tablets and internet devices for data entry. We also conducted trainings to data entry clerks on the HIS too, and also developed a data entry plan which classified our data into, our data and facilities into small, medium and high volume. And we then conducted data cleaning exercise in the four high volume facilities where we had to organize and update the fragmented paper reports before we could transfer that reports into the HIS to electronic system. During the implementation stage, the data entry started by transferring the records from the standard paper source document into the HIS system, as you can see on the photo on the right hand side at the top. We can see the data entry clerks there using the yellow client charts to transfer the records into the electronic devices. And after we did enter the data into the electronic system, we then organized files and stored those client charts using unique identifier codes, and then we continued with routine monitoring and support activities to ensure successful implementation. And the last stage was a sustainment stage where currently we are working with the Ministry of Health here in Liberia to support the migration of the data from FHI to the 60B HIS to cloud server to the MOH server. And this migration of the server to MOH server will facilitate ownership and ensure that we have data integration and systems interoperability as stipulated in the country HIS strategic plan as you can see on the right hand side at the bottom. Next slide. After the implementation, after we successfully transferred the records from the paper best records into the electronic systems, and as I highlighted before that we did data quality assessment and then we did data cleaning exercise to organize the paper records. Then after that, we managed to recount and also enter in the DHIS to track a 91% of the data that the project has reported in the period of June 2022 from these 21 health facilities. With the transition from the paper best electronic system, the quality of data improved significantly, especially in the four high volume facilities, which the quality of data before the transition was 58% and when we transition the data to electronic system it improved from 58 to 90%. And I will draw your attention to the right hand side. You will see there's a table there with the four high volume facilities out of the 21 that we are supporting. And you will see there's on the left side, there's the data quality during the period where we were still using the paper best system. And on the right side, you will see the data and the quality of data after we did the data cleaning and transitioned the paper records into the electronic DHIS to track. And you will see how the individual facilities improved before and after and the overall as I mentioned improved from 58% to 90%. And with this improvement, then we learned that fragmented paper best systems were and are a major factor for poor data quality, more especially in the high volume facilities where there's a huge number of clients where you need to collect the information from you need to manage the data, you need to process it and to report it. And we also learned that from this implementation that for the implementation to be successful and sustainable, we require collaboration and coordination among various stakeholders. And also we need to continue providing close monitoring and support activities so that these systems can be sustainable. And as you can see on the right hand side, we have photos there of the data entry plaques, doing the data entry in the electronic devices, but you also see also myself there and a colleague from the National AIDS control program, who will come together to review the client charts and update them before we transfer the quality data into the electronic system. And we also learned that the success and the lessons that we learned from this implementation in the 21 facilities that FHI 360 is supporting is being used by various stakeholders here in Liberia to expand and improve their data systems as well. And currently the National AIDS control program is using the same model and also learning from FHI 360 experience so that they can also use the same to expand to other non-profit supported facilities. We are also working with other organizations and donors like DOD and Plan International to support the data management system so that they can transition their data from paper based to electronic system. Next slide. So as I highlighted before with the transition from paper based to electronic system, our data quality have improved significantly. And with this improvement, we have been able to retain clients on treatment. If you look on the right hand side, you will see there are two charts there showing the continuity of treatment between two quarters Q2 of 2022 and Q2 of 2023. And in this waterfall analysis, if you pay attention to the red bars starting with the top chart, we had extreme losses on our project in this Q2 of 2022 of about 3000 individuals who had interruption in treatment. And we also had 2000 individuals who were transferred out. And most importantly, we had about 3000 plus individuals which were an attributed gain. And this one was an issue of data quality, especially for the attributed gain. And for these losses, it was because records were not properly organized and classified. So after this, then we transferred the records into the electronic system and everything was clear, and we were able now to classify clients properly on their categories of whether active or inactive, whether they were lost to pull up, or if they were transferred out, etc. And you see now with the chart at the bottom, our data for Q2 in 2023, the data have improved significantly. The losses have gone down significantly and also the quality has improved significantly. And this, as I indicated, this quality and this improvement in service delivery is really helping in us managing to reach the last miles in the HIV treatment cascade. Next slide please. Again, with the implementation of the DHIs to Tracker, the client-level DHIs to Tracker in the 21 facilities that we are supporting, we are able to generate various dashboards and be able to make use of data to improve service delivery and also improve the performance and be able to reach the last miles in the treatment cascade. And for these two, you'll be able to see that we are able to track monthly the performance trends for HIV testing by sex and population type. And with these disaggregations by sex and population types, the project is able to perform various granular data analysis and be able to come up with targeted interventions for specific at-risk groups, all of which are helping in reaching the last mile. Next slide please. As a way of, as a way forward, we now continue to facilitate data analysis visualization and use in the DHIs to system that we have implemented so that we can be able to, as I said, perform granular data analysis that will be used for decision making and to help improve the service provision at these health facilities and also be able to improve performance and achieve the per-for 95, 95 targets and also in reaching the last miles. And we are also, as I highlighted before, we are working with the Ministry of Health so that we can be able to migrate the DHIs to server from FHI servers to the MOH server so that we can facilitate ownership and so that the MOH can use that system to integrate it with other health systems and from other partners as well. And we are also continuing to support the Liberian Ministry of Health and other partners in improving their data management systems. As I said, we are working with the National AIDS Control Program and other donors and other implementing partners to support their efforts to transition their data from purpose to electronic systems so that we can ensure quality of data. Next slide. In conclusion, we therefore say that the successful implementation of the DHIs to system that FHI and the EPIC project has implemented in the 21 facilities in Liberia and the significant impact it has had in proving the quality of HIV AIDS data makes it a benchmark for further efforts to expand and improve data management systems in the country and also support in the efforts to reach the last mile in the HIV treatment cascade. Thank you for your attention. I will stop there and hand it over to the organizers. Thank you for listening. Thank you Rachel. So I think we have some time for questions. Is there any question in the audience? We have one question for you. Thank you very much. Thank you for this good presentation. So my question is about the challenges you made during implementation in these 21 facilities so that we can plan for the mitigation plan for the next expansion in the next remaining facilities. That is one and maybe you can share experience about the use of this tool in the remote areas where there is a limited internet connection. Thank you. So as I highlighted in my presentation, before the implementation of the DHIs to electronic system, these 21 facilities that we inherited from the government of Liberia were still relying on fragmented paper based systems. And these fragmented paper based systems were characterized by poor documentation. And the poor documentation was the result of the fact that the filing and storage systems were poor as well. So even when a client will visit a facility retrieval of their client chart was difficult. So providers will just resulted to prescribing medication without documenting anywhere. And at some point it became more worse that they will even prescribe medication to clients without being registered. And when a project inherited these 21 facilities, then we had to do an assessment to determine these gaps like the documentation gap and see to what extent documentation is not being done in these health facilities. And after seeing the gaps, then we had to do a data cleaning exercise. And this one we had to review one chart after the other one unique identifier code after the other and know whether they are active or inactive by pulling from various sources from various pieces of paper from notebooks from dispensing dispensing forms and registers and then gather these most small pieces and updating one standard source document, which is a client chart. And after we managed to pull the information from the various scattered and fragmented paper based documents that were being used were able then to transfer the records that we updated into the electronic system. And to answer your second question about how we are managing with the remote areas of the country where it is difficult to have internet connectivity. Yes, we are experienced that we experiencing that challenge as well. And what we are doing currently is we are using one application blue stack. It's an Android app that you can install it in your computer and be able to enter data offline. And then when you are in a place or when you're at a time where you have internet connectivity, then you can upload your data. Though that application comes with its challenges as well because it is an Android mobile app, it takes a lot of space and sometimes data clocks will report that it becomes slow and data entry becomes slow when you're using blue stack, but that's how we are managing it. Thank you. Thank you Rachel, I think we would love to know more about that last part, but we are going to have to move on to the to the next presenters. I want to say that there are two questions in the chat that we don't have time to take with about the data quality processes and the management of the data entry clerks so I encourage those of you making the questions to contact directly with Rachel through the platform. She is listed as a speaker and she can probably give you details about that. So with that we are going to move to our second abstract of the day. So we are going to have Maria Jose Blanco, health medical officer in MSF Spain, sharing the experience of setting up an offline data collection system and analysis in a field hospital in South Sudan. Thank you. Thank you, Martha. Yes. So hi, my name is Maria Jose I'm working in MSF Spain Metsons and Pontiac as a health medical officer, but also being in the, in the field for many years for being one of the final users of the system for for long time. So, one we are going to present today is the use of mobile data collection offline system in one of the projects that we run, actually. Here. It's okay. So, MSF Spain Spain started using develop a system health information management system in 2014. 2014, adapted from the DHS. And in 2018, a diagnosis exercise was conducted in order to discover a bit the challenges and the issues that the field teams had in relation with data collection and analysis. So, in this exercise, we realized that the main, the main issues faces by the problem was related with data collection as a process that was perfect. It was a repetitive process time consuming spending a lot of time for the people that was in charge to collect this information in the different services. And also prone to human errors are they needed to do the total calculations for different information that should be introduced in the in the analysis first and then in in the system. So, one of the solution that was proposed was the use of offline mobile data collection through tablets in the different services. So the solution was based in the use of the DHS as to Android app for data collection of individual registries, and then custom web app that integrates these records into the system as aggregate data. Additionally, it was a review the use of the existing registry book to understand a bit how it works usually in our projects. In relation to data, the processes start with the introduction of the information in the clinical file, then this information is translated in a registry book in a linear manner manner with the information for each patient. And then in a weekly basis in most of the cases and some of them in a monthly basis is translated into a tally sheet. Then into the system through the data entry. And finally the information is the storage in the in the system. As you can see the first three steps are done in paper inside the the facility. The provided solution. facilitate the data flow of this process. So, is keeping true of the steps until the storage in the system. So we go from the clinical file to the general registry book and then introducing directly the information in a tablet. aggregate the information and storage the data directly in the system. So, we facilitate and make easy this process. This project was piloted and conducted in the mission that we have in South Sudan. This is a long term mission that was established in 2013. Very soon after the civil war breakdown in 2011 and to have an idea about the situation in the country since that moment up to now. We have more than 400,000 people that has been killed. More than 2 million people have fled to other countries and we face more than 1.8 million people that has been displaced inside the country. And it's true that in 2018, 2018 there was some peace agreement that improve the situation, but still there are many people that still language in different in different camps, where the access to health primary health care and secondary health care is still very poor. We need to consider interethnic clashes and other problems in the in the communities. So the situation is still not stable in 100% less, let's say, so it's still adapting every day every week to, to the situation in them in the place. The project that you are seeing here these photos are in Malacal that is the capital of the upper Nile state close to the border with Sudan in the north is part of the country. So we work in the community but also we have activities in two hospitals, one of them the POC inside the camp and also in the Malacal town. The system was established in seven of the services. Nowadays is still running in five. Two of the services were closed related to the operational decisions. So to start the problem, the project. We're going to start with two weeks work in the with trainings in the services and directly with the people that was going to be in charge for data collection that was mainly the nurses supervisors that are local higher staff. And it was these colleagues that have was in charge for for data collection. And after those the there was evaluation to assess how it works the mobile data collection in this in these services. So during the evaluation. We saw that this mobile technology to support data collection was feasible and was well accept for the people working in the in the services and for the person in charge. So one of the main advantages was related with the efficiency as this automatic aggregation of data was saving many, many time for the people that was spending hours every week at the beginning of every week, collecting this information from the different registry books in the in the services. So the data collection process improve. There was a good feedback from the users related with the time and the completeness of data was not changed so much. Also one of the improvements related to the system was the improving the data quality and data data accuracy that was the feedback given by the by the users. So in general, since the moment that has been implemented through the years that we are using the system, we can say that it was introduced without major problems. And that was really well accepted by the users, because we need to think that from a public health perspective, one time is the assessment that you do during the final of the pilot, but then it's also the acceptance and the use through all these years that was very positive. However, to be realistic, we need to take into consideration some challenges that change a bit in nature from the starting of the pilot up to now is true that at the beginning, most of the challenges and the issues were more related with technical problems. And mainly with the remote support, we need to think that we work in places where connectivity is very poor. Sometimes not just connectivity but even the access to electricity. So it's one of the challenges that that we face. So there were some identifications of the synchronization zeros in the Android app. And also anytime that there is an update in the health data model, we need to map in the project indicators and data elements. However, those problems are decreasing through time. And we need to focus also in the problems and challenges related with the, with the context. So the way we work there is a high turnover of managers that work in the project, the one that should support also the, the local staff in these activities. And also, one of the main challenges that we face is the high workload for supervisors, as there is a lot of competing priorities in their daily life and activities to face. Sometimes they don't have time to check to introduce data to exit basis and the patients in them in the system. So, time to time we need to review a bit and reinforce the training. As we were facing problems with exit syndicators and one of the key elements in the key indicators in the use for hospital management that is the bed occupancy rate. So in order to implement, we would like also to implement this project in other services in other places also where the use of these tablets can be can have an add value to the different projects. We are working perhaps also in remote settings where the electricity and connectivity is very poor in the way that they can introduce data in the tablets and then synchronize later on. Also for outreach activities, avoiding the daily work of introducing data in a tally sheet, or even in facilities with a high volume of patients were the process to introduce data and to count. All the different data elements for the tally sheet is really time consuming. Additionally, as having this information enter in an individual way, there is the possibility to do analysis for this individual data and make use of offline analytics. Final conclusions about all these years using the system is that bringing technology to the place where the data is generated is simplified all the process for data collection, and it's well accept by the final users of the system. We can reduce manual steps and these motivate a lot the people that need to collect data, recognizing that data literacy is not so the priority from in some cases related with the clinical priorities that our colleagues is facing the in the field. Some of the challenges that we encounter was the provision of technical support, the hardware management, digital skills of the people using the tablets, the robustness of the app, and the loss of knowledge due to the high tournament staff. However, now we think that we can implement the system in other places and having an app value for the use of data and the use of the adoption of the system for our colleagues in the in the field. This is just one of the sentences from one of the people who participate in the pilot. And just from our side also we need to thanks our colleagues in the field that are in charge to collaborate and and to help us in this process to implement the the pilot. Thank you. If you have any questions. Thank you so much. So about this, this automated mobile data collection which you have just told it's great. And definitely it's a need of the time. But I just wanted to know about that. How are you validating the mechanism of data quality in this because this is something which we are also doing in Pakistan for quite some time, because we get to have our field staff and we turn them as TFS district field service. And many a times the data entry they do not match with a master file, for instance, duplicate entries, and let's say if the TFS is having a bad day, he putting you know male to a female and female to mail so you know these kind of input entries they are expected so they're working all day and day out in on the fields. So how do you really validate the mechanism of data quality and data integrity on this complete system. I really like to know about it. Thank you. Yes, I would like to have the proper answer for this, but trying. Yes, and this is one of the problems that we face not just in this project related to mobile data collection of line, but also in many of the different in all the places where we work. The use that we establish is a validation that is done at the project level is the project medical reference that should review and validate the data, then also at coordination level. It's true that we can also implement some rules, but it's not the case in is always the what we are using in this moment is manual review of the data cross checking what we can see first from from the dashboard that we have compared with the registry books and then also with files. Yeah, thank you for the presentation. Tomas Matas WHO Geneva I have just quick question about the periodicity how often did you enter the data and and for example if it was daily why it was daily not weekly and if you can talk about this thank you. Yes, the data are introduced in real time, let's say, at the moment that when there is a patient that entering the in the service, they start introducing data. And then, at the end of the day, they come with the tablets to the base of the project to charge the tablets, and at the end of the week the synchronization is done, because the analysis of data is done in a weekly basis in this project. Hi. Thanks a lot for your presentation. I just wanted to ask you, did you had many instances where patients were coming over and you couldn't find them in the database already and you had to go back to your paper records to sort of, you know, find the client, or did that not occur very often and did what did you introduce in terms of a process to mitigate for that. Yes, I would like to say no. There is always some basic human part. Yes, the good thing of the system and is one of the things that help us to improve data quality is that if we enter a patient, then the patient is in the tablet so if a certain point we forget to introduce some additional information or information related with the exit of the patient. We can't see because the exit indicators as for example the better capacity rate will increase. So it's one of the ways that we have to detect any problem. So the issue to forget some data is fewer than the one that we have in other projects where everything is based on paper. If we get the next question, I would add that I think it's important in that sense to to think of the purpose of the digitalization. So in this case, for the presentation I see the purpose, and I know the purpose is to generate the routine information at the end of the week, and not the clinical care which happens with the clinical file. So it's like if you can find the patient again great but if not it's not a it's not the worst case scenario. In this case. Hi. So in the process of setting up the the pilot, I want to know what were the criteria for site selection for the project to be scalable in the future. And what's the size of the hospitals you, you are using and the size of the team or the people working with the system how many tablets. What can you elaborate about about that for us to think about scaling it up. For the selection of the site perhaps Martha can tell you a bit more because she was involved in this process. I was involved in the beginning of it. So the selection was as in other pilots we were looking for a place that was challenging but also stable somehow. And this was a long term intervention with it was English speaking it has an at that moment I don't know nowadays you can tell us average volume situations that would stress a little bit the system but not to the limit. So that was the selection from the top of my head, but currently I don't know the situation hardest. Yes for the implementation in other places. There are many considerations to to put on the table, not just in relation to the data or the connectivity, or even digital skills, but also other related with high security context the use of tablets in places where perhaps can put in some of our colleagues. And we work mainly with in this moment with under demand. So there are other projects that are interested in implementing this, but always considering competing priorities for the operational level. And I think that you also ask about the number of tablets. There is one in the each service. We are running five minutes long. Thanks very much, Paul Saunders and I'm a leprosy specialist. Can I ask about the servers so do you have your own DHS to system that is dedicated to this program or are you going into the whole national DHS to program. Because I'm, I'm interested to know whether there are many separate systems in a country like South Sudan that may not have a very coordinated central government. Yes, we use an independent servers and data that are generated here in the projects, not just in South Sudan, but in the different projects and missions where we work are stored under the HMIS health information management system from MSF. So it's not directly shared with the at country level in a daily or in a routine base. But I would say this makes part of the reporting mechanisms that the organization has either with the health cluster or with the MOH they are just getting the information from here. It's not directly connected. But they report like they share the information. That's what it says. Thank you I'm working. My name is George I'm working with his sentence Oslo sorry. I have two questions so on the third slider. I think you had the register I wonder whether you consider digitizing also the register to eliminate another paper based tool. And the second question was that I'm not sure I understood correctly so you use DHS to as a platform, but did you use a native tracker program or did you actually build a customized app. It is linked to DHS to understand that technical part. Thank you. In relation to the digitalization of the registry registry book was proposed and suggest to the project, but final users request to keep having this book as a confident tool for them also. So there is this possibility to directly introduce data directly from the clinical file to the mobile data, but for them is also a way to estimate its number of beds to have their planning. So it's just because they request that it is possible. And perhaps for the technical part I have two. It's a tracker program with a default tracker, a default DHS to up. And it was a tracker program because the users, it was an event program at the beginning, but they wanted to search patients. So that complicated a little bit the data protection aspects, but yes. Max, do we have any question in the chat. Otherwise, I think we move on to the next. Yeah, we don't have. Thank you. So we are going to. Yes. Okay, so we have our. Oh, but he's going to share the screen. Okay, so you can share the screen we have our third presenter, which is that one alpha K from USA GSI it's going to present a project for child uptake of child vaccination at community level with mobile technology. So if you want to share your screen. Did you see my screen. We see it but but not where we should see it one second. Okay. We see your screen. And it's perfect. Okay, thank you. Thank you all. And I'm prepared from Yemen sharp project. We going to present our abstract. We got into the increasing child vaccination uptake in Yemen through mobile, mobile notification system at the community level. And so we will give short brief about the overview in Yemen regarding the situation of engineer. There is also some people from one government to another view to the war, as you know, and also recruiting natural disaster depleting the country health care system and critical infrastructure. As a conflict escalating many families become less likely to the vaccinating their children because of the benefit program, health services are not urgent or tangible as for the system and treatment of acute conditions challenges. According to the last report published by the UNICEF regarding to the vaccination. It's reported that Yemen has the most, the most under vaccinated children of all the countries in the Middle East and North Africa region, almost a third of the population under the age of one year has missed routine vaccination for the preventive disease. They are above 1174,000 zero dose and about 9700 under vaccinated children of one year. This means presenting 80% and 3% of the total population of the under year children. Beside that chart project conduct a study in 2021 and vaccinating rate in adding government where lower, much lower than the other government where we are supporting these governments. Rumors and measures about vaccinating persist lowering confidence in the child, communication. Also, we find that displacement of part of the population are also contributing to missed organization. Regarding to the quality of the health services deliver we found that paper based delay register result in board data collection and quality, leading to insufficiency contact and demographic information about vaccination children. This create difficulties in contacting caregiver for further notification for their vaccination or for the next vaccination of their dose. In addition, the goal was the goal of the pilot study is to test if the short message services reminder message can help to identify children, whom is their vaccination appointment using the DHS to traffic. In October, 2022, the USA chart project initiated a six month pilot project in Elbroga district. In adding government to this if the reminder message sent through mobile device could be affected enriching those children with the organization. Elbroga was selected because for the reason that the vaccination rate was lower comparing to the other district where we are supporting the three governments. The system sent SMS reminders to the parents or to the caregivers, not finding them of the next vaccination appointment states of their children under 18 months. There is two message sent, one message is welcoming at the start of the enrollment and the other message is for the mind of their vaccination. The mobile system or the tracker system chart project dictates the community health information system using Android application DHS to capture as you all know about this application. This is a pilot electronic registry mobile. We customize this in Arabic languages. DHS to capture for the Android devices works with the DHS to instance to capture the individual data related to the vaccination schedules. The IRR or the electronic registry of the organization sending the leakage between the community and the health facility enabling the health facility to identify children who missed their vaccination appointment and share a list with the community midwife to follow up and to track the status of that children who vaccinated. Here we show how the system is configured and we show in the right of the slide our platform using DHS to as well as the DHS to tracker which show three programs is used in our community health information system. We are the community midwife using the reproductive health program as well as a registry as well as the child health. The IRR customizing the DHS to the digital package for the immunization registry with the configuration of the bulk SMS API to activate and schedule the notification SMS system using mobile application DHS to capture. We use this gateway service provider the bulk SMS in order to link the DHS to instance and this in order to enabling sending the notification SMS. The end user training and launching the pilot project. The pilot the project provide the end user training for 23 community midwives and 80 and eight facility organization focal point from that district we will supporting this AR pilot testing. Char also conducting a lunch workshop for the pilot testing involving the central level of the ministry of public health and population. Community organization program as well as the district health office and also we invited the local part of the community outreach. The community midwife and the health facility organization focal point work together to mobilize those families for their next vaccination appointment. Also we conduct a feedback workshop with those health facility focal point and the community midwives there and we they reported that the families were who received this SMS reminders were happy and they are encouraged and also this SMS message motivated them in order to reach the health facility to take for their children that the vaccine. Here we show the results that we found during these pilots the study during the pilot testing period the the pinta three vaccinate up to increase by 41% from 23 eight in November. And in March 2335 in March. If you if you look at the red line in the in the chart at the right, you will find in November that the value was 23 eight, and in March increase it at 335 that's mean the improvement was about 41%. Also we found that there is overlapping between the blue line and the red line, which represent the pinta three and the pinta one, which means that most of the children take who received the pinta one, those are taking the third dose. At the bottom, we are showing the calculation between the pinta three and pinta one and pinta three drop rate, which which is decreased from 42 at October's when the Romans started and decreases to 70% at March. The conclusion and the list of lives. They are application helps to do something that the center led the link between the community and the wife's as well as the between the community with wives and the health facility enabling the health facility to identify the children who must their vaccinating appointment and alert the community with wives. Also, we learned that active participation from the commitment wife and their present in the community plays a crucial role in increasing the child immunization, supplementing by the SMS reminder system to reach the care difference. Even though in the areas where the telecommunication has a problem like the low coverage. The SMS reminder message link with the community health information system enhancing that the enhanced by the DHS to showing promise results by decreasing the drop rate in the children vaccinating program. As a solution to this challenge because if by the busy phone network chart chart flexibility adjusted the time to send the SMS national to the client, this improves the rich by minimizing challenge with the network's problem. Thank you. Thank you very much. We do have time for questions we have any question here in the audience. We have one over there. Thank you. Thank you for the presentation. It's very interesting to see how immunization uptake has increased. I would like to know what is the fraction of population in Yemen, at least in the in the governorates that you have used this system who owns the phone. How many households, just to know what fraction of the people are left out receiving an SMS. Yes, that the due to that to the war and the, you know, the infrastructure what collapse, especially the telecommunication system. We have only one one surface provider for telecommunication. And also the SMS application reminder. It's not that the not all all the the the balance received. There is this 70% of the SMS received to that to that balance. Yes, but but we as a backup we were the community with wife also reaching these, these children who rule and follow up when they went when the system notified the community with wife inside their phone. To reach that that children when when as as a route as a routine. So we have two things to follow up for the vaccination the SMS is the reminder. If the SMS not reach also the community with wife, let you that house in order to inform or to encourage those partners to take the children to that facility. We have another question here in the audience. Yes, well, I kind of a continuation on that question is that what is the reading comprehension level in this country in this governorate. And do you have you tried voice messages, possibly a good question. This this also one one of the idea has been provided to us during the feedback that's we conducted for both the community with wife and the health facility. They also advise us to include the voice message because some of some of the families providing the land, the land phone contacts, not all the families has mobile phone. So it was a challenge to send SMS message to that land for land for so this this one of the idea have been recorded and documented for the for the scale up land. Thank you. I think we don't have more questions here. There is one in the chat. Okay. Oh. So we have three questions in the chat. So I'm going to read the first one, because we have time. How did you obtain the phone numbers of the clients for follow up. Do they provide the numbers were coming to the health facilities. The client provide their phone numbers and it's one of them and then the three attribute during the enrollment. The community with wife or the health facility focal point must feel it. If they if they didn't have that sometime the community live the report or entered her phone numbers in order to notify her. So at some time, as I mentioned, that some families provide land phone contact numbers, which would give us some challenge to provide that SMS. Okay, and the second question is, was this intervention only for follow up on those who missed follow up appointments, for example, not for zero dose child or those who have not made contact with the health facility because it follow ups from the previous at the start point, yes, it was only for these who missed the follow up. But as we make it as a back load, but currently it's going to one continuous for and child who are wronged for for organization, the, it's from the start point. So, as I mentioned, it's in the it started like to follow up for the mystery but now it's for for for general organization. And the last question in the chat. It says that you mentioned a community health information system in your last slide. Could you describe that part of the system of it. Is it based on some other tool. Is it the DHS to capture up. Yes, it's the DHS to capture up which include three programs, reproductive health programs for the community in my wife, as well as child health, and also including the third one with the history for this purpose. We have another question in the chat we don't have any here so I'll keep going with the online participants. This one is for SMS reminder, I would like to understand how Yemen are sending SMS to people missing appointments within the highest to currently I heard that it is through a custom script. Do you have another way. Yes, as I mentioned, we will contract the bulk SMS service provider to provide as a gateway in order to send the message. So we link the this gateway bulk SMS this, this company, it's a global company. We contract it and we provide us the gateway API, and we link it with the GTI stone in order to create these keywords SMS. I'm going to extend a little bit and then we have one comment here. It is a native functionality of the highest to to send the SMS is to the track entity instances. If there is a phone number if it is configured and with a gateway, which is exactly what what is being explained in the presentation. All right, one more one more from the room. I have no idea what kind of capacity or what utilization level you have in in each clinic or health facility. Did this have an impact on on the capacity of the clinics that received more patients after this I understand that vaccinations kind of low effort more or less. And in the time spent sense. But did you have any challenges based on that given given that you have 43% more patients on the load. Do you mean a patient in general or your question as talking about the children and organization. Yeah, did you have more load and how did you handle that the volume in the facility. Actually, we are focusing for the organization children. I think if if I understood the question well is that did the sending reminders with SMS increase the volume of patients in the facility and if so, how did you manage. Actually, actually, they are application capture application. Was helping the health facility to organize the profile of the children. So they are interesting of using this mobile application or the to capture, because as we get it from their feedback. It's easy to search for children profile. Also, they can found that the tractor those of the organization easily. Despite that the way that in the daily register variable. We find that it's more difficult for them to to give the history of that children. So, the mobile application or the DHS tracker was supporting the health facility in organizing and saving their offers and enabling them to effectively monitor the children doses. Okay, thank you. We don't have more questions in the chat neither in the room. So I think thank you very much for your presentation. Thank you. Thank you. If you can stop sharing your screen. We are going to move to our last presentation of the day we are going to change of the session of the day. The perspective we are going to hear now from the person behind all the implementations. And then, yeah, setting up, maintaining, give support, mostly maintaining and supporting. Okay, these days, maintaining and dealing with problems. Okay, so all yours. Yes, how's everyone doing. It's a little sleepy in here after lunch, I think. Okay. Excellent. So, cool. So the title here max CD notes title here is key decisions for mobile data collection. I'm not going to focus solely on DHS to but just general data collection in an offline setting. Thanks. Right so. Yeah, so we're all here today for the common use of, as I said, collecting data with DHS to this session specifically. One more question max. Is this the camera that I'm on because it's at my chest level is that one. I can't read my notes and beyond the camera. Yeah. So anyway we're all here for collecting mobile data in an offline remote location. So this topic is going to look very different for each of you, depending on whether you are from the Ministry of Health, you're backed by a donor funded project, you are a donor, or if you're nonprofit, or a consultant. The concept of these challenges that we're going to discuss today will be completely different than someone else in this room. No, those challenges are not exclusive as I said the DHS to or even the health sector. They're common issues that arise whenever we're implementing technology. We've all run into the issues, but particularly when you're offline, things get more complicated exponentially more complicated. The justice challenging to in these locations when using pen and paper for those that have you have done it, or doing hybrid sorts of things with pen and paper. So we use technology to find an advantage over traditional methods of data collection otherwise we would just stick with the paper registers that we all know and love. So throughout my presentation today, please try and look and think about the advantages, the disadvantages and why we even use the technology in the first place. Is it cost savings is it better data quality. Typically I think data qualities are are like go to was it faster reporting, so on. There's one prominent advantage can also be a disadvantage especially in our context. It's always at the center of technology implementation, and it's typically a major challenge for us. I'm going to be a little bit more interactive today. I know like I said it's a little sleepy in here but does anyone know what typically the biggest constraint is for us in our projects. Fear of tech that that's a pretty good one. Okay, bias that you can't do it. I was thinking money. So just remember that it's always a balance of trade offs. Those are were two great answers as well. I know that technology literacy has increased in the past couple of years decade. Everyone seems to have a smartphone these days. But I was thinking of money and again throughout this presentation, I also wanted to point out that we are living in the future. So by that, I mean that we have very, very real technology now there's change in the landscape of remote data collection. Specifically, I'm thinking about Starlink. Everybody knows Starlink's satellite web of Internet providers providing satellites. It's now available in Nigeria, Wanda, Mozambique, very recently, and much of South America. I think it's going to continue being adopted as soon as regulatory bodies approve of it. That's really the only challenge left. So again, I want to encourage you to think about the trade offs between money, what's possible, what's an advantage and what's a disadvantage. So, I'm not going to have any groundbreaking thoughts or assertations today. I really just want to facilitate a conversation. The goal is to spark that conversation to continue beyond the conference and if we get any good feedback or notes. Is this being recorded, Max? Excellent. I'll try and do a write up on the community of practice so that we can keep this going for a while. Because it's a lot to think about. No one's going to have the same context on a project. I have a clicker. Great. Nope. Yeah. That was, I think I'm there. Yep. I was changing the slides on the Zoom call. Okay, so I'm Chase Freeman, Solutions Engineering Manager at BAO Systems, and it's my job to create sustainable solutions. And my approach in doing that is mostly collaboration and communication with those who can do things very well. I do this internally at BAO for my colleagues. I do it for our clients, and whether it's tracking down DHS to bugs brainstorming effective workarounds until patch comes through or building custom applications and working with teams to implement technology. Typically, it's always DHS to these days. Because that's what we do. So here's what we'll be talking about today. And please be prepared that my slides are very sparse as the titles. That's about as sparse as the titles. But I'm hoping that you will engage with me a little bit and share your experience with some of the scenarios that we'll be talking about. So to start off, I want to pull the room. All you have to do is raise your hand. We'll have some questions so I don't know if you'll be getting a workout today with the microphones. But just for this one, if you please raise your hand. Who is conducted an offline data collection program or campaign with zero to 25 devices. Anybody. Okay, a couple. Anyone with 100 to 200 devices. Okay, we got one. So I'm assuming no one with 500 1000 3000 or more. Okay, cool. Interesting. Clicker. So, I want to take a second before I get into all of this and promote the incredible work that the Android team and DHS to in general has done to document all the software. It's an incredible resource. And I want to point out that you just always go here when you have questions first search documentation search the community practice. The points that I'm raising how they said they're not unique. And in fact, most of them are right here in the documentation. They've been thought about well they've been tested. They're from the field. So always go the documentation. And I want to say also that the Android web settings app and the new troubleshooting workflow. Is that what we call it is very incredible that it's going to do a lot of the heavy lifting that we need to help implement in the field and remote locations. So that screenshot is just from the implementation docs for the Android app. Okay. So my first slide, as I said, quite sparse. I want to talk about scaling. It's an essential factor that impacts all aspects of our project planning and execution and the strategies that work well for small pilot projects may not actually do not translate effectively to larger countrywide implementations. We've got to deal with resource allocation. So large scale projects. We've got to deal with resources. We've got to deal with the human factor technology, and the need to be deployed, or they all need to be deployed strategically to cater to the extensive and diverse needs of all the places that we we go. Whether it's remote. Well, we're going to we're talking pretty much about anywhere offline so consider it remote. Data management, as the scale of a project increases so does the complexity of the data management, the cost implications I talked about money earlier. Large scale implementations obviously are going to come with significant more costs. Risk mitigation potential risks and challenges multiply as well. And lastly, continuity and transition. So with larger projects, we need to ensure that the continuity of services. There's a place for smooth handovers and upgrades and we someone else was talking earlier about actually I think two or all three of the previous presentations talked about human resources and recruiting and hiring and training along the way. Keep forgetting I have the clicker. Okay, I one thing I didn't mention earlier is that I'm not going to talk about SMS or well that I figured that those would be two things that you'd have an entire session on. So we're going to just avoid those topics entirely. So the logistics. I think most of you have seen a road like that. If you've been out in the field. So I want to dive in a bit deeper and look at some of our key challenges. Mostly, I think logistics is the biggest one. So here's a scenario, you're in an area with no internet, you need to upload records from a device. And this may require that that device be transported to a location with internet. There's no magic way to upload something to the internet when there is no connection. So needs to be transported somewhere or bring the internet to it. And to do so you can either do that as the data collector or you can give the device to someone else. So a few things to consider. If someone who needs to transport the device have to leave their post or can it be given to someone. Does the data clerk or collector have another device to collect continue collecting data after that device has been given to them to be uploaded. And then who maintains the custody of the data and the device during this process with aggregate data, you know it's not so not as important to know that but when you're dealing with sensitive data PII. So who has physical control of that data. And then when the data collector gets to the place that has the internet. How is it going to be uploaded are they sharing accounts. Other credentials in order for them to log in and upload the data with their own account. And are they qualified and trained to address any issues that may arise during data upload. specific questions but they happen. So, when addressing these concerns, it calls for a strategic approach. That's going to be a common theme planning here for secure data transport. You got to make sure that the database is encrypted. If the tablets traveling through the countryside, it could end up in many different places that is a native feature in the tries to it can be toggled on and off on the Android devices. Typically always be toggled on, but there are trade offs as I discussed earlier. When you have encryption on the device. It could impact your performance and querying and searching. But again we'll get into that always goes back to money really. It's just a constraint with the device that you have. So anyway, always encrypt the database, you never know where they're going to be. Also, for continuous data collection, as I mentioned, will the person who has to upload the device be traveling somewhere on their own, will they give the device to someone else. How does that impact the data collection process are you going to have someone else with another device to continue collecting or is it going to be a data collection period. Take a week off while you're traveling through the country. And then you do reporting and then you go back and collect more just again food for thought I don't have all the answers here, just scenarios. So does anyone here have experience with something like this where you have to travel to a internet cafe or a hub, or anything like that, handing your data data off to someone else to go upload. George. You give the mobile phone you take it with you give it to the pharmacist they take the stop. And in the evening or next day you give it back to the team they go to an office and synchronize it. Excellent. Sounds pretty efficient so they have two phones. Or sorry they go twice a day you said. Yeah, I mean in this case, you know the stock count is done once a month to leave it overnight and the next day or next evening you take it back. That makes sense then sort of working in in ships. Anyone else have a similar experience. Sorry, I just add another one some because I think for the connectivity. I find it fascinating you mentioned Starlink we need to think out of the box because technology is moving fast. You know, there's also one NGO that has is actually featuring their vehicles with mobile Wi-Fi connected to a satellite which is expensive. But think about it if you had a car with a district health officer passing down this lonely road. Once a week even once a month and every all the clinics could go out on the road and upload the data through the Wi-Fi to the satellite is just one idea maybe coming. Definitely I think that could be I'm not advocating for Starlink here it's just kind of a popular thing that could definitely be achieved using Starlink. Starlink is very expensive though so let's not know it's not going to solve everything it's not going to fit into every project here. But that's that's absolutely right. Another thing that at least we offer a BAO is a remote server so you don't have to have the internet you have a local network. At least upload all your data there. Visualize it in DHIs too. And if you want to connect it to a central server, then it still needs to be transported somehow. That's the that's the setup. That's the setup that MSF has. Oh, yes, yes. I did know that actually with the two servers offline in the projects, not through us, but yes, you guys do that very well. Okay, next slide. About constraints. We've got two main factors that come into play. Software and hardware. That's, well, there's probably a lot of more constraints but this is the two I'm going to talk about. So from a software perspective, the complexity of your program may limit the amount of data that can be collected offline. So how intricate are the data collection forms? How many calculated fields? These are the factors that influence the total data that you can get before having to offload or sync the device. On the hardware side, the device capacity, particularly ram and storage determines how much data it can hold or process rather. There's a balance to strike always a balance to strike and it's going to come back to money. Surprise. These hardware and software constraints need to be assessed against the project's budget and the specific needs. So has anyone encountered the issue where you run out of space on your tablets or you cannot query while trying to search for someone? Not yet. Okay. Okay, so as we navigate these constraints, it becomes clear. Well, I did want to point out. Sorry. This is again from the wonderful documentation. Android implementation. Great recommendations. When you navigate these constraints, it becomes clear that striking a balance between project needs, financial considerations and technical capabilities is absolutely crucial. And it's completely varies depending on your use case and your location. So while we could theoretically invest heavily in advanced hardware, you know, Starlink or remote servers, whatever it may be. This approach is often prohibitive in terms of cost is prohibitive in technological requirements, training, the capacity of your project and your staff, and also security considerations. So, instead, we need to evaluate each of our projects unique needs. And on the ground capabilities, we need a form an assessment so that we can identify happy medium that enables effective data collection and management without overshooting and overspending. So, how am I doing on time? I think I have five minutes before questions. So, okay. So metadata management. This is more depends on the system that you use, you're using, but it's quite unique to DHS to and I want you to picture this scenario. You've identified an issue after testing after you've deployed your application. You're in a remote area. Perhaps an option set is missing a valid option, or there's a glitch in the program or something like that. So the question now becomes, how do you make sure that this update, you know, you fix it and you're, you're all good to go, but how do you ensure that this update is seamlessly integrated into the device of all of your users. At five users, 10 users that's somewhat manageable, you can call them all, but at a national level when you have, say 3000 or 5000 devices, the problem becomes much larger. So communication systems are crucial. All data collectors need to be notified of upcoming changes, prepared on what to expect, giving clear guidelines on how to ensure correcting or correct syncing on the devices. Again, at the scale of 1000 plus devices, if something goes wrong, you can get 1000 calls one day. So you need to, you know, be ahead of that and send out messages, have a, I know one ministry at one point had set up a call center for these types of things when they were doing a pilot. I also had seen mobile device management system used to put out alerts and notices on everyone's tablets. Again, this was when they have connectivity, but at least a call center so you can travel to the nearest place where you can make a call. I mean I have internet but at least a call. So this scenario underscores the importance of robust metadata management strategies which is good, whether you're online offline doing remote data collection or not. So in such cases we need to diagnose the issues, push the necessary updates and make sure that everybody is aware. I'm going to skip ahead. Okay, access control. This is a similar one. I only have a few short minutes left so in a similar situation, when you need to make a push for metadata. What if somebody forgets their password, and they're offline. How do you manage that. It's happened before. And, well you can use your thumb print. In other cases, if somebody else needs to log in. Again, I would just say make sure you have a communication channel to call a centralized support service. Whether it's through the ministry or it's your project or whatever it may be, perhaps use a password manager, have a way to identify yourself in a secure manner over the phone, etc. But also most importantly use the built in permissions and roles in DHS to. Okay, and my last slide here, managing Wi Fi networks and the process of uploading data. If you have, well, I have in the past had a team of 20 or so data clerks data collectors, all show up to a hotspot and need to upload their data for the day to finish their day and be done done working, but you can't necessarily have all the bandwidth for everyone to upload all the data at all the right times. So how do you manage that. Do you go to an internet cafe that costs money. Do you have a hotspot. Do you use your phone. Maybe hotspots are out of fashion now because all the phones can do it. But how do you manage the bandwidth how do you manage access to a Wi Fi network, sharing the password. These are just questions that I'm posing. If there's no really great answer other than proper planning, or you can spend money to make it convenient. My conclusion here. We've walked through several challenges. There's potential solutions related to the implementation and management of remote offline data collection, which is inherent to last mile service delivery, especially with the hs to but always remember that it's the human factor. And so what I say, my opinion, it's, it's the human factor, not the technology that makes the most significant challenges in my opinion. I think the technology is sounds technology works. Yes, there's issues but they can be solved, but it's logistics planning and money that pose the biggest challenges to us. Those can be overcome with proper planning. So as we wrap up, I'd like to hear any insights will open up for questions. But yeah, if you have any challenges. I'd love to hear them. Like I said, I want to follow up on the community practice keep this conversation going. And I hope this conversation has been fruitful. Yeah, very good tips and things to consider if we are going to jump into the mobile. Yes. So do we have any questions on the audience. Just a quick question to you and to other members of the audience in that the sort of mobile implementation implementations we've done and this has been, you know, quite a few years ago now for me but we found that MDMs were utterly essential. You know, it's easy to do a pilot with 50 tablets or something. If you're going up to 200 or 500, you cannot scale unless you've got something that really locks them down and helps you really quickly push out. You don't want to pull back 500 tablets and reinstall things. It's just not feasible. Right. But as far as I know, and I'd love it if anyone knows a one that isn't a free and open source MDM out there yet. I don't know if anyone's found anything that comes close like a one that's actually cost effective enough, but we've ended up using MDMs that are like, you know, $40 a year or something in that style which is okay while you've got donor funding that it's not sustainable and I think that's incredibly important but also a real challenge. I'll be really interested to hear if anyone else has had good experiences. Yeah. So, again, the documentation has a really good chart of various MDM providers, their advantages, their pricing, things like that. The one that I have used the most is Mirador. I don't work for them. I'm not saying that you should use them. But the reason that we liked their model was because their pricing was somewhat sustainable. You could leave the software on a device and scale it from their free version, which was almost useless, but you can at least have it on the device, leave it there. And then when money came in, we were able to scale up to the full features or the features that we needed. So I think that was the biggest feature for us. The biggest need as well for MDM stemmed from having to manage the versions of DHIs too, which we can now do. I forget what it's called. It's so new. The APK distribution app. So I think that's going to be a big game changer, but for security and just tracking of assets when you've got thousands of tablets. I don't know how else to do it without an MDM. So I agree with you. I did want to say as well, I made a note because I forgot to put in my slide, is that anyone that is going to procure say more than 20 tablets because it's a, it's a pain to install everything. Well, I don't know if they'll do it for 20 tablets, but at the thousand tablet level or beyond, at least Samsung will preload the tablets for you with an MDM or the APK of your choice so that they ship with it. You don't have to go through and set up a Google account and blah, blah, blah, blah, a thousand times. So that is my biggest tip for you and they'll they'll typically work with like a local distributor, at least in Ghana. They had a Samsung provider so they came from Korea straight to Ghana loaded with MDM and the DHS to APK. That would save a ton of time. All you had to do is charge the tablets. They're ready to go. George. Yeah, comment more than a question. So I'm George from his center, you are working on LMS. So you said that satellite communications is prohibitively expensive. I agree. I just want to add today it is. I don't know if 10 years ago people would be watching HD TV movies on the subway today they are. And nobody could have afforded it 10 years ago. I don't know the time scale. So I think that we have been promised that this low earth orbit satellites. We're hearing about it since 1015 years so it might not happen. But in five years and 10 years, the cost might be really low and maybe you know the human returns get together and they have a good package for all the HMIS data. And it takes many years to digitize the last mile. What if in five, six, seven years, it's affordable, but we are not ready. It will take us five years to catch up. So I think we don't have to jump on it today, but we should be ready and plan and watch it. And if it's coming so that even five years. It's affordable. We are ready, at least for for some projects. Thank you. Very good points. Indeed. Anyone else online. They weren't. I don't think so. No, nothing. No, these are from the previous. Yes. So about the MDM question some we also didn't we were stensibly looking for providing solutions when we build the documentation and there was no full open source solution. There are some that are part parts or certain things others that are willing to make agreements, but you'll know where that ends. So, so hopefully we soon have something I mean we are we are doing and try to provide the most basic ones but obviously not to become an MDM provider. Not you because we don't have time. We need more team. Oh yes, the community. I mean, please. There's a question over there. I am Gary from the M supply foundation. We, we make M supply, which is any LMS. And just to add to the MDM thing we also have not found there's no free open source MDM solution but the only thing we've been able to do because of scale in 40 plus countries. We've been able to drive the price right down because we stick to the same supplier all the time so maybe that's a, I don't know where that's a potential solution, always using the same one as enabled us to drive the price right down for that. Yeah, that's a great point. Yeah. Okay, we have a, we are five minutes ahead. But I don't know about you I'm freezing cold. So we can leave it here but before closing I want to thank our four presenters for sharing their amazing work and projects. And since you came to this session, I guess there is some interest on Android so I just want to or offline. So I just want to remind that we are going to be today and tomorrow in the experts launch at five. We have specific questions current challenges ideas to share. Please join the experts launch. And then tomorrow there is a session at 1pm on the Android web apps. So that's the big unknown part of Android, which is that you can support your implementations through three web apps. And explain you what can you do with all of them. One of them is the APK distribution but there are two more. Yeah, one is a bit specific but yes. And I think I'm thinking now, and I'm going to tell Jose and Marcos, one part of the session can be also collect ideas. How could we expand those apps, those web apps for the use cases if you are interested please join the sessions and and and I think we'll leave it here. Thank you very much.