 Hello, everyone. Good morning, good afternoon, or good evening. We're starting at 10 a.m. Depending on where you are joining us from, good morning, good afternoon, or good evening. Welcome to Engineering for Change, or E4C for short. Today, we're very pleased to bring you the latest in E4C's 2016 webinar series on the topic of bringing the infrastructure and planning for the impact. Best Practices and Lessons Learned from VecnaCare. My name is Mariela Matata and I'm Program Manager here at E4C. I will be the moderator for today's webinar. If you're following us on Twitter today, I would also like to invite you to join the conversation with our dedicated hashtag, hashtag E4C webinars. Please feel free to tweet during the webinar or after. I would like now to take a moment now to tell you a bit more about today's webinar. Improving healthcare infrastructure in low-resourced settings requires system solutions that close the information gaps between patients, caregivers, and decision makers. The social enterprise VecnaCare designs and deploys information management and technology solutions which help people live healthier and better lives in environments that have limited access to infrastructure, specifically power or internet access. Today we're joined by Paul Amendola, the Executive Director of VecnaCare's Charitable Trust, who will take us through VecnaCare's eHealth solutions. Key lessons learned and best practices deploying electronic and medical record solutions in low-resourced settings. Welcome and thank you for joining us today. Before we get rolling, I would also like to thank the E4C webinar series team. If anybody out there has questions about the series or would like to make a recommendation for future topics and speakers, we invite you to contact the team via the email address visible on the slide. So the contact will be webinars at engineering4change.org as seen in the slide at the moment. Today's webinar is part of E4C's professional development offerings information on upcoming installments in the series as well as archived videos of past presentations that have been found on the E4C webinar's webpage. Our next webinar will be in January and will be shared details on our site and E4C members will receive an invitation to the webinar directly. So before we move on to our presenter, I would like to tell you a bit more about engineering4change. E4C is a knowledge organization and global community of over one million engineers, designers, development predictions, and social scientists leveraging technology to solve quality of life challenges faced by under safe communities, including access to clean water and sanitation, sustainable energy, improved agriculture, and more. We invite you to join E4C by becoming a member. E4C membership is free and provides access to relevant current news data on hundreds of essential technologies in our solutions library, professional development resources, and opportunities such as jobs and fellowships. E4C members enjoy a unique user experience based on their side behavior and engagement, essentially the more you interact with the E4C site the better we will be able to serve your resources aligned to your interests. We invite you to join E4C's passionate global community and contribute to making people's lives better across the world. Check out our website as seen in the slide, engineering4change.org to learn more and sign up. A few housekeeping items before we get started. Let's see first where everyone is from in the chat window which is located at the bottom right of your screen. Please type your location. If the chat is not open on your screen, you can access it by clicking the icon on the top right corner of the screen. Any technical questions or administrative problems should go into the chat window. Feel free to send a private message to engineering4change.org and if you have any issues. Let's see where everybody is from. Cambridge, Houston, New York, Washington D.C. Florida andapolis in Brazil, welcome. Diego, I see new countries today. Great. So you can also use the chat window to type any remarks you may have. During the webinar please use the Q&A window located below the chat to type in your questions for the presenter that will be doing the Q&A at the end of the session. Again, if you don't see this, you can access it by clicking the Q&A icon on the top right corner. If you're listening to the audio broadcast and you encounter any troubles, try hitting stop and then start. You may also want to try opening WebEx up in a different browser. Again, if you have any problems, please type directly to engineering4changeadmin through the chat. An important information is following the webinar to request a certificate of completion showing one professional development hour for this session. Please follow the instructions on the top of E4C professional development page. Let's just get started. It's now a pleasure for me to introduce our speakers today, Paul Amandora, who is the Executive Director of ECNAC's Shadow Will Trust. He pre-release work at the International Rescue Committee in Health Information Systems. Paul has designed and implemented data collection projects and epidemiology studies in Sub-Saharan Africa and Asia. His work focuses on increasing data quality through data training, reviewing and advising on information systems and database creation. Paul is based in Cambridge, Massachusetts. We also have today Emily Taylor. Welcome, Emily, as well. Emily Taylor has worked in global health for over 10 years focusing on strengthening high quality patient-centered health services. Her areas of expertise in LMICs, her areas of expertise area are in clinical quality improvement, process improvement, and digital health initiatives. She has particular experience in proposal and grant development, business development, project management, strategic planning, and board development. Without further notice, I just want to introduce Emily and Paul so that they start the presentation. We have a brief comment before we get started. They are part of our webinar series of this month, which is based in ICT for Development. They will be talking about one of the tools that is... Well, the good examples of e-health solutions that is being deployed and implemented in the world. Just for... As a general context, e-health is electronic processes and solutions that are being implemented to improve health in the world using ICT or information and communication technologies. Without further to say, I will pass it along to Emily and Paul. Great. Thank you so much. This is Emily Taylor. And this is Paul Amandola. We're really thrilled to be here. We've been fans of Eversy and the Solutions Library and their work for a long time and use the Beta Solutions Library to keep an eye on the landscape. We are here to talk about specifically what Beck and Keras does, our use cases, some of our projects, and we'll also be taking a deeper dive into a couple of our projects and looking at best practices and key lessons learned and how to apply those. I want to say from the beginning that people should feel free to use that Q&A. And I know that we have dedicated time at the end. But if we're ever at a level of detail that is either too microscopic or too broad, please let us know because we can tailor this presentation to the attendees on the call. And we're happy to do that. We want to make it a good use of your time. So we're going to go ahead. Okay. The first thing I wanted to do is say thank you because I have been on the side of Jackie and Sam and wrangling presenters. And it's no easy feat and they did a really nice job and it's great that Eversy is so dedicated to bringing people together and for disseminating content and highlighting various projects and looking at ITT for deep. So thank you to Jackie and Sam and Yana and Mariela for coordinating all of this and her team paths. Do you want to talk a bit about just the overall presentation? Sure. So the structure of the overall presentation we wanted to give an overview of the VECNA CARES technology solution and then a more use case of how it's implemented and what our approach is in the field. The approach meaning or the implementation approach but also the approach as to how the data and the data quality is structured. Great. So just as a quick overview of the flow of the presentation so you know what you're getting into. Great. So this is repetitive. I apologize but we're going to do a brief introduction to VECNA CARES from where we came. Current projects and core use cases. Take a deep dive into the COCMA and TRANSMARA digital district case studies and then talk about best practices and lessons learned. Okay, great. So VECNA CARES is a nonprofit organization. We were started in 2009. We were born of VECNA technologies which is a for-profit privately funded healthcare information technology company started in 1999 by a consortium of MIT engineers. They decided that they wanted to leverage the intellectual property and the free procedures that they had garnered from their time deploying solutions to established markets and figure out how to leverage them and deploy them in lower resource settings. So it started as a sub-initiative of VECNA. We were initially just, we were initially a sub-initiative as I said and had community service hours and then with our success and the uptake of our use case and our solutions, we spun off as an independent 501c3. So in introduction it mentioned that we're a social enterprise. We're actually certified as a, we're registered as a nonprofit organization that we do have a social enterprise-like business model. The great thing about being tied with VECNA CARES, we are physically co-located with them. We are two legally separate organizations but we have a lot of access to their expertise around engineering both hardware and software and robotics and we do a lot of brainstorming with them and also are able to use some of their corporate structure. So we benefit greatly from being with them. Great. So as a quick overview of VECNA CARES mission statement and we've had a couple of mission statements over our eight-year life. This is our most recent and its intent is to be encompassing. VECNA CARES designs and deploys information management and technology solutions to create measurable impact, helping people in low resource settings live healthier and better lives. And I know that the title of this presentation is about, or the series is about eHealth and how ICT for G and how ICT is strengthening eHealth but it's important to note that what we do, we are not preferentially focused on health. We happen to be doing health right now and it's kind of our home base but we have a lot of ideas and some pilots out about how to use this for education and wash initiatives and things like that. So I loved yesterday. This is a great segue to our presentation yesterday. Engineering for Change sent out an email with a lot of great content about ICT and I loved how the tagline was how ICT is improving everything and I think the most important word in the sentence is improving. It can't fix everything and we really have to remember that whenever we look at ICT for development and when we look at eHealth. So the question isn't what could be fixed with ICT because ICT or a technical solution is not a fix. The questions to ask are what current processes could be improved with ICT, what current issues could be mitigated with ICT and what outcomes could be amplified with ICT. Because as you all know, either digitizing or automating or whatever the verb you put in there, if you digitize a bad system, it's still a bad system. So you have to think about is the process right and then you have to think about how can ICT bolster those processes and improve your outcomes. So our basic theory of change and this is not unique to us, this is sort of the eHealth theory of change and the access to health data theory of change is that with better data, with high quality data, administrators, it allows administrators, it allows clinicians to make better decisions. When they can make better decisions, they can implement better programs because they can better allocate their resources, be it human resources, health commodities, infrastructure resources, et cetera. And with more targeted, better programs that are more tailored and use resources more efficiently and effectively, the idea is that you can improve health outcomes. Okay, so our basic implementation plan at Becna Cares that achieves the theory of change is we do an initial site assessment, we look at community support, we engage community support and local buy-in is so important. We do the installation of our harbor, which I'll get to more in a little bit. We do a central training so that we build capacity at the site. We do a shadowing period. The intent is that we train the trainer and then ultimately we leave and the system stays and continues to serve their needs. And then we do a go live and then we do keep in touch with them afterwards. Becna Cares' three core competencies are in hardware, software, and what we call data consultative services. And we really see these things as totally intertwined but also able to be disaggregated from each other. We have hardware solutions, we have software solutions that we've developed in-house, but at the end of the day or actually at the beginning of the day, the most important thing is to find the right solution that works for the setting where we are working. So we can look at hardware, companies hardware, we can look at our software, we can look at other companies' software, but ultimately we're kind of tailor the right solution for the setting. Paul, do you have anything else to say on that? Okay, great. So just to give you a sense of where we came from and where we're going, as I said, we started as Becna Technologies and you'll see some of the Becna products over there which will look familiar to the engineers and any roboticists who are in the crowd. We have drug delivery, you know, physical, they physically move around a ward or a hospital and those are widely used in France and in the U.S. and as I said, in 2009 Becna Care spun off, in 2012 we underwent a huge expansion in Africa. In 2013 we started working with more U.S. clients and then also some medical missions who do pop-up work in either disaster relief settings or in low resource settings. And then from 2014 to 2016 we've experienced really tremendous growth. We went from a staff of about four to a staff of 20. We've had a lot of, you know, our global projects have really expanded and we're scaling up. Our next big step is that we are deploying, we are developing and deploying a version 6.0 of our hardware. So you'll get to know our hardware a little bit better and we'll talk about the size, but just keep in mind that we're also currently working on a bigger or smaller, better, faster, stronger, cheaper version. So as I mentioned, we'll give you a quick overview of our products and services and I still encourage people, you know, if this is becoming too focused on, if we need to take a step back and talk about e-health, we're happy to do that. But this is our flagship product, the Clinapak. It's about the size of a traditional briefcase. You can carry it in a backpack and it was a developed, designed, engineered, manufactured in-house here in Massachusetts. We do all our manufacturing in Uber in Massachusetts. And what is, it is, it's a rugged, easily portable solar powered power management and service. It can be, as I said, it can be solar powered. It can be plugged into AC power. You can plug it into a car battery and when you power it up and literally just press the on button, I am not an engineer, and literally just press the on button, it instantly throws a local Wi-Fi network. You can see the Wi-Fi antenna on the top left there. And what that means, this is intuitive, or you know, obvious to all of you, is that any Wi-Fi enabled device underneath that area, which is about 50 meters in radius, can instantly connect to the Clinapak network, to the local access network that the Clinapak has popped up and any software that is on the Clinapak, that has been loaded on the Clinapak, can instantly be used by all of those tablets. So really here the, to just break it down, the benefit is that in a clinic, in a refugee camp, which is one of our core use cases, patients, consumers will come in, they will get registered, they will, you know, a nurse or an administrator will take information about them, take their birth date, take their name. Then they might send them to a nurse to get their vitals done. Then they might send them over to get some labs done. Then they might send them over to the pharmacy. And previously this involved paper records, this involved, you know, a slip of paper that was easily lost or destroyed. And at the end, they're all stored, you know, in a pile. And it's really hard to parse that sort of collection of data for any meaningful analysis. And of course, if you go back to our theory of change, you can't make better decisions if you don't have access to that data. So what this does is the person registration has a tablet. The person at labs has a tablet. The person at vitals or triage has a tablet. And as soon as the patient is entered at any of those points, that information is instantly synced to the clinic pack and then to all of those tablets. So that as the patient moves through the clinic, their medical record both precedes them and follows them with no actual exchange of paper. It also syncs to a master database that's listed on the clinic pack. And that's a password protected database. And that can produce aggregate level reports, which can be used for looking at resource allocation, looking at, you know, health burden, et cetera. The clinic pack can also power devices. As I said, it's a power management system. So once it has an inverter in it, so once you power it through solar or AC or a car battery, it can then power things over using USB or Ethernet and see what else I want to say about the clinic pack. Great. So I mentioned it throws a local Wi-Fi network. This is just a graphic for those of you who are visual like me. This gives a quick overview of how we were helping in the Ebola response in Sierra Leone. So you'd have one clinic pack and all providers underneath that dome were able to access information. Again, this is up on the ground in Sierra Leone. In addition to our hardware, we have software, which is based on the Vecna patient solution software. The key software functions of the clinic pack software are patient registration and demographics, vitals, capture, diagnosis, treatment support, case review, administrative tax report, pharmacy and inventory distribution and patient flow management. Those are all pretty typical EMR capacities. I want to point out the pharmacy inventory and distribution because it's pretty unique. We're able to manage a dynamic inventory and track prescriptions and dispensation so that people can have a chain of, quote unquote, paper chain of their health commodities. A couple screenshots. We deploy in three different ways and I'll speed this up a bit, sorry. On the bottom here, you have asynchronous, post-it and synchronous. The asynchronous is when we are completely offline and we leverage com carriages and open source platform for conducting surveys. We can also deploy the software as a cloud-based solution, which we do in areas with better connectivity. I'll talk about that with the work we do with Special Olympics. And then clinic pack local, like we were talking about, is where maybe there's no power, there's no connectivity that you have to collect sophisticated information and be able to do analyses in the absence of that infrastructure. And so you would deploy the clinic pack. Two of our major use cases are in refugee camps and disaster relief, e-health, and the clinic pack system in particular and other systems that exist can help with the management of camp health clinics, the administration of them, like we've talked about the resource allocation and then also monitoring and evaluation, both for health burden and also for reporting to funders in different areas. And then we also have a community health use case where we focus on better patient care, surveillance, resource management, and community health work. I want to take a moment and just run through a few of our projects that show the ways that we deploy our different solutions. One is Project I Deliver, which is with Merck for Mothers. And the goal is to develop a digital clinical decision based, clinical decision algorithm and data capture tool for the interpartum period based on the Better Birth Checklist, which was recently released by the WHO. So we work, we are the soccer organization and also the integrator for this with a couple other firms with a prime on it. And the solution that we developed is a customized workflow and data capture tool based on that Better Birth Checklist. And it's being locally hosted on the Clinapak Node Hardware. So that third use case we were talking about where it's locally hosted in areas with no infrastructure. We're piloting it in three clinics in Transmara where we've been working for about five years. So this great work piloting it in clinics that are known to us. And we're currently doing some train-the-trainer sessions and remote support from in-country and then also from second headquarters staff. And then next I just have a quick screenshot of the, a sample screen of the UI of the iDeliver tool. And what this shows is a word dashboard, which, well, it doesn't look super exciting to people who are used to EMRs that are deployed in higher resource settings or established markets. Having a word dashboard of, you know, who, like just a literal snapshot of who's in the ward, what's their status, what needs to be done next. This is a very simply a task list, but it's a huge improvement for ward management. As I mentioned, we also work with the Special Olympics. Special Olympics is a really great thing that I actually didn't know about before I started here, but at each of their events, and they have hundreds of events around the world every year, they pop up a health screening event where they will have six disciplines. They have about teeth and dental health, eyes, ear health, et cetera, and they have local students and local doctors come and volunteer and do health screenings. In the United States and in Europe, these health screenings are great, not as essential because a lot of these kids are seeing doctors regularly because they have complex special healthcare needs. But the Special Olympics works locally, and these health screenings that are co-located with their events are open to the public. So whenever they do an event, they can come and also have those health screenings, and it's literally sometimes the first time someone's seen a doctor. So we've piloted so far a 15 Special Olympics events globally, and we're getting ready for a big deployment at the World Games in Austria, which we're really excited about. And just to give you a sense, I'm so impressed by the Special Olympics, but if you look at the green box in the lower right, they held almost one short of 900 clinics. They did 134,000 athlete exams in 57 countries last year. So the potential reach there is great, and it's a huge public health database. We also work with AmeriCares, where we do a pharmacy inventory management system. They wanted a way to track the donated inventory and then to track where it's going and also be able to tell the funder, tell the donor, and be able to look from a clinical perspective at what medications were being most prescribed. There are opportunities to scale internationally with that. Some of you may know AmeriCares does a lot of work in many, many countries with drug distribution and essential commodity distribution. Similarly, we work into clinics in Haiti with care to communities where we digitize and standardize the outpatient visit workflows and pharmacy operations. And then I'm going to let Paul talk about the work that we're doing at Kokoma Refugee Camp and Transmara Digital District on a broader level, and then he's going to dive a little bit deeper into the use cases and some best practices and lessons learned. Cool? All right. Thank you, guys. So when you're talking about e-health, a lot of the challenge focuses around that we're talking about aggregated data. Aggregated patient data that's being reported into an information system through paper forms. For instance, you'll know a clinic saw 300 people yesterday, or they saw 2,000 people last month and for which diseases they saw, but it won't break down to a patient level. There's the potential there to use ICT, to use e-health, to use digital technology to get that information down to a patient level to have more of a patient level outcome. So for the next two projects, for the EMR at the Kokoma Refugee Camp and for the Transmara Digital District, our goal was similar for the two projects to design and deploy a patient-centered EMR. As a patient was going through a clinic, either at the refugee camp or through the Transmara Digital District, there would be a longitudinal patient record that would be collected at the point of care. When a patient came in through a registration, their profile would be registered and then the clinician would use a tablet to record their diagnosis, their signs, their symptoms, and then their treatment. Our overall goal for the project, this is basically why we were doing these projects. There's no patient-level digital data. There's a high burden of reporting and poor data quality. So let's think about what this means for all three of these. With no patient-level digital data, as a patient was seen over multiple visits or over multiple facilities, their previous medical history was unknown. Doctors would be relying on patient reporting of their own patient history to make a better diagnosis and to use treatment algorithms. High burden of reporting in some clinics, especially in rural clinics, clinics would be closed for a couple of days a month to collect, to tally reports from paper tally sheets onto monthly reporting forms to then be reported into ministries of health or to UNHCR systems. If clinics are being closed for a couple of days a month, that means patients aren't being seen. There's the potential for digital solutions where if data is being collected in real-time and being reported in real-time, there's the potential to keep the clinics open for additional days per month and more patients being seen. And poor data quality. Honestly, I think that the data quality is so poor in a lot of rural settings that the actual disease burden and health status of a population is mostly unknown. As a patient would come into a clinic, they would be... Their visit is most likely recorded as a tick mark on a tally sheet. That tick mark then at the end of the month is manually calculated and added to a monthly reporting form. There's so much potential for human error both in the diagnosis of the patient, the recording error, the tallying error, that the actual reality between what the information system is saying and what the reality on the ground is can be so far from reality that resources aren't being allocated correctly and patients aren't being treated correctly. This is what we're talking about when we're talking about patient records. These are two different, both from Transmarin and from Kakama, what the patient records look like. Even though the one on the right, this is from a different clinic in the same refugee camp, looks more organized. It would be really difficult to say with any level of precision what the data collected in those paper forms are and what was being aggregated up to the information system. Our outcomes. What are expected outcomes for the project? Increase in data quality, increase in treatment adherence, reduce time for data collection, better use of medication and management, and availability of the data for action. The reality of the ground in both instances, there's unreliable power. So between off-the-grid or off-the-generators, power is not a commodity that can be relied on at either the clinic or the hospital level. The connectivity is very poor. So no reliable internet, no reliable cell signal. And there's a significant problem with human resources, high staff turnover, limited time for training, limited time for supervision and follow-up. This was our plan. A user-centered design process to assess the feasibility of an ICT solution, address the considerations of the solution, and then design and deploy. This is our user-centered, our human-centered design meetings at the beginning of the project when we were looking at what our expected outcomes of the project were, and going through the information system to structure the approach. All ICT projects, especially around health, are going to fall into four different categories. When you're talking about user-centered design, instead of thinking, should we be designing an ICT project, it's better to think of these four categories as possible places for improvement. So data collection and data quality, which is simply digitizing forms. Probably anyone on this call would be in a position to digitize a paper form. It's easy to learn. It's easy to deploy. Increasing provider capacity. So in this setting, it was instituting clinical decision support, so providers would be able to have instant real-time feedback on a patient's diagnosis and treatment. Beneficiary knowledge. There's more of a potential as connectivity and mobile phones are more accessible instead of always thinking about taking data in to start pushing more data out to beneficiaries. And system strengthening. We wanted to use the ICT solution to strengthen the health information system of both the digital district and the refugee camp. Once we outlined with the clinicians, the data clerks, the pharmacy management, what our ideal solutions would look like, we started going through considerations. The considerations listed here are a blueprint for other ICT projects and e-health projects that you might be implementing. The considerations represent lessons learned of dozens and dozens of ICT projects in low resource settings. I'm going to, we'll go through them one by one with some examples. So the most important consideration is the staffing resources. If there is not someone who is owning the project on the ground with the correct ICT or e-health profile, there is going to be a serious constraint of moving the project forward. Out of all the considerations of deploying ICT in low resource settings, the staffing consideration, it's just, it's the most important one. It's sort of the maker break, decision maker for assessing feasibility. It's also unrealistic in most cases to think you can divide the responsibility of the project up over multiple people. So not 20% of someone's time, 30% of someone else's time. This is a specific ICT profile that needs to be filled and needs to have the roles and responsibilities clearly defined. The next consideration, existing technology and literacy. When deploying a surveillance project in two different areas, it was obvious to see that if phones and technology exist in a certain setting, it's a lot easier to deploy. We were deploying a surveillance project in Congo in the DRC and we had to start the training. We had allocated five days for training and ended up being 10 days of training because we needed to go back to a level of this is what a mobile phone does, this is how you turn it on, this is how it's powered. Meanwhile, we did the same exact project in Sierra Leone where the community health workers already had multiple phones and the training took three days. So there needs to be some assessment of the existing technology and the literacy level of the data recorders. Literacy also does not translate into new mercy. So although someone might not be able to read and write, there's still a potential for them to report accurate numbers. Data access and use, it's easy for, it's far easier to collect data on a mobile phone where the challenges is data access. What you don't want to do is implement an ICT project that will only collect data but then make a lot harder for the data to be accessed. If a clinician is already collecting data on paper, that paper is always accessible. If they're not collecting data on a tablet or a mobile phone, if that data isn't as easily as accessible as on paper, you're now withholding data from someone who needs it. It's just another consideration that needs to be addressed. For mobile coverage, gaps that are supplied by most of the network providers are usually inaccurate. There's no substitute for going to the ground to the clinic to see if it's possible to get a signal at the actual reporting site. Logistics, what if a reporting device, if a phone is lost, if a phone is broken? What are your plans for replacement? Data protection, especially if you're collecting individual-level data. What is the plan for confidentiality for privacy? And what are the ethical considerations around collecting the data digitally at all? The ethical considerations are something that's often overlooked but it's something that needs to be essential within the feasibility assessment. And evaluation. How do you determine the difference between evaluating the ICT portion of a project versus the actual health services? If a project is not working, is it because the digital ICT solution wasn't working or was the actual service implementation of the project poorly designed? So within the feasibility, this is the technical assistance and what at BACNA CARES we're structured to provide. The overall solution feasibility, staffing assessment, which hardware and software are most applicable, what's your overall cost and then your cost to sustain the project? What the training looks like, what partners are available, and then how do you structure the evaluation and indicators for both the service provision and the ICT implementation? One of the key lessons learned is to always make sure your data systems are as interoperable both the data structure and the systems as possible. There's an open HIE initiative which outlines the different interoperability levels, the different interoperability service layers needed, but this is another discussion, a more technical discussion. Several technical discussions. But something key to keep in mind as essential in your planning. And then planning for impact, understanding the outcomes and impact. For a lot of projects you're thinking about reducing mortality or having an impact. But the reality of what the routine monitoring data is going to be collecting are just outputs. And then how to structure those outputs to show causality and correlation between collecting your routine outputs and linking them to outcomes for the patient and then impact for the project. Within that evaluation methodology structuring your indicators to track the uptake of the system with usability indicators and how the system is better improving the service provision through efficiency indicators. For example, for usability indicators just tracking the number of users and the number of reports over time will indicate to you the uptake of your system. If the system is well designed as something that's helping the clinicians and the staff it's going to be used. And that's why it's so important to measure because oftentimes clinics or health systems that are they're severely under resourced. So they are low to give not low as I shouldn't say that they may be reluctant to give critical feedback and they may say yes this is a great system because they're getting computers, they're getting attention, they're getting perhaps a cash incentive through a salaried position for that ICT manager. But it's really important to kind of lift a rug on that. And I think a lot of what Paul's talking about is lifting the rug on those design assumptions and the things you hear and looking at the data. There's that great quote like in God we trust all others must bring data. And I think that's really what we're saying is you really do have to as engineers I'm sure you all use data, data, data. You think something? No, show me the data. The usability indicators are really essential for knowing how the system is actually being perceived and used. A quick snapshot of routine evaluation we do around the qualitative attitude about the information system that again just helps track usability. Data quality and results, this is something important to track. Something like this is only possible through collecting data digitally. This is a real-time example of the completeness and quality of different clinics within the digital district. So our key lessons learned. Design with the user, bring the infrastructure, manage expectations both around the data outputs and the data use. Be sure to use data interoperability and structured data standards and understand the best use of data. What data is needed at each level of the information system? A lot of these key lessons learned were based on the principles of digital design which are available online and is a good resource to review. Just to close out the presentation, there are still some ongoing challenges and considerations. Unique patient identification is still a challenge even within using technology. It's still difficult to correct for multiple patient visits to correctly identify a returning patient. There are still security concerns whenever deploying technology to low resource settings. I think there is time now for a question and answer. Thank you so much, Paul and Emily for being here. It was an amazing and insightful analysis of the eHealth challenges, specifically what you have encountered in the field. We have time for Q&A and I will kick it off with the first question. You mentioned that you at VACNA CARE disaggregated hardware, software and consultancy. I'm really curious how you do that and which side is failing in implementation of the project? Can you elaborate a little bit more about how VACNA CARE is handling these issues? It starts with an assessment. Likely an assessment of what the program is really looking to measure. An assessment of what the reality is on the ground and the resources and sort of the consideration slides. But starting with the data and starting with the measurement objectives and then going through there and assessing what's the best software in terms of software we provide or other software available and then a hardware assessment of what the connectivity is in the setting and what's the best solution? In terms of the second part of your question with how we determine if it's the ICT solution or it's the health or it's some combination one of the things you're talking about with the usability indicators is quantitative measures and cross referencing the attitude measures or things like on this slide where you see HR stability and these are anonymous surveys that we give to dappers in the clinics where we're deploying. If they reveal to us an anonymous survey that there's real instability in HR or leadership really isn't engaged you'll see a direct correlation in terms of and so if it's being used very well in areas where staff satisfaction is high and then not well used in areas where staff satisfaction is low let's look at the what's our variable there, it's process and it's staff attitude and leadership attitude. So making sure that we I come from a quality improvement background and the process improvement background and that's what I always say if you digitize a bad system it's still a bad system. What ICT for D one of the things that it can do with health is that it really elucidates those issues and makes them pretty glaring and it can be messy. Things often get worse before they get better in terms of process because you can really pinpoint where the problems are and where the system is failing. And during that assessment it's important to remember that when you're looking at information systems, when you're looking at data flow a lot of times digitizing might not be the solution it's if you're seeing challenges with a difference between with the programs aiming to collect and what they are collecting if their data collection tools aren't correct it's still more appropriate in some cases and in some settings to just use paper to just have a better structured paper system collect the data that the program is aiming for and that's part of the assessment process as well. That's great. Thank you so much for that answer. It's very insightful and of course that's the direction that makes that paper based and digital because we know now after so many years of experience that it's actually the case, but systems don't work as you mentioned before. So you need to fix the system and then digitize them. We have another question and this is very specific. They're asking if electronic medical records or IT systems stand alone or has it been designed to some type of electronic medical record standard or has it been designed to some type of EMR standard? For example, does it dust it or will interface with full functionality to an epic site? So the sort of a two-part question when we are looking at the reality on the ground in the places we work and the level of information that is currently collected. As Paul was saying earlier the data collected at a site might literally just be a number of fevers seen in a month. So going from that to patient level data is a huge improvement. Going from basic patient level data around vitals and diagnoses and a longitudinal patient record about your BMI over time if you're looking at malnutrition it's a huge leap to something like Epic. We are not as complicated as Epic. We are designed to standards since we piggyback on the Vecna systems we are a little different but we have a compliant we can explore in HL7 all those sorts of things but it doesn't interface with an epic system. If I understand the question correctly and then I see part two of that question by Bob are you going to use this as a diagnosis and did you want me to answer that one as well? Yes. This is a great question and the project we are doing with Merck is incredibly facetious about whether or not things are classified as a medical device. Currently it does not provide diagnosis so it's not considered a medical device. There is in the works potentially integrating what we've developed with iDeliver with a predictive algorithm known as Selma which is a simplified electronic simplified effective labor monitoring to actions algorithm that does provide not quite diagnoses but clinical care recommendations at which point the actual software may have to get that classification but currently it does not. Great and if you can address like this next question we have just two minutes left but it's a very good question that Gary put here. What's your approach to updates in hardware and software and on the ground equipment like how do you do to update? Yeah so it's a great question but what we quickly realized was that we have these and I'm going to loop this in with the question right above how technology can be deployed in rural remote areas and how we did it in Kenya because they're really linked. In Kenya we have about 35 clinics right now and since they're all offline you obviously can't do math updates to all of the software and so what we have is we have a field team based in a city in Kenya in Kilgores who does regular visits to the clinics. The hardware itself is really really designed and optimized to be used in rural settings. We actually put it against a wall and put up like a cage around it because if people can access it they try to fix it or they try to like mess with it and really it's just best if left completely all the time it's meant to be hooked into the solar panel or battery and really not be touched. The software is also designed to be really rugged and not require a lot of troubleshooting. So for updates what we do is our software team here will send upgrades to our Kenya team who will then go around and perform the upgrades on the actual hardware so that we can update the software. That's super interesting. Thank you so much. Well the time is up but I just wanted to thank you Emily Paul and everyone who attended. As a last bit of information I posted on the chat the link to the Solutions Library and offering here at Junior Front Change that this product is in so you can take a look at that as well to learn more about this product that the Acnecors offer. I want to thank you for attending and have a great day.