 the trackers who you just look at the paper and exactly what it is on screen. But we can also make it more complex by giving user interaction using program rules. And it can be as simple as just correcting user inputs. For example, if you put in some information that is not correct, the system will then respond and say, you know, this is not the correct range. For example, black pressure, et cetera. And you can also go into something a lot more complex which is clinical decision support where if you enter the black pressure and it's really high and the system also looks at other characteristics of the client, they can say, for example, in our ANC program, they can say, for example, this client we suspect has chronic hypertension. So please refer her to the next level of care for management of chronic hypertension. Or for example, if the system recognizes for antenatal care, if the system recognizes that this woman did not have hypertension when she started her pregnancy and now has it, the system will then say, hey, this woman actually developed hypertension while she was pregnant. So this is gestational hypertension and please refer her or give her this particular support. So that becomes a lot more complex. We can also have client messaging with SMS. And you can have just generic messaging that sends a message to everybody saying, hey, antenatal care is available at this and that clinic or antenatal care is you should take care of yourself during antenatal care. You should take your folic acid, et cetera. So that's just blanket messaging. Or you can have targeted client messaging which is more specific based on client information that has been entered into the system. For example, you can send a message to a particular client telling her, the last time you came, you were identified as having chronic hypertension. So we want to make sure that you come for your next appointment on such and such a date at this particular clinic where we will check your blood pressure and advise you on what to do to make sure that your child does not suffer any consequences from your chronic hypertension. So this is where the system takes information about that specific client and also about the org units where the client was registered and sends information to the client about her specific case. And of course we know that the more targeted SMS messages are the more likely people are going to respond to it. So the more complex you have, the more results you potentially can get. But of course complexity involves also means a lot more work. We also have feedback dashboards where you can do it by org unit or you can do it by the user, the particular clinic or the particular health worker and tell her, hey, you were supposed to screen every client who comes in for blood pressure for antenatal care. We want to make sure that our chronic hypertension numbers are managed. And so you can provide that user with a dashboard showing her that out of all the clients you saw this month, you only screen 30% for chronic hypertension. And we can also give what we call action items and say, hey, you know, make sure that you increase this number by making it part of your routine or make sure that if you have a problem with your blood pressure equipment, if that's the problem, then please contact your supervisor. So then the more complex the tracker program is, the more targeted it is to either the user or the client. And we know that based on health behavior, the more specific data that you get is, the smaller it is to you, the more likely you are to work. So again, as I said, the complexity, you then have to counter that with coverage and skill because if you have high coverage, you might not be able to be as complex as you would want to be because that requires a lot of management. And we can also look at Android versus web or in some projects, we use both. So I will now move on to looking at this graphic which you potentially have seen earlier in this academy, which shows the interaction between coverage and complexity. So the higher you go in coverage, right, the most likely, like for example, this Ghana NCH program has very high coverage but very low complexity. And that usually is what happens. This also has high coverage but has actually been able to achieve high complexity. And I know that the TV program in Ghana has required a lot of work in order to achieve that. Then you have an example of high complexity but low coverage, which is Ghana using Ghana's ARTHIV tracker program. So exactly as I discussed before, the complexity of the tracker configuration can go from paper and screen to very complex functionality, like feedback, passwords, interactions with users and program rules. So when we look at some key considerations like scale, you have tracker has a possibility of reaching all levels of an organization. And that also increases, if you go, if you really increase scale, that increases the number of users, devices and support requirements. So that's something to take to think about when you're trying to scale up. If for example, you start a project, pilot project in a district and you want to go nationwide, you have to consider increased costs like devices, training and support. If you are trying to do scale, you should probably consider reduced complexity because you have so many users who you have to train and you have to support. And you also have system maintenance. One of the biggest issues with tracker and going to scale, national scale, et cetera is the user support and the training. You have to, for every person that you train, previously if you had aggregates, you just need to support maybe the data manager or one data manager at a facility or even when data was done at the district level, aggregates data was done at the district level. You just needed to support that one district health data manager. Now you have every single nurse who could lose their phone, could have problems with their login, who didn't come up for part of the training, who doesn't have enough data or has some problems, she doesn't know why it is. So this is why we say when you are looking at increased complexity, increased scale, you probably should consider reduced complexity because you're going to make sure that everybody understands this level of complexity. Then you have to look at Android versus web if you're trying to decide what type of limitation that you want to do. And in some contexts in low and middle income countries, we know that we have challenges with internet access. So that is something to consider if you are having serious challenges with internet access, it might be a good idea to use Android. But Android also means that you can have a tremendous number of devices, right? If you have a tablet, it usually just goes to one person or if you have a phone, it usually goes to one person. But if you have a laptop, maybe it's at the facility and everybody uses it, et cetera. You also have to think about the technical literacy of the users and the amount of data collected on Android. Android also has challenges with complex systems which have a lot of program rules. And in some of the implementations that I have done, we designed primarily for the web and then we realized on Android that it was quite a challenge because even though the Android users were not using that complexity, the program rules existed and would run and was quite a burden for Android users. So is that something to consider when you're developing your system, make sure you test both on Android and web to make sure that the beautiful complex system you designed works well on both systems. Then one of the key issues is real-time versus secondary data entry. My and the Norwegian Eastern Republic Health, as I mentioned, we use the eRegistry which is the most complex form of the tracker and involves real-time data entry because our primary goal is to give decision support to the health worker most of the time midwives while the client is there so that if you enter blood pressure, the system will be able to tell you this person is hypertensive. And so we require that it is done while the client is there in real-time. So as we said here, real-time data collection provides opportunities for decision support, data validation and a guy's health workers and the data entry and clinical processes. But you sometimes might need to change the secondary data entry and why would you do that? You could have a problem of bad connectivity or power at the point of care. So you might then have to do data entry at a later stage. And we have that, for example, in the Bangladesh National MCH Tracker. We did an implementation in Bangladesh where some facilities don't have power. So they basically would leave there or it was just really bad internet. So they would leave their laptops at home and then they would take their registers with them home and then enter the data later. Also, secondary data entry gives you a lot more flexibility. You can afford to have some system downtime or unreliability. So if you are doing a complex system or you don't have the resources for somebody to respond really quickly, that is an option to think about because, for example, for weekly data entry or data entry that's done at the end of the day, you can have some flexibility if the data doesn't get entered right within 24 hours, for example, you could have a weekly data entry. You have a reduced number of devices and health workers who have to be trained and that is a reduced cost. Because if you're doing secondary data entry, you can have a data manager who just sits and gets all the data from all the nurses, the midwives at that facility. And at the end of the day, they just enter it. So it's just one device that you need and one person that you need to train and support. Then, if you're only entering a subset of data from the paper register into the tracker so that you still need some paper data, then you might consider secondary data entry because you are not getting rid of paper, right? So then you might as well have the health worker write on paper if they can, if they can enter into the system at the same time how the client is there, that's fine. But they do need to write in, if they do need to write in the register information that does not go into the tracker, then you can be more flexible and say, hey, if you have bad connectivity while the client is there or the client needs more care, you can just take a break and not enter the data there, just make sure you write it in the paper register and then later at the end of the day, you can enter it. If you're doing a transition from paper to electronic, that is a very tedious process. It involves a lot of change management and so you might want to do secondary data entry to really help the transition be smooth. People, obviously a lot of countries do not want to do paper because they don't want to do only electronic because they don't trust the data. So then if you choose secondary data entry by somebody who is really good, you can then produce the data to the powers that be and make sure that it's good and that will help with the transition. If your system is only used for reporting and there's no decision support needed or involved, then there's no point in doing point of care really because that is really expensive in both the devices, the support, the trainer, et cetera. So if you're just using it for reporting, then and there's no user interaction, you can just hire a data manager for them to put the information in. You're just using the tracker so that you get individual level data. If you're just using it also for generating appointments or supporting such, you know, you start for work processes that do not involve the clients physically being there. You can also just enter the data at the end of the day and then, you know, the SMS then sends the client's information later on that, hey, you have your appointment coming up. So anything that is you don't need any sort of user interaction during the data entry, then you can safely choose secondary data entry. Now there are two types of secondary data entry depending on who does the secondary data entry. You can have the data entered by the same health worker who wrote the information on paper or you can have data entered by a different health worker who was going to write an entry from paper to electronic. So when it's a different health worker, it usually is what we, you know, what we refer to as the data managers or the data clerks. So why would you choose a secondary data entry by another health worker? So you can have that enables you to have dedicated data entry clerks and that helps reduce the data entry burden on the skilled health worker. If you have a midwife who is busy, you know, giving clinical support to health workers, she does not want to be bothered by, you know, entering the data at the end of the day. If there's somebody who is less skilled, they can do it. After all, you can, you know, maybe have only one access to two midwives in a sub districts, but I mean, you can always find five people who have some basic IT skills or train them to do that. So you want to protect your skilled health workers and make them focus on doing the clinical specialized training work that they have been trained on. If your clinical health worker has inadequate or low tech skills for doing data entry, we had this challenge in a Bangladesh eRegistry implementation pilot that we did, the Norwegian Institute of Public Health, where the midwives are people who have been in the system for a really, really long time. A lot of them were actually on the verge of retirement. And so they had no interest in trying to learn this new skill. And so, you know, that was quite a challenge for us in trying to figure out how to get them to do data entry. So if you have such, if your focus is antenatal care and you need the data that a midwife is entering and you don't need, you know, clinical decision support, you might consider having the data entry being done by someone else. Because otherwise, your implementation can be challenged by low data entry because this health worker just cannot. She doesn't have the time, she doesn't have the skills, and you spend so much time training her, you might as well give it to a younger less clinically skilled but more technically skilled person. You can also look at the volume of patience and the workflow and see how it makes it challenging to do real time data entry. So for example, in Bangladesh, we chose the community level clinics. They don't have huge volumes of people. So you can take your time and enter the data into the system while the client is there. You might want to avoid real time data entry if you are at a hospital, you know, at a district level or higher, you know, at a higher level of care where you have high volumes, you just, you know, may not want the person who's doing the care to take on that burden as well. But you also have to take notes. Some people may lose their jobs if the same health worker is doing data entry. You know, you have all these data claps who might then become obsolete. But actually, you could repurpose them to be the technical support to the health workers because, you know, most of the time, the health workers are not as tech savvy. So you could repurpose them to be technical support. They, because, again, as we said, for track time implementations, if you want to go to scale, you need a lot of people to give you the support for the many users that you now have. So even if you have data entry by the health worker, you can use these data clerks to then, you train them to then be the technical support for the health workers. So we have some challenges with secondary data entry by a different health worker. You can have issues with errors. The person can read the writing. They usually are not, you know, clinical staff. So they don't understand some of the technical terms. And sometimes they are not skilled enough to recognize errors. You know, somebody's handwriting may make the blood pressure be four digits, or it will look like four digits. And this different health worker, usually the data clerk, will not recognize it and just put in four digits. And if you don't have an inbuilt program rule that detects the correct range of blood pressure, that error is going to go in. You also have that the health workers don't have ownership of the system. And therefore, they don't trust the reports that are generated by the data entry clerks. I remember when I used to work in Ghana, you go to meetings and the clinical, the data people present, what has been done. And it shows that, let's say, the maternal unit is not doing as well. And then the mental unit shows up and says, we are doing very well. In our clinics, we know that we have done this and this and that. And therefore, the problem is with these people who are entering data. We don't know how the edge of the data. We don't have any insight into it. So that's not a problem. And therefore, how do you then proceed? If you cannot convince the people who are actually doing the work that there is a challenge that they have to address, then you will never achieve your goals. You always will have problems with your donors and supporters. So there are certain mitigating measures that you can take. You can create option sets to reduce errors from typing in data. You can train the people who are entering the data to give them some clinical training for them to better understand the clinical data that they are entering so that they can more easily recognize errors. You should also have robust validation procedures. Some of them built in while you're doing data entry. And some of them done at the end of data entry. There are different validation features within DHIs, too. Then you should also have a process for the skilled staff who generated the information, the mill to bribe, et cetera, to review the data before final submission. And so that at the end of the day, when all the data has been aggregated to national level, they then can't come and say, I have no idea why our indicators are so low. I don't have anything to do with it. No, you signed off on every piece of data that was entered into the system. So if we look at real-time point of care data entry, these are some of the key considerations that you have to consider to decide if you're actually going to do that. It has to work all the time. It has to work all the time because the health worker is using it while the client is there. And if they are having challenges with it, the clients will become frustrated, the health worker will become frustrated, and you are going to have clients not coming back and your health care indicators are going to go down. You have to have reliable power and you have to have good internet connectivity so that it is working all the time. You are going to have increased costs. As we mentioned, if every single health worker has to have is dealing with a patient, they have to have their own device because if you have three patients who are seen by three different health workers at the same time, you need to have three devices as opposed to before when all the three health workers were just right on paper and then at the end of the day, the one data click and says the data. You have your users that you have to train, more users that you have to support, and that is a lot of work and that is a lot to consider. So when you think of all these challenges, why would you choose real time points of care data instead of secondary data entry? So it's because it provides user interaction, right? And if you need, for example, clinical decision support, if the reason why you are doing this particular type of implementation, this complexity of implementation is because your health workers are not following the clinical guidelines. They have them in books, but it's difficult for them to refer to those books when the client is in front of them. So if you have it built in as they're entering the data, then you are sure that everybody is following the same rules and following the same clinical guidelines that you are doing, that the country has decided on, and also for that particular cadre of health workers. You have program rules that guide data entry to help reduce data entry errors. You have data validation right there. You don't now have, when the data gets to the district level and then they realize, oh, there's a problem and they have to trace who actually entered it, et cetera. You, if a client tells you information that you didn't hear correctly and you put into the system, the system can then point out to you and say, wait, this doesn't seem right. You, the client's date of birth that you entered makes her 11 years old. And this person is supposed to have had chronic hypertension. I mean, there are people who can have, who can be 11 and pregnant, but let's say somebody showed up and is, you've entered 60 instead of 16, for example, they can just front you and say, hey, are you sure? It's unlikely that a pregnant woman will be 60. And then you can, you know, you can make sure that that is entered correctly and the person actually is 60, which does happen. Then you can go ahead, but at least that buffer is there to make sure that you reduce the data entry errors. If you have actions that you want the health worker to do based on the clinical information that they entered, for example, if you want to generate risks and tell the health worker about it so that she, you know, counsels the client or makes a certain decision, you have, as I keep mentioning with blood pressure, if you enter the blood pressure and the client then, the system can tell you that the client has hypertension. If you put in, you know, data that, for example, the baby is breech, you can have the system tell you that this client has to deliver in the hospital and you can then decide that the client, the system can also tell you that you need to refer this client immediately or within a week, you know, depending on what the situation is to the next level of care or this client, you know, has to deliver at the next level of care and not at the community because, you know, her situation is so complex. So it gives you all these recommended patient management actions and if you're doing secondary data entry, by the time this information gets you, you know, the person is gone, right? The person, the health, the client who was supposed to get this information is gone. So that's one of the reasons why we would have real-time point of care data and you can also have SNSs that the client needs to get during the visit if you need it to but, you know, usually that is not that common. It usually is about health risks, referrals and recommended patient management actions. So let's take a break for questions before I go on to the eRegistry and the clinical decision support.