 So, we're going to go ahead and start this session. I know some people are still wandering in, no problem. So my name is Rebecca Potter. I'm our health domain lead for the DHS to implementation team. And I'm here today with a number of colleagues who are going to share multiple decades worth of experience on implementation lessons learned for integrated HIS. Of course, since 2017 HISP was designated as a WHO collaborating center for innovation and implementation research and health information system strengthening. And so my doctor, my colleague Dr. Andrew will be joining us shortly leads this this collaboration through the WHO division of data analytics and delivery for impact so some of those DDI colleagues will also be joining us today. So at the core of our collaboration is the shared principle of the integrated health information system. And so what we aim to do is break down data silos, ensure that we're sharing resources that we're making this kind of data as efficient as possible as democratized as possible. And we focus on things like like governance and these other mechanisms that make it possible for these systems. Without further ado this session will focus on lessons learned through our implementation research and new tools that we have been developing to support the integrated HIS approach. So it is my pleasure to introduce Nora Stoops, who is an original member of his, and she has been supporting integrated HMIS for 28 years. She comes with a really special view as a nurse who then got assigned as a public public health monitoring and evaluation officer in post apartheid South Africa. And so with this experience she has also been supporting countries for many, many, many years to adopt this integrated approach. And so she's going to be learning lessons, sharing lessons learned from this experience over the last 28 years, including most recently some of the work to revamp the HMIS and Somalia. Let's see how this goes. The other way around. I wasn't looking in a mirror. Testing, testing, any, many, many more. All right. Let's start. I, where do I point this? Or is it not. Can I use this? Did you check it before? Here's the slides that we're using. Okay, can you do, can I ask you to do enter? Okay. That's fine. All right, so I know that in these presentations, we are supposed to only give good messages and you know, challenges, problems get called challenges and get a positive spin. But I think if we don't explore why we've got a problem and how do we know we've got a problem, it becomes very difficult to move on. And these are some of my thoughts on why our HHS can fail. And you go to many DHIS trainings. And it's the DHIS techie team that are there. It's really sold in the program team. So who owns the data? And the program people say, no, it's not my data, it belongs to them. We frequently see this pattern is changing, that the interpretation and reports are written by the information officers. Vertical programs want special things and you can totally understand that. So they set up parallel systems, because the formal system doesn't work for them. The system is inflexible. WHL changes their guidelines on HIV or art or TB, but we still stuck in the old system. So we have duplicates. People ask for data without considering the resources and that's one of our biggest issues. Extensive disaggregation of data by age and gender. What are the things that stop us having a good RHS. A lot of research methodology is put into looking at routine data and PMTCT is the best example going. How many of us have seen clinic pictures with a table dedicated to the registers. A handful of longitudinal registers, they look very nice. They have pitfalls are collecting clinical data. For your diabetic patients, asking how many of the diabetic patients that you saw today have no pedal pulses. That's not management of diabetes. That's clinical data. The donor demands and this is becoming more and more apparent that the donor demands need to be revised dramatically. Because our routine system works, we end up throwing everything into it. Because you can get something from a form at the end of the month. No thought given to what are other ways, better ways to collect data. And it may seem weird, but I do know of countries where because of the way that all hierarchy is structured in the DHAs too. You don't get all the data coming through for the same level because hospitals have been put on their own little castles. If you go home with nothing else, imprint this slide into your brain cells, burn it in. The HMI is the cornerstone of the information policy and planning in a country. District management or health management information systems, the people and the processes and eventually the software provide the mechanism to monitor this change from policy into action. My best example is vitamin A. We never gave children vitamin A. We now count the vitamin A. When we collect the state on a routine basis, it's an RHS. And so what is ever is decided by health management to implement is and reporting. The other collection tools are based on this. The HMI is RHS enables districts, facilities to assess whether or not the goals, indicators, targets, annual performance plan call it to her, are being achieved. That is the aim of an RHS. Some guiding principles. Use the WHO routine data standards as a basis. Trick question. This one at the top. What WHO facility guide is this one at the top come from? What is that one come from? WHO facility guide for managers. Malaria is a guideline on malaria. What are your routine indicators or MNCH and immunization? We need to look at those. If you collect any data twice, if you collect it weekly, do not collect it monthly. And it goes the other way around. Never ask for totals. Never put a column called total on a form. Five and five always makes 18. So which is the wrong figure? It's not necessary the 18. It must be based on a minimum data set. Less is more. No data is collected, which does not form part of a derived indicator. I will give you two or three count indicators, but that's all. Indicators must have targets. RHS data lends itself to activity data. Doses given antinatal visits. If you have to include specific program requests, it's a balancing act. And status data is sometimes very difficult to collect. Do you have a power source? Does your telephone work? Those sort of things. Not always easy to collect. What is a successful RHS looks like? It only collects data used for indicators and targets. You need to set gender disaggregation unless crucial. And I know that that's a controversial topic. And I know that there are certain things where you do need the gender. Campaign and routine data are reported separately. Campaign data must be reported. Please don't dump it into the routine box. Population data is available to the lowest level in age cohorts. If you don't have it, your RHS must be updated. I've got so many seen so many countries now. We did it about five years ago. Do you not ever ask questions about the client's intention? Are you going to breastfeed after delivery? Or what happened yesterday? One of my favorite countries has a question. What are three meals a day which consisted of a minimum four essential food components? Consider other types of data that you can collect. Record reviews, Sentinel sites. When did we ever see somebody doing a record review or a Sentinel site? Sentinel sites maybe, but no. Data quality, just some ideas. If the data is not good, you can't use it. Data improves when there's less. Third bullet point. I've never seen this in any DHIS to implementation plan. Three months after implementing something, you must have a workshop to go through what has been captured. All of they made interpretation type mistakes. You've got to fix it then and there because otherwise those mistakes continue forever. You never have a yearly data cleanup workshop. And can we collect problematic data in DHIS too? Are there the tools available for us? Is tools to tell us about statistical variations and outliers and missing them? But are there tools to help us fix the identified wrong data? DHIS, WHO tool kits must be basic language for all low and middle income countries. I did some work on it now with color, did some work in a country in West Africa. And we managed to reduce the data collection tools, the routine monthly data collection tools. It can be done. I helped Somalia with the HMIS revision. We used the WHA data standards. The death causes for neonatal and maternal mortality were too complex and I said, ouch. We reduced the age desegregation wherever possible. We revised the hospital form. Yes, hospitals. Do we have anything going on about helping improve hospital management information systems? I don't see anybody ever talking about it. That worries me. Hospitals spend the most money. Separate HR. And I want to end off with a slide. I'm not going to read it to you. It stands by itself. And we need to make sure that we are using all our resources in the best possible manner. And that we're doing the right things. That's all from me. Thank you so much, Nora, for sharing your decades of wisdom with us here. Will someone else? I'll switch it over, yeah. So I'd like to welcome Dr. Louis Tina Day of the London School of Hygiene and Tropical Medicine to present the preliminary results of the impulse study on newborn indicators. Well, good afternoon, everybody. And thank you very much, Nora. And thank you very much for this opportunity to present at the DHS to annual conference. I'm going to be presenting on behalf of my colleagues on the impulse study. And the title of our talk is newborn and still birth data quality and use some preliminary results from our study in the Central African Republic, Ethiopia Tanzania and Uganda. And it's brought with me and a mannequin of a small baby just to remind us of about whom we're collecting this data. So here are my colleagues. And this is collaborative research from the London School with myself and Marzia as the copie. Our implementing partner is doctors with Africa quam. So in Italy, Francesca and Giovanni and Central African Republic Usman in Ethiopia, free and Mary. And I'm going to be presenting on behalf of my colleagues at Africa Health Institute in Tanzania, calling out Jackie and Donat, and also in Uganda Macquarie School of Public Health calling out Ronald and Peter. They're going to do four things this afternoon. Oh, sorry, beg your pardon, I forgot to mention, and we're really grateful to our national advisory groups in those four countries, and our international advisory groups, and including people from the University of Oslo, So I'm going to do four things in this short talk, talk about why focus on newborn and still birth data. Talk a little bit about the new E and mini tools as this session is about new tools, and then share some lessons we're learning from impulse. And then just invite you to join the conversation about what intervention should we test in phase two. So firstly newborn and still birth data. A month ago this report was published. And the top line of the key message is this preventable still birth and newborn deaths remain extraordinarily high newborn deaths account for more than 50% of under five mortality now 2.3 million newborn babies die each year in our world and another 1.9 million newborns and still birth. When we look at what priority actions should we take data improvement and use at the bottom is there and this is really how it's exciting to be together and think about how can DHS to contribute to improving data further for newborns and still births. In the same report there's a figure that looks at some of the core indicators and what proportion of 105 countries are able to report these indicators and their routine health systems and you can see it varies from 90% of countries for some indicators right down to 40% for the newborn interventions. The newborn mortality is a sustainable development goal and we've got seven years left. And we really need this data to understand how to take priority actions. So in the every newborn action plan there was a measurement improvement roadmap, and a couple of studies in birth was a valid indicator measurement validation study. In the first two, one of the outputs of this study was funded by USCID was the E and mini tools I'm going to talk about in a minute. And then finally the impulse study is a two phase intervention study which I'll share some of our early phases from phase one. I'm going to talk about using data for action which is I know what's a motivator for everyone in the room in order to end preventable newborn death and still birth, but also for babies not only to survive but also to thrive. So what are the new Ian mini tools. These are global goods freely accessible tools on the data for impact website they were launched in 2022. So a collaborative development by this team in Bangladesh in Tanzania and London school and with data for impact, and the impulse study contributed to version two of these tools. We get newborn data from different sources from population based surveys from civil registration. The many tools are about optimizing routine health information system data like DHS to say that the data can be used for reviewing performance and policy and action. The many tools try to get that conversation up the data pyramid between the health workers we've just been hearing about in the facility right up to all the data users. We're going into three groups map use newborn data improve newborn data quality and essentially they guide priority actions to improve availability quality and use of newborn and still birth routine health information system data. And they're in support of all these guidelines that we've just been hearing about there it is that guidance for our men CH managers that Nora just shared with us. I said the tools are groups around these three areas. And there are currently seven tools. And just starting with a map tool. This helps us look at what data is in the system where the gaps are, and what's the measurement burden. We have enabled Excel, the indicator definitions are already embedded but there's also flexibility so countries can add indicators are so important to them. You map all the different layers so the electronic like DHS to but also the summary forms and the tally sheets and the routine registers where the often the data comes from. Having mapped, you upload the Excel onto a shiny app site and get this editable report, which I'll show you some results from. We used to be using it an impulse study. The rest of the tools are really built around this virtuous vicious cycle of data use and data quality. And they function at the health facility level and also at the sub national and national levels and different tools are relevant for different levels. There's essentially an adaptation of the prism series prism being performance of routine information system designed by measure evaluation about a decade ago which comprehensively assesses the RHS. We adapted them for the priority core newborn and stillbeth indicators and we've also tried to make them as automated as we can just to make it as easy as possible for for national uptake. One conceptual framework is to do with improving the health system for health outcomes and the how the health information system contributes to that. And they divide up the health information system into inputs processes outputs and then outcomes. We've been using this conceptual framework and to visualize the many tools and also the impulse study. Look at the use tools are four of them. And they help us listen to the users of routine data and explore electronic HIS and the improved tools there are two of them, and they use routine data quality assessment methods. So, and when I said we tried to automate them we've, we've updated all the tools using ODK survey CTO actually, and so they're all these they're ready to download off the off the website and can be captured on mobile phones or tablets. And those forms can be uploaded into another macro enabled Excel which generates these report ready tables. There are more than 200 tables that get generated, but also some figures that are useful in writing reports. So actually in summary the many tools are that designed for national uptake that open access and with digital data collection platform and automated reporting the emphasize newborn and still that data at subnational and source health facility level the registers aggregate data and so on. They promote data for action for every newborn to survive and thrive. And they're around those three topics map use and improve, and they align with the score, the WHO score principles, but also of course with all the DHS to work that you're involved in. So what about the impulse study. What lessons are we learning from the impulse study. I've already introduced you to the team earlier. And we have a website where you can follow follow the progress of this study. We started by doing a systematic review looking at newborn data quality we identified 19,000 studies, among which we could only include 34. And there was very little about individual case notes data there was most of the work was about routine register aggregate data. There were many different ways of measuring data quality in these 34 studies, and the data quality itself was very heterogeneous with, you know, wide ranges in completeness internal and external consistency. And then there was limited evidence available for many of the really important data elements such as gestational age, and the, the countries where this research was done was very limited and case notes was underrepresented. So impulse then refined our objectives into the, the two phases phase one and phase two and I'm going to share some results from phase one today. Phase one is being conducted in four countries with the colleagues I introduced you to earlier, Central African Republic, Ethiopia, Uganda and Tanzania if there are people from those countries here I'd love to meet you and continue the conversation. We're in 15 regions in those countries and I'm going to 146 sites between health facilities and data offices. And today our results are around the first 76 sites, and about 170 respondents health workers and data professionals. So, and we're using the many tools for our baseline data collection for phase one. The first objective of impulses to map newborn indicator data availability and existing systems. And this is the automated report I showed you earlier. And when we look at I'm just going to show you examples from our country so comparing Tanzania and Ethiopia here on that mapping report for the electronics a DHS to is the system both countries are using. And then the numerator denominator and full indicator whether it's available or not. You can see in both countries interestingly preterm birth isn't there. And then, you know, among the other indicators there's quite a bit of variability. So it also has this section where we look at it really resonates with what Nora's just shared about, you know, don't collect more than you need to. So this is looking at among these registers, what proportion of the data elements are used for core indicators, as described by W. And you can see in Tanzania, for example is, there's more data elements collected than needed for core indicator measurement in Ethiopia, it's a little bit more balanced. The second objective is to assess data quality for these newborn and still that indicators. So we've looked at it in two ways we've looked at the registers, but also the case notes, but typically at the moment data is coming out of the registers but there's interest in measuring quality of care and that doesn't really lend itself to more more more columns being added to registers so the case notes where health workers right their clinical care is much more potentially useful. So here's an example of a structured case notes from one of our countries, and then a register that we all know. It's just really important to keep thinking about how those two different data sources relate to one another. And there's, you know, to reduce the duplication of effort. Impulse as I've said contributed a version two of the many tools. And with the tools already existed in English and Swahili and there's now translations in Amharic and French. And we've added on a new tool tool seven so there are now eight tools. And that's about the individual case notes. We have a data quality in the registers for newborns and still births just looking at the denominators we have to measure total births and also live births. And you can see here in this figure that once it starts going up the system from reports into the electronic health information system I mean the, the consistency is fairly good but if you look at the right at the bottom, the completeness of the register. And this is from the pilot data in Tanzania was only 88% so I mean even before you start pushing it up, you know you've got a data quality gap. We look at the numerators and this is from the impulse study for 76 sites, looking at eight numerators for newborn and still birth. Again, the completeness is actually not not so bad, but the timeliness and this consistency it really really drops. We're seeing really mixed picture of where the gaps are and where to intervene. Interestingly, our respondents, 48% of them indicated that they were aware of data manipulation for various reasons taking place. There've been a number of publications about that in recent years so we decided to ask and so far that was what they're telling us. Let's look at the case notes with this new tool, and we've split the variables we're looking for the clinical information really into admission history admission examination and discharge. And among 59 pieces of clinical information that we as doctors need to look after newborn babies, we're finding it's legible 99% but completeness is very variable. We're also showing Ethiopian Uganda again preliminary results, but you can see there's a wide range in data availability for example gestational age to measure preterm birth is less than 40% in both those countries. We're also looking for proxies of data use, and this internal and external consistency with registers, and also looking at how does the design of the case notes affect data quality. We're looking at some of the other determinants from the prism framework to improving data quality. It was really striking this is in the Tanzania pilot, that the district was performing much more strongly for efforts to improve data quality maybe identifying a gap that we haven't thought so much about the source data at the health facility level. The third objective in impulse phase one is to understand data use by different stakeholders and this is a colleague picture of my colleague us implementing the many tools in Central African Republic. So you can't read all these numbers but I just wanted to show them to you to show that there was really a gap across all the measures that we are capturing for data use the highest measure and anything we can find is 74% and much as a lot and so there's I mean as everyone in the room is aware there's a big challenge with data use. And again, particularly at facility level, whereas the district there's some evidence of visualizations and so on in the facility level it's really very much less. So we're looking at some looking at a score and the many tools has a couple of schools in it and combining 10 components of evidence based decision making. We're really seeing clustering, I mean around 40 to 75% across all levels, whether we're talking about the data offices or the facilities across all the countries. The next objective is to analyze those technical organizational and behavioral factors to improve data quality and use for newborns. And again just reminding us of this prison framework we're talking about this section the inputs of the RHS. If we think about that promotion of a culture of information. And again just showing examples from Uganda and Tanzania. We've already talked about the lack of evidence based decision making that's the lowest scoring component in both countries, but essentially all of it is around that 30 to 70% again, similar in both countries. And then if we look at resource availability we just again had more talk about this a few minutes ago. Everyone has a delivery register which is encouraging. And but when we look at the availability in over the last six months off to the right is a little bit more vague but look at this kangaroo mother care. Very very little paper based register availability but yet that's an indicator that's really being promoted but where are we getting the data from. And also death registers are very very variable in the sites that we've been visiting. And then we looking we have been exploring availability of internet and electricity both needed to digitize our data. We're finding again very very variable 20 days or more of internet access in a month and very varied and up to 30% of the equipment isn't isn't working. The other gap we're finding is in health worker RHS education training. There seem to be plans, but then as you look at the different actors in the data pyramid, not everyone who's being asked to do RHS tasks is trained to do that between data capture and report writing. And then around the topic of RHS motivation, a behavioral factor. Again this is one of the scores that the EMI tool captures, and we're finding it's clustering about 60 to 80% in terms of respondents reporting they feel motivated for RHS tasks. And again it's very much across all levels and in all countries. So we've got the opportunity to go one step further and to really dig into the data and really explore which components of those scores contributing the most and we're doing that analysis at the moment. Another thing that EMI measures is that we call it the confidence competence gap so people report they're able to do something but then we'll actually have a little like a role play a little scenario, and then we measure can they do it and again this is from Tanzania, some pilot data. We found very large gaps between confidence and competence between one and 47% gap. So we've asked all our respondents what do we think about RHS, do they think anything needs to change. And what do they suggest and we've got literally thousands of ideas that we're working through at the moment that there are just a few listed here, but we really want to ask the users and the health workers the data health professionals. What do they want to, what, what do they want to change. So far in post phase one what are we learning and using the present framework as our conceptual model, we're finding gaps across many, many of the determinants technical organizational and behavioral, which are contributing to newborn and still birth data quality and use. But of course there are higher performing sites and respondents. So we're really digging down to try and identify what those are to see if we can incorporate that into an intervention to test. So, as I've shared with you, and the aim of the impulse study overall is to improve newborn data qualities still birth data quality and use. So phase one we've been using the end mini tools as a, as a baseline assessment to look at the current situation among these 146 sites. We're in the middle of data analysis now, both quantitative and qualitative and asking end user ideas to improve. And we're working towards a peer reviewed supplement and the picture here is my colleagues who are who are working on that right now. But I just want to leave us today with this question, what intervention should we test for phase two. Obviously it'll be driven up from what we find in phase one but I'm just aware that in the room there are many many experts who've been thinking about this for years. So I'd love to just invite you into that conversation. We've said in phase two that we want to design an intervention and then test it and that testing could be effectiveness cost effectiveness we can test it in different ways. And we want to design an intervention that's co created with our national advisory groups, international advisory groups, but also we're really welcome the DHS to community to this conversation. Some of the questions we've been puzzling about is, how can we show frontline health workers their value in the data they capture that's pushed up the system. I loved it in the role play when the community health worker this morning said, I've set my feet mice my data, I hope I get some feedback as to whether it's useful or not. How can we strengthen that part. Do people here know of educational materials for RHS competencies for health workers that perhaps we could use and use for newborn and still birth data. What about visualizations at the health facility level is, including for data accuracy. It seems to be a bit of a gap then just wonder if there are people in the room who can advise us on that. How can we strengthen that information culture so that that motivation for really important data tasks increases data isn't an end in itself. I'm a clinician so for me data is so totally linked to that kind of quality of care piece for the baby that I brought along today, but how can we just get that balance right for health workers, providing care and also documenting the care they've provided so that those two things really resonate with one another. So today, I just them have shared with you a little bit about the importance of newborn and still birth data please whatever you do, please think about how we can strengthen newborn and still birth data. I've talked a little bit about the many tools which are available for your use on the D4 data for impact website. I've shared some of the lessons we're learning in impulse phase one on behalf of my colleagues and who I think a lot of them are online so greetings to you, and just invited you to help us work out what intervention we can test for phase two. Thank you to all the health professionals and data professionals who participated in our phase one data collection and special thanks to our study team, including all the data collectors, and to the KZ Foundation who are funding us. Thank you. Thank you so much Louise Tina I just I really appreciated this abstract because it is so rigorous and goes into all of these kinds of issues that have also been raised by Nora. I know it can sound like doom and gloom but it is really not we have many stories tomorrow about what is the impact of the use of these types of routine data so there is a session tomorrow impact and effective use. But I think a lot of the lessons learned here and what we need to pay attention to is the rationalization of these routine health information systems, and particularly because they are expensive. I mean it's expensive in terms of connectivity, I mean expensive in terms of the health workers time. So approximately 30% of these health workers are spending their time documenting. If they are documenting data that is not being used for decision making that is time that they are not spent providing better care. So I think these are lessons learned to take in the future. So this is my pleasure to actually invite our next colleague, who also has a clinical background has now turned into an HIS expert. Also sitting on the same team as myself so Stefano Parotti who is a DHS to implementation expert in the health domain. So thank you very much, Rebecca, you can hear me. So, yeah, I mean it's very inspiring. And what I'm going to talk with you today is about health facility attributes was something that was already mentioned in the previous in the previous presentation, and why this aspect, like, they run. They should not run. Sorry. Anyway, and how this is this this aspect should be really considered to be integrated in a routine health information system for improvement of health. The performance indicator so when I'm talking about health facility attributes say something that it came up several times during our work as well. So in the previous presentation and when we're doing the revision of the different abstract that we receive this year is something that is already present so this type of information information that are collected by different platform by different in different programs. So now we are going to see why this is very important to be integrated in one routine health information system. So what are these health facility attributes and facilities attributes really in the practical are semi permanent data that are collected that are specifically for health facilities so for example the key information related to human resources to availability of, of stuff as well to train stuff training availability of the different and provision of different services availability on the logistic. So this is something that, as I told you information that is present information that is well known normally at the whole facility level but sometimes missing these steps at highest level. This treat the national level to be able to be to be analyzed this data are already being collected that are coming for example they can come from the very well known master facility list. Maybe they can come through other vertical programs. So here is an example of what normally is is being collected the availability of key equipment and essential medicine so everything related as well to pharmacy. And infrastructure, electricity availability internet availability as was shown before the preparedness of our whole facility of our health staff about the response to any type of routine and emergency cease circumstances, and availability clearly for specific services and how is this data collected so they're already some standard tools that are provided for example by WTO like the harmonize facility assessment the HFA that this is a survey that is already ongoing in different countries in which the information are collected are core and optional question related to availability, readiness, quite of management and finance. This type of information is collected as a survey. So normally is an external person from the health facility that is coming and collected is information. The information are collected for example for HFA are very complete information so normally when an HFA is done in a country is not done for every whole facility, but just some of them are picked to be able then to represent a specific area and a specific type of facilities. As I said, there is as well the E-RAMS platform. Normally the E-RAMS platform is used for collecting information specifically at the moment in emergency settings. So the information are collected are very specific for, okay, number of facilities are open, closed, wider, open, wider, closer service availability for key specific services. So during this year we have been working with, in collaboration with Global Founds, with WHO, with GAVI, about on the production of what we call a global toolkits. So to be able to provide a kind of a starter pack for the different countries to be able to collect this key information. I don't want to go into details here because finally the information that normally are required to be collected. We try to standardize with the tool that are already available like HFA or E-RAMS, but I would like to have more focus on the, how these HFA modules and HMIS they are related. So, first of all, HFA, so every type of information that is collected, related to the attributes of our facility doesn't replace other standardized tools as HFA. Also because the type of information collected and the use of the information is very different. Normally when we say, when we talk about HFA, our information that are collected by the whole facility. We don't have an external person coming inside the whole facility to do a survey. Normally this information are semi-permanent information. Okay, so information that should be collected, not in a monthly basis. Maybe it can be collected once per year or twice per year. Or it can be collected ad hoc. For example, there is any change for specific emergency situation in that specific area. And the idea and the added value to this information is the possibility to integrate them in the routine health information system. So in some countries the information is already collected as kind of HFA, but they are stored in a separate instance. So they are not really analyzing integrated in the routine health information system. So here is the most interesting part for me. So how this information can be helpful at the country level. First of all, to plan resources allocation directly inside the country. So you, you are able to see at the central level. If there is any gaps on the different region, different health facility, for example, on service provision, then you're able to identify any bottleneck and accessibility issue, because maybe, as I say before there is a specific area in which we are lacking a specific area. And then as well to prepare respond to public health emergency in case we have to mobilize in in case we have to decide okay for referral system where we are sending patient where we are going to support specific hospitals etc etc is important to have this type of information. So here are some example. Okay, some visualization that already present some of the tool that we developed in collaboration with the WHO for other ability of services. Okay, I mean, these are very simple information, but they can have a great impact. So we are providing the ability of specific services at country level. We can see if there is any area that is not covered or if there is a very low proportion of health facility that are providing this type of services. Availability of staff. Availability of trained staff. So to be able to, if we have to allocate any resource or for example as well a country level, if we need to orient the different donors. Okay, and where we should have an intervention, then we are able to have it through this information. And the most important one for me is the triangulation. So why we are telling to that the integration of this information our core are very important to have integrated in the routine health information system, because then you are able to triangulate this the health facility attributes with other health facility indicators. Okay, so for example, you are able to see, let's maybe take another example here. For example, in this case we have a visualization on case load per staff. So to be able to see this indicator normally can be used on quality of care that we are providing because we can have a whole city are providing thousands of consultations, but only with three health staff dedicating on this. This will be a basic one but for example, can be more advanced to try to say okay we have a specific mortality, for example malaria mortality, do we are providing service for management of severe malaria or do we have all the treatments in this health facility to be able to take care of these cases. So the ability to have them integrated in a routine information system can enhance this improvement of key health indicators. Here is another example. This is more for an emerging situation. So these images taken from the ERAMS is one of the report that was just published recently, but just to give you an idea, okay, how this information can drive the different donors. Okay, to decide that which are the audience in which we should intervene. What is the situation for specific disease like tuberculosis or connectivity of these health facilities. So as a summary, we say that, okay, health facility attributes are semi permanent information that are collected at health facility level. Information that are easily collected because normally in a health facility, the different manager, they already know what they have in situ, what they have in place or not. Information can be collected through survey, what we suggest and that's why that the value to have in a routine information systems that this information are collected integrated in the routine system. This information should be analyzed with other health key information from the health facility to have to be able to have an impact. The DHS2 is national health information system platform can make it more cost effective for the health facility to be able to update this information in a regular basis. And it can be available for decision makers. So that's all. Thank you very much. Thank you so much, Stefano. And so the second part of the presentation of course is about new tools. So with the support of global funds we have been working on with WHO, developing the core metadata to kind of support this to be able to integrate this into national DHS2 systems. And what we found during this process is actually, everyone is collecting the same data everywhere. The HHFA is collecting it. The Global Fund Pulse Surveys is collecting it, and it is coming all over but then people don't have it when they need it. So the whole idea of these tools and we'll be developing and publishing a toolkit on our website at some point in the next six months. We do have some metadata available if there are any countries reach out to us. We are working with some pilot countries such as Uganda and Tanzania and Ghana as well. And we have the idea of actually making this data available more accessible in the system. So we can stop wasting money running out and collecting it over and over again. So our next presenters, we have colleagues from the WHO division of data analytics and delivery for impacts. So Dr. Doris Ma, who I'm going to call her the godmother of mortality statistics. In fact, she's going to introduce a new tool that has been named after her. As well as Ninad Kostansyak, who is an expert in classifications and terminology to tell us a little bit more about ICD-11. And then we're going to conclude with our own John Lewis from his Vietnam, who is going to share a little bit about how we are actually developing ICD-11 compliance applications to support the proper coding and underlying cause of death for better mortality statistics and use. And I found that incredibly interesting, Luis Tina, that that was actually one of the key gaps that you had found in the impulse study. So I think we have our presenters online, Doris and Ninad. The floor is yours to share your screen please. We shared it now. Can you see it? One second. Can you see our screen? Okay. Hang on just a moment. Good afternoon, everybody. I'm Doris Maffat and I'm working at the WHO statistician and handling causes of death statistics. So I'm here by colleague Ninad Kostansyak. We start the first part of presentation followed by Ninad. So there's one moment I'm going to try to add your volume up a bit. Just give me one second please. Can you try again now? Okay, I go back. Much better. Thank you. Just go ahead and start again. Thank you. Yeah, sorry. I'm Doris Maffat statistician in the Department of Data Analytics working with Anchu and Ninad Kostansyak, who's here with me. So I give a presentation on the role of the health sector in strengthening the obvious system. So the score which was carried out in 2020 shows revealing gaps in CRBS, that is counting births, deaths and causes of death. As you can see, the green bars are only the number of countries, the portion of countries that have full birth and death registration. And when you see the longest, the shortest green light is the fact showing in Africa, Southeast Asia and the Western Pacific where the gaps are most important. When you look at the right hand side on certification and reporting of causes of death, the situation is worse even. And this map just shows where the gaps are, mainly in certification and reporting on the causes of death. And as I mentioned earlier, these are mainly in Africa, as you can see all the red parts. And the issue is that despite many initiatives to improve certification and reporting of causes of death, there are fundamental issues. I think in mind that DHIS2 is implemented in those countries, this is where I got a comment that maybe we could leverage on the DHIS2 system to start the reporting of causes of death. So why are we interested in CRBS? Because it's a benefit also for her. So if you look at that, there are records on the left hand side of the happy function systems, ambulatory care, the EMR, and these are able to give us the birth and death records in fact. And that is linked to the civil identity systems where you have national ID, local registrar office where they will register the birth and the death and get the legal certificates. And this information then are used for other administrative systems and the corporation register, for example, for the voter roster, for the passport, vehicle registry, etc. So why do we say the health sector can contribute? If you look at this map, there are actually mis-opportunities. The bar, the yellow bar showing the bi-registration, which remains very low in most countries, but the maternal health and immunization coverage are very high. The same babies are taken to the health facilities for the routine health services, but then they are not registered. And the worst is that death registration lacks behind birth registration. This is because birth registration, babies need to be registered at birth, because when they reach school, most of the time they ask for a birth certificate for the enrollment, and sometimes also to get access to other care facilities. For death registration, the main issue remains that there are no motivation for people to register their deaths, and also there are not enough medical certifiers available when the deaths occur in the community to certify the deaths. So what are we doing here to improve the CRBS system? The Beatrice contribution is really the development of norm and standards in the collection, reporting and analysis of course of death statistics. For example, we have the definition of what is the maternal death is still above, and we develop the international classification of diseases according to the causes of death, as well as the rules for the section of underlying cause of death. And we design and personalize a collaboration between health and the CRBS system for mutual benefit by leveraging on the health sectors routine or MHCH interventions to increase the registration of births and deaths. So you're advocating for it that whenever someone bring a baby for these services in the health facility, the health worker whether it's there or whether the health worker should encourage or an advocate for the family to go and register their other births. We also developed the data collection tools within the R routine health information system in countries for completion of births and death reports, and we're going to have a demo of this data collection tool later after this representation. We also developed global products guidance from the BHU DSEP on the role of a health sector in improving birth and death registration, and we developed the ICD-11 suite of tools for coding, sectional data and cause of death analysis of course of death statistics, as well as training materials and curriculum to support ICD implementation in countries. So, what type of tools we have developed so far? In 2007-15, I guess it was that we developed the start of mortality module SMOR, but that was based on ICD-10 at that time, and a few countries have started inventing it. But now with the advent of ICD-11, we're moving towards the new app, which is more, I would say, which has more functionalities also and is better tied up to what we expect of ICD-11 implementation in countries that we have them in doing all this work. We had in the middle something called rapid mortality surveillance system. That was during the COVID-19 when you were asking countries to report only total deaths, not about causes of death, total deaths, how many deaths we are capturing every week. And that SMOR package was developed by Oslo Cooperation WHU. So, I will switch on to the floor tonight, we will continue the presentation. Thank you. Yeah, so regarding ICD-11, it came officially into effect last year, and there are a couple of novelties. One of them is out from the updates in the content itself, making it up to date from a clinical perspective. There is, of course, the point that we have now in ICD-11 for the first time integration of terminology in the ICD-11 classification, which means that in addition to the 17,000 statistical categories, we have about 135,000 clinical terms and synonyms integrated. And that makes it, of course, much more clinician-friendly, because clinicians or health worker-friendly find the level of detail they are searching for when they make the transition from the clinical diagnosis or cause of death to the correct ICD code. And the second novelty is that in order to manage that complexity, obviously, when ICD-10 was disseminated, it was disseminated as a book. ICD-11 comes with a whole suite of tools which allow for digital electronic search, but also analysis of the data, etc. So it is primarily a digital tool, and it is disseminated as a digital tool, but it also operates in environments where, for example, internet is not available. You can see here the suite of tools which are characterized for the mortality reporting with ICD-11, and they have been basically integrated in the DHS-2 app for mortality, medical certification, and coding of causes of death. And that is part of the largest suite of tools which you see here depicted, and we basically categorize them into implementation-related tools, as well as tools for training or maintenance. And most importantly, all of these tools which are available for end-user consumption have the ability to be easily integrated into any kind of software application through the ICD-11 APIs. And that is something also which is very different to how ICD was integrated in the past, so it's not anymore uploading a flat list of hierarchical categories and then displaying them as a drop-down list, but it's really consuming the APIs with the search algorithm in the background, which allows you to search and to assign codes using Google-style search functionality and smart search functionality. And that has been basically happening in the integration of the ICD-11 in the DHS-2 app. So, where we have the coverage of the data entry to the data analysis, to the data dissemination, and John will demonstrate this, but let me just run you through some of the key features here. We have in the app on the data entry side, what eventually will become the ICD-11 and WHO MCCT form specifications for the electronic medical certification of causes of death. We have the integration of the ICD-11 coding tool, which is, as I mentioned, the Google-like search where you get from the cause of death, as it written by the clinician or the health worker now to the correct code in the ICD-11. We have the integration of Doris, which is the rule engine for automated selection of underlying cause of death. So what we did in ICD-11 is to digitalize the mortality coding rules. And we have now an algorithm which allows to make this assignment of the underlying cause of death, which in the past was done manually, or sometimes in some countries also using other software like Iris integrated into the realm of the ICD-11. There's also a new feature which will come up that allows even processing not only of coded data in the rule engine, but also processing of pretext so that you have actually these two steps. One is automation of the coding and then subsequently automation of the selection of the underlying cause of death. And then we have data analysis through the integration of code free and code edit so that the ICD-11 code data can be immediately analyzed and then visualized in the respective ways for which I need it. And all of this is customizable to the respective user needs. So this is what I think you will also see in the demo. And I think in terms of next steps just to highlight here that we obviously one of the key big challenges is to facilitate this process of automation of the input data and the cause selection data. So that is something which is still going on and we are working on to refine this and then of course all the aspects related to integration with other disease surveillance applications and also with the mobility data collection. So with that, let us stop here and I think we with this we hand over to John for the demo over. Is that fine? Yes. So like what I'm going to try to explain our show is the cause of death app which we built inside DHRs to. This is a generic app which we want to we will soon put this one in the DHRs to App Store where you can try to download the app and install your own DHRs to instance. So we've been supporting this app from the version 35, 36, 37, 38, 39, 40. So all those versions you can if you are using any of the DHRs to instance near one country above 35, you can use it perfectly fine. So I'll quickly go through the few of the things. Let me just like to go through them. I have already installed this app in the DHRs to instance. So I'm going to see like what are the different features you have. And then at the end when we have some time like I can just like show you how you can try to install the app itself in your own DHRs to see. So like here in this app like you have data entry module I'm just like going through that very quickly and like we just say like how best we are we can do the registration. So I'm just like doing a registration of the cost of that. I just have reported. Data of reporting is today. That was let's just say on a Saturday. Name the reporter. And here you can actually just select the age or the things that means just select is in them. Yep, 26 years, age, male. Not as you can just like leave these things blind. None of these things are mandatory. It's okay. And you just say once you do that one this frame and frame be what you're seeing it's exactly the same as you're seeing in your medical costs of death certificate so are there in the program. And based on your selection by the gender and as well as age, few of the things will appear in frame be. Okay, so now, like here this is the free text, like whatever the clinician enters so you can enter on here, let's just say, very quickly I have noted down the the diseases. And like here when you click on it. So that's when like you're actually looking looking into the choice to recording to. So now like I just say, so the recording is huge email. So like here like you have spread. Oh, let's just say, okay, that's fine. So you can see around here these are all the different details. These are nothing stored in the HRs to. So now what you're trying to do is like the all the list. It is inside the HR, but you're talking to the other website of our city embedded coding to you have the list of all the different details you can see the hierarchy and everything. And then like you can just select. Okay, this is looks fine. So just select this as the cost. Just once again. Let me just select any of these things. It's not right, but oh, sorry. I just want to select the right one because I want to also show you the other door is to gets calculated. There we go. And then we select it's in, let's just say five years. You also have this one is hypertension. The select here pension. And then the last one, which I just like is. Okay. This one. Yeah, so I'm not the ICD coder or a clinician. I just, this is the same example which I use. So now what I've done is just have entered few of the details. So on the free text side is what the doctor has entered on the, the next side is the work by city quarter is entering it is just selecting and searching all the things. Before we had this underlying course of that which people were supposed to see and add by themselves. And then with the door is to, you can click on compute what it does, it will go through the engine. And these are all different full report of the, the door is to and then it will select the underlying cause of that for you. And with the app, like you also have frame B where you can try to enter all the different details that's okay. I'll just like save. In itself you have this is simple medical certificate, of course of that which can be printed and hand over to the people at the clinic level. So that like they can go around, and this can be customized based on the other things. So this is the simplest data entry form so this is already finished. Now what we also what we have is the analysis of the dashboard. These are all the list of all the dashboards which we have created which is embedded in the app itself. So you don't have to, to go somewhere else to do the analysis, because usually like individuals to what happens like we enter the data in either the track capture or a capture app, and then you go for the dashboard or data visualizer to all the So here these are all the dashboards or the charts which is already predefined if people can also go to be a chance to do the further analysis, but at least for most of the things, you already have it around here. You can, if you don't like you can just select it and if you can include more data and all things it's it's here. You can just save for NCD non-communicable list is a frequent cost of doubt, or by the chapters, or by malaria and TB and AIDS combined one you can also just see on here. So these are all the things which which you get embedded automatically in the app itself. After you install you have all these things. I will quickly show on the session two. So for example, if you want to add some other attributes because every country they want to include different things. For example, they want to include the national ID or the addresses or the information about who is collecting the data, those kind of things can be configured by the system admit itself. It doesn't require any programmer or anything to customize this, this application and also including them. Sorry. And also including the money or cost of certificate of death. So this is simplest one you can upload your logo on the side, and then you can just see what are the different fields you want to include or not to include, whether you want to include in the food or the movie you can do that. And if you, if any of the people have like how to build a standard report if you already have a predefined format, you can also upload that one. So that's all the other two things which can be to work, work around. And then is the translation so this is also something which is we want to work on so anytime that we usually the translation is the people entering it, but sometimes you want to edit your own translation for your own local implementation. So here are a list of all the things you can add many different languages. Let's just say, and then like you can enter. So these are all the keys, like what you're seeing in the software so you can just change. And then you can, based on your user settings it will automatically will switch into different languages what you're seeing and then other part is on the unaccord export. So let me just select this, this is something which takes time. There's on the data what you have so here is like what are the different data what you have entered. It will convert everything aggregate and put it into an accord export file, which you can download and use the unaccord analysis to so that you don't have to do the export again in different cases. So all these three things is in built in here. So let me just quickly just show you one more things. Okay. So you just in the dashboard when you log in. So when you go forward this website dhs to dot world slash w h o c o d. The username password is already mentioned in the login screen. So again, you have the functionality in solution manual and the app itself. If you want to try please don't do it in the production instance. We just say do it install it in the demo demo instance, or the development instance, and then like see how the app works. And if you have any problem please reach out to us. And then like you have all the manuals and everything. So just to quickly show you how the installation works is if I can open this one. So here, when you first time install you don't see anything you will just like see this screen, which is here. So it will show you like where what all the different things you can select the IC recording to whether you want to use the WHO one or if you already have installed the IC recording tool in a Docker container. So you can actually point out to which service. And then like you can just like say, you have to select the all the attributes. Where is going. Hold the solution. Yeah, here. So, in DHS to how it works is like you already have all the patient attributes on the fields in in your DHS to we are not making anything duplicate you can use your first name last name gender sex and date of birth. If it's not there you created if it's there you can use it. Sometimes, like the first name is called given name. Last name is called family name so it's up to you in your own local installation you can include it. And if you want to have any other additional information like investigator source name and other things that you can also add in your frame and frame be the additional section you cannot remove anything which is there, but you can add additional things and also patient attribute what you want to try to collect. And then other part is like after doing that. So you need to just say, where are you collecting, which hospital or health center so you select all those things. And this part which most of the DHS to implementation when you do, you're not setting the roles properly. So we usually any program it should have three roles one is admin, that means we can change the one is capture that we can do the data entry and one is only viewing. So these are all the three groups, which we already have predefined it, and then you can select whichever the user who belongs to admin who belongs to capture and will belongs to view. And then it will give you the review and then the installation is done. So it's as simple as that one is so that's, that's why like we want to try to deal with the entry and solution, let's just quickly go through. So this was the expert which I was just saying, we imported a lot of data so that like we can show the dashboard. So this format, what you see here is exactly the same. And you can download it as an Excel and then you can try to use it. So this is basically what I want you to present. Just quick demo, and then you can always like when you let me just log out. So with this one, like you only have we can use the C already demo and then the user and password is already there you can go through it later on see how things are, and like the app, and everything is there will soon put that up in the App Store, so that like you can try to install it in your own DHRs too. Yep, that's basically it Rebecca. Thank you so much to the presenters. Before we close this session, which we will do just a couple minutes early to give people time to transition. I just wanted to give a final reflection to my colleague Dr. So just a quick sum up on especially on the last presentation that link to actually the two presentations one from Stefano on the health facility attributes and then the big one is on the mortality report things. So, I'm just coordinating a loss of the work within WHO on several different programs that implement the standard on routine health information system but a lot of it also support implementation of the HHS. The routine, the health facility attributes assessment I think that is something that coming up, but also that is something links a lot of work on the data use in the country and I really appreciate the presentation from the London school colleagues I think the data use would be something that is important to look into and then that should be what drive the data collection the item, and the design of the data collecting the collection system for the cause of death. As the link with the ICD 11, the loss of country have asked so far that by last week we have at least 11 country asked for implementation of the apps. Lots of different ideas coming up but I think we need to keep on thing it cannot be separate from the training of the first of the physicians on the ICD 11. And that's why my colleagues Doris and Anna, one is on the mortality and one is on ICD 11, and they have to work together. Also, it's important to understand how the country design, who is doing the coding, we're doing the, the certification for for medical cause of death, and how we're going to bring the data together and what would be the final products a lot of it goes into but also what would be the benefit for the facility when they look at the cause of death that is happening in the facility or in the catchment area. So I think this is something we can continue to buy time over the breaks or coffees or whenever and then have a good communication but just something that you know we will be happy to be touched and see how we can support the implementation of the cause of death and ICD 11 because both of them are quite new so thank you and thank you for this opportunity to present here. The final thank you to Doris and Nina who actually got up quite early this morning I believe they're in Barbados giving a training so thank you all so much. With this couple housekeeping in this room we will have DHIS to for health emergencies next coming up in just a couple minutes. There is an ongoing session in auditorium five right now on DHIS to an ncds so if anyone is interested in that, please feel free to move yourselves over. And then the other session happening in auditorium for is the integration tech session and getting data out of DHIS to. So thank you all for joining us for this session.