 I think that's the best one. I think that's the best one. Yeah. Is this working? Hello. That's. That's. That's. These ones. Does that one's really like. Okay. And you're going to end up giving yourself some. What does this go on? So like that. So that way around. Okay. And then just curve that in a little bit. There we go. We'll just make sure that this is on. So that's the mute button. So it will show you just say big. And in black. If the mute button is on. So like that. So if that's worth play that. Hello. Hello. Hello. Hello. Hello. Hello. Is this working? Thank you. I'm getting positive feedback. Excellent. So we're going to go ahead and start the session. I know some people are still wandering in. No problem. So my name is Rebecca Potter. I'm our health domain lead for the DHS to implementation team. multiple decades' worth of experience on implementation lessons learned for integrated HIS. Of course, since 2017, HISP was designated as the WHO Collaborating Center for Innovation and Implementation Research and Health Information Systems Strengthening. So my colleague, Dr. Aang Choo, who will be joining us shortly, leads this collaboration through the WHO Division of Data Analytics and Delivery for Impact. So some of those DDI colleagues will also be joining us today. So at the core of our collaboration is this shared principle of the integrated health information system. And so what we aim to do is break down data silos, ensure that we're sharing resources, that we're making this kind of data as efficient as possible, as democratized as possible. And we focus on things like governance and these other mechanisms that make it possible for these systems. So without further ado, this session will focus on lessons learned through our implementation research and new tools that we have been developing to support the integrated HIS approach. So it is my pleasure to introduce Nora Stoops, who is an original member of HISP, and she has been supporting integrated HMIS for 28 years. She comes with a really special view as a nurse who then got assigned as a public health monitoring and evaluation officer in post-apartheid South Africa. And so with this experience, she has also been supporting countries for many, many, many years to adopt this integrated approach. And so she's going to be learning sharing lessons learned from this experience over the last 28 years, including most recently some of the work to revamp the HMIS in Somalia. So let's see how this goes. Wasn't looking in a mirror. Testing, testing, any, many, many more. All right. Let's start. I, where do I point this? Or is it not, must I use this? Did you check it before? Yeah. Oh, here's the slides that we're using. So it's not okay. Can you do, can I ask you to do enter? Okay. Yeah, that's fine. All right. So I know that in these presentations, we are supposed to only give good messages and, you know, challenges, problems get called challenges and get a positive spin. But I think if we don't explore why we've got a problem and how do we know we've got a problem, it becomes very difficult to move on. And these are some of my thoughts on why our HHS can fail. And you go to many DHS trainings. And it's the DHR is Tiki team that are there. It's really sold in the program team. So who owns the data? And the program people say, no, it's not my data, it belongs to them. We frequently see, fortunately, this pattern is changing, that the interpretation and reports are written by the information officers. Vertical programs want special things. And you can totally understand that how they set up parallel systems, because the formal system doesn't work for them. The system is inflexible. WHO changes their guidelines on HIV or RTB, but we still stuck in the old system. So we have duplicates. People ask for data without considering the resources. And that's one of our biggest issues. And extensive disaggregation of data by age and gender. What are the things that stop us having a good RHS? A lot of research methodology is put into looking at routine data and PMTCT is the best example going. How many of us have seen clinic pictures with this table dedicated to the registers? Be careful of longitudinal registers. They look very nice. They have, but false. Ah, collecting clinical data for your diabetic patients asking how many of the diabetic patients that you saw today have no pedal pulses. That's not management of diabetes. That's clinical data. Donor demands. And this is becoming more and more apparent that the donor demands need to be revised dramatically. Because our routine system works, we end up throwing everything into it. Because you can get something from a form at the end of the month. No thought given to what are other ways, better ways to collect data. And it may seem weird, but I do know of countries where because of the way that all hierarchy is structured in the DHAs too, you don't get all the data coming through for the same level. Because hospitals have been put on their own little castles. If you go home with nothing else, imprint this slide into your brain cells, burn it in. The HMI is the cornerstone of the information policy and planning in a country. District management or health management information systems, the people and the processes and eventually the software provide the mechanism to monitor this change from policy into action. My best example is vitamin A. We never gave children vitamin A. We now count the vitamin A. When we collect the state on a routine basis, it's an RHAs. And so what is ever is decided by health management to implement is and reporting so your data collection tools are based on this. The HMI or RHAs enables districts, facilities to assess whether or not the goals, indicators, targets, annual performance plan, call it to of being achieved. That is the aim of an RHA. Some guiding principles. Use the WHL routine data standards as a basis. Ah, trick question. This one at the top. What WHL facility guide does this one at the top come from? What does that one come from? WHL facility guide for managers. Malaria, there's a guideline on malaria. What are your routine indicators or MNCH and immunization? We need to look at those. Do not collect any data twice. If you collect it weekly, do not collect it monthly. And it goes the other way round. Never ask for totals. Never put a column called total on a form. Five and five always makes 18. So which is the wrong figure? It's not necessary the 18. It must be based on a minimum data set. Less is more. No data is collected, which does not form part of a derived indicator. I will give you two or three count indicators, but that's all. Indicators must have targets. RHAs data lends itself to activity data. Doses given, antenatal visits. You may have to include specific program requests. It's a balancing act. And status data is sometimes very difficult to collect. Do you have a power source? Does your telephone work? Those sort of things. Not always easy to collect. What is a successful RHAs looks like? It only collects data used for indicators and targets. No sets or gender disaggregation unless crucial. And I know that that's a controversial topic. And I know that there are certain things where you do need the gender. Campaign and routine data are reported separately. Campaign data must be reported. Please don't dump it into the routine box. Population data is available to the lowest level in age cohorts. Your RHAs must be updated. I've seen so many countries now. We did it about five years ago. Do you not ever ask questions about the client's intention? Are you going to breastfeed after delivery? Or what happened yesterday? One of my favorite countries has a question. Did you give your child three meals a day which consisted of the minimum four essential food components? Consider other types of data that you can collect. Record review, sentinel sites. When did we ever see somebody doing a record review or a sentinel site? Sentinel sites may be, but data quality, just some ideas. If the data is not good, you can't use it. Data improves when there's less. Third bullet point. I've never seen this in any DHI is to implementation plan. Three months after implementing something, you must have a workshop to go through what has been captured or they made interpretation type mistakes. You've got to fix it then and there because otherwise those mistakes continue forever. You have to have a yearly data cleanup workshop. Can we collect problematic data in DHI? Are there the tools available for us? Are there tools to tell us about statistical variations and outliers and missing them? Are there tools to help us fix the identified wrong data? Our RHI's WHO toolkits must be basic language for all low and middle income countries. I did some work with colour, did some work in a country in West Africa and we managed to reduce the data collection tools, the routine monthly data collection tools. It can be done. I helped Somalia with the HMIS revision. We used the WHA data standards. The death causes for neonatal and maternal mortality were too complex. We reduced the age disaggregation wherever possible. We revised the hospital form. Yes, hospitals. Do we have anything going on about helping improve hospital management information systems? I don't see anybody ever talking about it. That worries me. Hospitals spend the most money. Separate HR. I want to end off with a slide. I'm not going to read it to you. It stands by itself and we need to make sure that we are using all our resources in the best possible manner and that we're doing the right things. That's all from me. Thank you so much, Nora, for sharing your decades of wisdom with us here. So I'd like to welcome Dr. Louis Tina Day of the London School of Hygiene and Tropical Medicine to present the preliminary results of the impulse study on newborn indicators. Well, good afternoon, everybody. Thank you very much, Nora, and thank you very much for this opportunity to present at the DHS2 annual conference. I'm going to be presenting on behalf of my colleagues on the impulse study. The title of our talk is Newborn and Stillbirth, Data Quality and Use, some preliminary results from our study in the Central African Republic, Ethiopia, Tanzania, and Uganda. And I hope you know what I brought with me, a mannequin of a small baby, just to remind us about whom we're collecting this data. So here are my colleagues. This is collaborative research from the London School with myself and Marzia as the co-PI. Our implementing partner is Doctors with Africa Quam, so in Italy, Francesca and Giovanni in Central African Republic, Uzman, in Ethiopia Free and Mary, and then our academic partners are Ifacara Health Institute in Tanzania, calling out Jackie and Donat, and also in Uganda Macquarie School of Public Health calling out Ronald and Peter. So I'm going to do four things this afternoon. Oh, sorry, beg your pardon, I forgot to mention we're really grateful to our national advisory groups in those four countries and our international advisory groups, and including people from the University of Oslo, Johann, it's very nice to have you also in the team. So I'm going to do four things in this short talk. Talk about why focus on newborn and stillbirth data. Talk a little bit about the new EN mini-tools, as this session is about new tools, and then share some lessons we're learning from impulse. And then just invite you to join the conversation about what interventions should we test in phase two. So firstly, newborn and stillbirth data. A month ago, this report was published and the top line of the key message is this, preventable stillbirth and newborn deaths remain extraordinarily high. Newborn deaths account for more than 50% of under five mortality now, 2.3 million newborn babies die each year in our world, and another 1.9 million are stillbirth. When we look at what priority actions should we take, data improvement and use at the bottom is there. And this is really how it's exciting to be together and think about how can DHS2 contribute to improving data further for newborns and stillbirths. In the same report, there's a figure that looks at some of the core indicators and what proportion of 105 countries are able to report these indicators in their routine health systems. And you can see it varies from 90% of countries for some indicators, right down to 40% for the newborn interventions. Newborn mortality is a sustainable development goal and we've got seven years left. And we really need this data to understand how to take priority actions. So in every newborn action plan, there was a measurement improvement roadmap and a couple of studies in birth was a valid indicator measurement validation study. And then in birth two, one of the outputs of this study was funded by USCID was the EN mini tools I'm going to talk about in a minute. And then finally, the impulse study is a two phase intervention study, which I'll share some of our early phases from phase one. But it's all about using data for action, which is I know what's a motivator for everyone in the room in order to end preventable newborn death and stillbirth, but also for babies, not only to survive but also to thrive. So what are the new EN mini tools? These are global goods freely accessible tools on the data for impact website. They were launched in 2022. And they were collaborative development by this team in Bangladesh in Tanzania and at London School and with data for impact. And the impulse study contributed to version two of these tools. We get newborn data from different sources from population based surveys from civil registration. But the many tools are about optimizing routine health information system data like DHS to so that the data can be used for reviewing performance and policy and action. The many tools try to get that conversation up the data pyramid between the health workers we've just been hearing about in the facility right up to all the data users. And the groups into three groups map use newborn data improve newborn data quality and essentially they guide priority actions to improve availability quality and use of newborn and stillbirth routine health information system data. And they're in support of all these guidelines that we've just been hearing about. There it is that guidance for RMCH managers that Nora just shared with us. So as I've said the tools are groups around these three areas. And there are currently seven tools. And just starting with a map tool. This helps us look at what data is in the system where the gaps are and what's the measurement burden. So it's a macro enabled Excel. The indicated definitions are already embedded but there's also flexibility so countries can add indicators that are important to them. You map out all the different layers so the electronic like DHS2 but also the summary forms and the tally sheets and the routine registers where the often the data comes from. And then having mapped you upload the Excel onto a shiny app site and get this editable report which I'll show you some results from we used for using it an impulse study. The rest of the tools are really built around this virtuous vicious cycle of data use and data quality. And they function at the health facility level and also at the subnational and national levels and different tools are relevant for different levels. There's essentially an adaptation of the prism series, the prism being performance of routine information system designed by measure evaluation about a decade ago which comprehensively assesses the RHS and we've adapted them for the priority core newborn and stillbeth indicators. And we've also tried to make them as automated as we can just to make it as easy as possible for national uptake. Prism conceptual framework is to do with improving the health system for health outcomes and how the health information system contributes to that. And they divide up the health information system into inputs, processes, outputs and then outcomes. And we've been using this conceptual framework to visualise the many tools and also the impulse study. So let's look at the use tools. There are four of them and they help us listen to the users of routine data and explore electronic HIS and the improved tools. There are two of them and they use routine data quality assessment methods. So when I said we tried to automate them we've updated all the tools using ODK, Survey CTO actually and so they're ready to download off the website and can be captured on mobile phones or tablets. And then those forms can be uploaded into another macro-enabled Excel which generates these report-ready tables. There are more than 200 tables that get generated but also some figures that are useful in writing reports. So just briefly in summary, the EMI tools are designed for national uptake, they're open access with digital data collection platform and automated reporting. They emphasise newborn and still have data at sub-national and source health facility level, the registers, the aggregate data and so on. They promote data for action for every newborn to survive and they're around those three topics, map, use and improve and they align with the score, the WHO's score principles but also of course with all the DIHS2 work that you're involved in. So what about the impulse study? What lessons are we learning from the impulse study? I've already introduced you to the team earlier and we have a website where you can follow the progress of this study. The study started by doing a systematic review looking at newborn data quality. We identified 19,000 studies among which we could only include 34 and there was very little about individual case notes data. Most of the work was about routine register aggregate data. What we found is there were many different ways of measuring data quality in these 34 studies and the data quality itself was very heterogeneous with wide ranges in completeness, internal and external consistency and then there was limited evidence available for many of the really important data elements such as gestational age and the countries where this research was done was very limited and case notes was underrepresented. So impulse then refined our objectives into the two phases, phase one and phase two and I'm going to share some results from phase one today. Phase one is being conducted in four countries with the colleagues I introduced you to earlier, Central African Republic, Ethiopia, Uganda and Tanzania. If there are people from those countries here I'd love to meet you and continue the conversation. We're in 15 regions in those countries and I'm going to 146 sites between health facilities and data offices and today our results are around the first 76 sites and about 170 respondents health workers and data professionals. So we're using the eN mini tools for our baseline data collection for phase one. The first objective of impulse is to map newborn indicator data availability in existing systems and this is the automated report I showed you earlier. When we look at, I'm just going to show you examples from our country. So comparing Tanzania and Ethiopia here on that mapping report for the electronics, so DIHS2 is the system both countries are using. Here are the indicators and then the numerator, denominator and full indicator whether it's available or not. You can see in both countries interestingly preterm birth isn't there and then among the other indicators there's quite a bit of variability. And the reporter has this section where we look at, it really resonates with what Norah has just shared about don't collect more than you need to. So this is looking at among these registers what proportion of the data elements are used for core indicators as described by WHO and Mortant and you can see in Tanzania for example there's more data elements collected than are needed for core indicator measurement in Ethiopia it's a little bit more balanced. Our second objective is to assess data quality for these newborn and stillbirth indicators. So we've looked at it in two ways. We've looked at the registers but also the case notes. Typically at the moment data is coming out of the registers but there's interest in measuring quality of care and that doesn't really lend itself to more and more columns being added to registers. So the case notes where health workers write their clinical care is much more potentially useful. So here's an example of a structured case notes from one of our countries and then a register that we all know and it's just really important to keep thinking about how those two different data sources relate to one another and there's you know to reduce the duplication of effort. So in past as I've said contributed a version two of the e-mini tools and with the tools already existed in English and Swahili there's now translations in Amharic and French and we've added on a new tool tool seven so there are now eight tools and that's about the individual case notes. So when we look at the data quality in the registers for newborns and stillbirths just looking at the denominators we have to measure total births and also live births and you can see here in this figure that once it starts going up the system from reports into the electronic health information system I mean the consistency is fairly good but if you look at the right at the bottom the completeness of the register and this is from the pilot data in Tanzania was only 88 percent so I mean even before you start pushing it up you know you've got a data quality gap. We look at the numerators and this is from the impulse study for 76 sites looking at eight numerators for newborn and stillbirth. Again the completeness is actually not not so bad but the timeliness and this consistency it really really drops so we're seeing really mixed picture of where the gaps are and where to intervene. Interestingly our respondents 48 percent of them indicated that they were aware of data manipulation for various reasons taking place they've been a number of publications about that in recent years so we decided to ask and so far that was what they're telling us. Moving now to the case notes with this new tool and we've split the the variables we're looking for the clinical information really into admission history, admission examination and discharge and among 59 pieces of clinical information that we as doctors need to look after newborn babies we're finding it's legible 99 percent but completeness is very variable. This is just showing Ethiopian Uganda again preliminary results but you can see there's a wide range in data availability for example gestational age to measure preterm birth is less than 40 percent in both these countries. We're also looking for proxies of data use and this internal and external consistency with registers and also looking at how does the design of the case notes affect data quality. Looking at some of the other determinants from the prism framework to that improving data quality it was really striking this is in the Tanzania pilot that the district was performing much more strongly for efforts to improve data quality maybe identifying a gap that we haven't thought so much about the the source data at the health facility level. Our third objective in impulse phase one is to understand data use by different stakeholders and this is a colleague picture of my colleague Usman implementing the eMini tools in Central African Republic. So you can't read all these numbers but I just wanted to show them to you to show that there was really a gap across all the measures that we are capturing for data use the highest measure in anything we can find is 74 percent and much is a lot lower than that and so there's I mean as everyone in the room is aware there's a big challenge with data use. And again particularly at facility level whereas the district there's some evidence of visualizations and so on in the facility level it's really very much less. This is some looking at a school and the eMini tools has a couple of schools in it and combining 10 components of evidence-based decision making we're really seeing clustering I mean around 30 to 75 percent across all levels whether we're talking about the data officers or the facilities across all the countries. Our fourth objective is to analyze those technical organizational and behavioral factors to improve data quality and use for newborns. And again just reminding us of this prison framework we're talking about this section the inputs of the RHS. If we think about that promotion of a culture of information and again just showing examples from Uganda and Tanzania we've already talked about their lack of evidence-based decision making that's the lowest scoring component in both countries but essentially all of it is around that 30 to 70 percent again similar in both countries. And then if we look at resource availability we just again heard Mural talk about this a few minutes ago and everyone has a delivery register which is encouraging and but when we look at the availability and over the last six months off to the right it's a little bit more vague but look at this Kangaroo Mothercare very very little paper-based register availability but yet that's an indicator that's really being promoted but where are we getting the data from? And also death registers are very very variable in the sites that we've been visiting. And then we're looking we have been exploring availability of internet and electricity both needed to digitize our data. We're finding again very very variable 20 days or more of internet access in a month very varied and up to 30 percent of the equipment isn't working. The other gap we're finding is in health worker RHIS education training. There seem to be plans but then as you look at the different actors in the data pyramid not everyone who's being asked to do RHIS tasks is trained to do that between data capture and report writing. And then around the topic of RHIS motivation a behavioral factor again this is one of the scores that the EMI tool captures and we're finding it's clustering about 60 to 80 percent in terms of respondents reporting they feel motivated for RHIS tasks and again it's very much across all levels and in all countries. So IMPOS has got the opportunity to go one step further and to really dig into the data and really explore which components of those scores contributing the most and we're doing that analysis at the moment. And another thing that EMI tool measures is that we call it the confidence competence gap. So people report they're able to do something but then we actually have a little like a role play a little scenario and then we measure can they do it. And again this is from Tanzania some pilot data. We found very large gaps between confidence and competence between one and 47 percent gap. We've asked all our respondents what do they think about RHIS? Do they think anything needs to change and what do they suggest? And we've got literally thousands of ideas that we're working through at the moment that there are just a few listed here but we really want to ask the users and the health workers the data health professionals what do they want to what do they want to change? So so far IMPOS phase one what are we learning? I'm using the prism framework as our conceptual model we're finding gaps across many many of the determinants technical organizational and behavioral which are contributing to newborn and stillbirth data quality and use. But of course there are higher performing sites and respondents so we're really digging down to try and identify what those are to see if we can you know incorporate that into an intervention to test. So as I've shared with you the aim of the IMPOS study overall is to improve newborn data quality stillbirth data quality and use. So phase one we've been using the EN mini tools as a baseline assessment to look at the current situation among these 146 sites. We're in the middle of data analysis now both quantitative and qualitative and asking end user ideas to improve and we're working towards a peer reviewed supplement and the picture here is my colleagues who are working on that right now. But I just want to leave us today with this question what intervention should we test for phase two? Obviously it'll be driven up from what we find in phase one but I'm just aware that in the room there are many many experts who've been thinking about this for years so I'd love to just invite you into that conversation. We've said in phase two that we want to design an intervention and then test it and that testing could be effectiveness, cost effectiveness we can test it in different ways. We want to design an intervention that's co-created with our national advisory groups international advisory groups but also we're really welcome the DHS2 community to this conversation. Some of the questions we've been puzzling about is how can we show frontline health workers their value in the data they capture that's pushed up the system? I loved it in the role play when the community health worker this morning said I've sent my data I hope I get some feedback as to whether it's useful or not. Now how can we strengthen that part? Do people here know of educational materials for RHI's competencies for health workers that perhaps we could use and use for newborn and stillbirth data? What about visualizations at the health facility level including for data accuracy? It seems to be a bit of a gap there and just wonder if there are people in the room who can advise us on that and how can we strengthen that information culture so that that motivation for really important data tasks increases? Data isn't an end in itself I'm a clinician so for me data is so totally linked to that kind of quality of care piece for the baby that I brought along today but how can we just get that balance right for health workers providing care and also documenting the care they've provided so that those two things really resonate with one another. So today I just have shared with you a little bit about the importance of newborn and stillbirth data please whatever you do please think about how we can strengthen newborn and stillbirth data. I've talked a little bit about the EN money tools which are available for your use on the D4I data for impact website. I've shared some of the lessons we're learning in impulse phase one on behalf of my colleagues who I think a lot of them are online so greetings to you and just invited you to help us work out what intervention we can test for phase two. I just wanted to say thank you to all the health professionals and data professionals who participated in our phase one data collection and special thanks to our study team including all the data collectors and to the KZ Foundation who are funding us thank you. Thank you so much Luis Tina I just I really appreciated this abstract because it is so rigorous and goes into all of these kinds of issues that have also been raised by Nora. I know it can sound like doom and gloom but it is really not we have many stories tomorrow about what is the impact of the use of these types of routine data so there is a session tomorrow impact and effective use but I think a lot of the lessons learned here and what we need to pay attention to is the rationalization of these routine health information systems and particularly because they are expensive and I don't mean just expensive in terms of connectivity I mean expensive in terms of the health workers time so approximately 30 percent of these health workers are spending their time documenting if they are documenting data that is not being used for decision-making that is time that they are not spent providing better care so I think these are lessons learned to take in the future and it is my pleasure to actually invite our next colleague who also has a clinical background has now turned into an HIS expert also sitting on the same team as myself so Stefano Perotti who is a DHIS2 implementation expert in the health domain. So thank you very much Rebecca I hope you can hear me so yeah I mean it's very inspiring and what I'm going to talk with you today is about health facility attributes was something that was already mentioned in the previous presentation and why this aspect like they run they should not run sorry okay anyway and how this aspect should be really considered to be integrated in a routine health information system for improvement of health performance indicators so when I'm talking about health facility attributes so it's something that it came up several times during our work as well during different presentation when we're doing the revision of the different abstract that we receive this year it's something that is already present so this type of information information that are collected by different platforms by different in different programs and now we are going to see why this is very important to be integrated in one routine health information system so what are these health facility attributes? Health facility attributes really in the practical are semi-permanent data that are collected that are specifically for health facilities so for example the key information related to human resources to availability of of staff as well to train staff training availability of the different and provision of different services availability on the logistic so it's something that as I told you information that is present information that is well known normally at health facility level but sometimes missing these steps at highest level this through the national level to be able to be analyzed these data are already being collected that are coming for example they can come from the very well known master facility list maybe they can come through other vertical programs so here is an example of what normally is is being collected availability of key equipment and essential medicine so everything right as well to pharmacy infrastructure electricity availability internet availability as was shown before the preparedness of our health facility of our health staff about the response to any type of routine and emergency cease conferences and availability clearly for specific services and how are these data collected so there are already some standard tools that are provided for example by WHO like the harmonize health facility assessment the HHFA that this is a survey that is already ongoing in different countries in which the information are collected are core and optional question related to availability readiness point of care management and finance this type information is collected as a survey so normally is an external person from the health facility that is coming and collected this information the information are collected for example for HHFA are very complete information so normally when an HHFA is done in a country is not done for every health facility but just some of them are picked to be able then to represent a specific area and a specific type of facilities beside that there is as well the ERAMS platform normally ERAMS platform is used for collecting information specifically at the moment in emergency settings so information collected are very specific for okay number of facilities are open close wider open wider closer service availability for key specific services so during this year we have been worked with in collaboration with the global funds with WHO with GAVI about on the production of what we call a global toolkit so to be able to provide a kind of a starter pack for the different countries to be able to collect this key information I don't want to go into details here because finally the information that normally are required to be collected we try to standardize with the truth that are already available like HHFA or ERAMS but I would like to have more focus on the how these HFA modules and HMIS they are related okay so first of all HFA so every type of information that is collected related to the attributes of a facility doesn't replace other standardized tool is HHFA also because the type of information collected in the use of the information is very different normally when we say when we talk about HFA our information that are collected by the health facility so we don't have an external person coming inside the health facility to do a survey normally these information are semi-permanent information okay so information that should be collected not in a monthly basis maybe can be collected once per year or twice per year or can be collected at least for example there is any change for specific emergency situation in that specific area and the idea and the added value to this information is the possibility to integrate them in the routine health information system so in some countries the information is already collected as kind of HFA but they are stored in a separate instance so they are not really analyzed integrated in the routine health information system so here is the most interesting part for me so how this information can be helpful at country level first of all to plan resources allocation directly inside the country so you you are able to see at central level if there is any gaps on the different region the different health facility for example on service provision then you are able to identify any bottleneck and accessibility issue because maybe as I say before there is a specific area in which we are lacking a specific service and then as well to prepare respond to public health emergencies in case we have to mobilize in case we have to decide okay for referral system where we are sending patients where we are going to support specific hospitals etc etc is important to have this type of information here are some example okay some visualization that already present some of the toolkit that we developed in collaboration with WHO for other availability of services okay I mean these are very simple information but they can have a great impact for example here we are seeing the availability of specific services at country level we can see if there is any area that is not covered or if there is a very low proportion of health facility that are providing this type of services then as well availability of staff availability of trained staff so to be able okay if we have to allocate any resource or for example as well a country level if we need to orient the different donors okay on where we should have an intervention then we are able to have a through this information and the most important one for me is the triangulation so why we are telling to that the integration of this information are core are very important to have integrated in the routine health information system because then you are able to triangulate this the health facility attributes with other health indicators okay so for example you are able to see let's maybe take another example here um specific for example in this case we have a visualization on case loader for staff so to be able to see this indicator normally can be used on quality of care that we are providing because we can have a health facility they are providing thousands of consultations but only with three four health staff dedicating on this this we are basic one but for example can be more advanced to try to say okay we have specific mortality for example malaria mortality do we are providing service for management of severe malaria case or do we have all the treatments in this health facility to be able to take care of these cases so the ability to have them integrated in a routine information system can enhance this improvement of key health indicators here is another example this is more for an emerging situation so this image is taken from the ERAMS is one of the report that was just published recently but just to give you an idea okay how this information can drive the different donors okay to decide that which are the audience which we should intervene what is the situation for specific this is like tuberculosis of connectivity of these health facilities so as a summary we say that okay health facility attributes are semi-permanent information that are collected at health facility level are information that are easily collected because normally in a health facility the different manager they already know what they have in situ what they have in place or not information can be collected through survey what we suggest and that's why they are divided to have in a routine and information systems that this information are collected integrated in their routine system this information should be analyzed with other health key information from the health facility to be able to have an impact and the DHS2 as a national health information system platform can make it more cost effective for the health facility to be able to update this information in a regular basis and that can be available for decision makers so that's all thank you very much thank you so much Stefano and so the second part of the presentation of course is about new tools so with the support of global funds we have been working on with WHO developing the core metadata to kind of support this to be able to integrate this into national DHS2 systems and what we found during this process is actually everyone is collecting the same data everywhere here EMS is collecting it the HHFA is collecting it the global fund pulse surveys is collecting it and it is coming all over but then people don't have it when they need it so the whole idea of these tools and we'll be developing and publishing a toolkit on our website at some point in the next six months we do have some metadata available if there are any countries reach out to us we are working with some pilot countries such as Uganda and Tanzania and Ghana as well all with the idea of actually making this data available more accessible in the system so we can stop wasting money running out and collecting it over and over again so our next presenters we have colleagues from the WHO division of data analytics and delivery for impacts so Dr. Doris Ma who I'm going to call her the godmother of mortality statistics in fact she's going to introduce a new tool that has been named after her as well as Ninad Kostansiak who is an expert in classifications and terminology to tell us a little bit more about ICD-11 and then we're going to conclude with our own John Lewis from his Vietnam who's going to share a little bit about how we are actually developing ICD-11 compliant applications to support the proper coding and underlying cause of death for better mortality statistics and use and I found that incredibly interesting Luis Tina that that was actually one of the key gaps that you had found in the impulse study so I think we have our presenters online Doris and Ninad the floor is yours to share your screen please. We shared it now can you see it? One second. Can you see our screen? Okay hang on just a moment. Good afternoon everybody I'm Doris Mafat and I'm working in WHO statistician and handling causes of death statistics so I'm here by colleague Ninad Kostansiak and we start the first part of presentation for the Ninad. So there's one moment I'm going to try to add your volume up a bit just give me one second please. Can you try again now? Okay I go back much better thank you just go ahead and start again thank you. So yeah sorry I'm Doris Mafat statistician in the Department of Data Analytics working with Anchu and Ninad Kostansiak who's here with me so I give a presentation on the role of the health sector in strengthening CRBS system. So a score the score survey which was carried out in 2020 shows revealing gaps in CRBS that is counting births, deaths and causes of death as you can see the green the green bars only the number of countries the portion of countries that can that have full birth and death registration and when you see the the longest the shortest green line is in fact showing in in Africa Southeast Asia and the western Pacific where the gap star most important when you look at the right hand side on certification and reporting of causes of death the situation is not is worse even and this map just shows where the gaps are mainly in certification and reporting on the causes of death and as I mentioned earlier these are mainly in Africa as you can see all the red parts and the the the the issue is that despite many initiatives to improve certification and reporting of causes of death there are fundamental issues so I think in mind that DHIS2 is implemented in those countries this is where I got come up that maybe we could leverage we could leverage on on the DHIS2 system to start the reporting of causes of death so why are we interested in CRBS because it's a it's a benefit also for her you look at that there are records on the left hand side of the happy function systems umbilatory care the EMR and these are able to give us the birth and death records in fact and that is linked to the civil identity systems where you have national ID local registrar office where they will register the birth and the deaths and get the legal certificates and this information then are used for other administrative systems and the corporation register for example for the voter roster for the for the passport they go registry etc so why do we say the health sector can contribute if you look at this map there are actually missed opportunities the bar the yellow bars showing the birth registration which is which remains very low in most countries but the maternal health and immunization coverage are very high so the same babies are taken to the to the health facilities for the routine health sorry for the routine health services but then they are not registered and the worst is that death registration lacks behind birth registration this is because birth registration babies need to be registered at birth because when they reach school most of the time they ask for a birth certificate for the enrollment and that sometimes also to get access to other care facilities for death registration the main issue remains that there are no motivation for people to register their deaths and also there are not enough medical certifiers available in when the deaths occur in the community to certify the deaths so what are we doing here to improve the CRBL system the virtual's contribution is we lead the development of normal standards in the collection reporting and analysis of course of their statistics for example we have the definition of what is the maternal death is still above and we develop the international classification of diseases according according to the causes of death as well as as well as the rules for the section of underlying course of death and we design operationalized a collaboration between health and the CRBL system for mutual benefit by leveraging on the health sectors routine RMHCH interventions to increase the registration of births and deaths so you are advocating for it that whenever someone bring a baby or we start for these services in the health in the health facility the health worker whether it's there or whether the worker should encourage or and advocate for the family to go and register their other births we also develop the data collection tools within the R routine health information system in countries for completion on births and death reports and we're going to have a demo of this data collection tool later after this by presentation we also develop global products guidance from the virtual unit set for the role of a health sector in improving birth and death registration and we developed the ICD-11 suite of tools according section of the course of death analysis of course of death statistics as well as training materials and curriculum to support ICD implementation in countries so what type of tools we have a bit we have developed so far in 2007-15 I guess it was that we developed the start of mortality module SMO but that was based on ICD-10 at that time and a few countries have started inventing it but now with the advent of ICD-11 we're moving towards and the new app which is more I would say the which has more functionalities also and it's better tied up to what we expect of ICD-11 implementation in countries that we have them in doing all this work we had in the middle something for rapid mortality surveillance system that was during the COVID-19 when we were asking countries to report only total deaths not about courses of death total deaths how many deaths we are capturing every week and that small package was developed by Oslo Cooperation WHM so I will switch on to the floor to name out who will continue the presentation thank you yeah so regarding ICD-11 that came officially into effect last year and there are a couple of novelties one of them is out from the you know updates in the content itself making it up to date from a clinical perspective there is of course the point that we have now in ICD-11 for the first time integration of terminology in the ICD-11 classification which means that in addition to the 17,000 statistical categories we have about 135,000 clinical terms and synonyms integrated and that makes it of course much more clinician-friendly because clinician or health worker-friendly find the level of detail they are searching for when they make the transition from the clinical diagnosis or cause of death to the correct ICD code and the second novelty is that in order to manage that complexity obviously when ICD-10 was disseminated it was disseminated as a book ICD-11 comes with a whole suite of tools which allow for digital electronic search but also analysis of the data etc so it is primarily a digital tool and it disseminated as a digital tool but it also operates in environments where for example internet is not available and you see here the suite of tools which are characterized for the mortality reporting with ICD-11 and they have been basically integrated in the DHS2 up for mortality medical certification and coding of causes of death and that is part of the largest suite of tools which you see here depicted and we basically categorize them into implementation related tools as well as tools for training for maintenance and most importantly all of these tools which are available for end user consumption have the ability to be easily integrated into any kind of software application through the ICD-11 APIs and that is something also which is very different to how ICD was integrated in the past so it's not anymore uploading a flat list of hierarchical categories and then displaying them as a drop-down list but it's really consuming the APIs with the search algorithm in the background which allows you to search and to assign codes using Google style search functionality and smart search functionality and that has been basically happening in the integration of the ICD-11 in the DHS2 up so where we have the coverage of the data entry to the data analysis to the data dissemination and John will demonstrate this but let me just run you through some of the key features here we have in the app on the data entry side what eventually will become the ICD-11 and WHOMCCD form specifications for the electronic medical certification of causes of death we have the integration of the ICD-11 coding tool which is as I mentioned the Google search where you get from the cause of death as it written by the clinician or the health worker now to the correct code in the ICD-11 we have the integration of Doris which is the rule engine for automated selection of underlying cause of death so what we did in ICD-11 is to digitalize the mortality coding rules and we have now an algorithm which allows to make this assignment of the underlying cause of death which in the past was done manually or sometimes in some countries also using other software like iris integrated into the realm of the ICD-11 there's also a new feature which will come up that allows even processing not only of coded data in the rule engine but also processing of pretext so that you have actually these two steps one is automation of the coding and then subsequently automation of the selection of the underlying cause of death and then we have data analysis through the integration of ana code 3 and code edit so that the ICD-11 the coded data can be immediately analyzed and then visualized in the respective ways for which are needed and all of this is customizable to the respective user needs so this is what I think you will also see in the demo and I think in terms of next steps just to highlight here that we obviously one of the key big challenges is to facilitate this process of automation of the input data and the course selection data so that is something which is still going on and we are working on to refine this and then of course all the aspects related to integration with other disease surveillance and applications and also with the mobility data collection so with that let us stop here and I think with this we hand over to John for the demo over. Is that fine? Yes. So like what I'm going to try to explain our show is the the cause of that app which we built inside DHRs too. This is a generic app which we want to we will soon put this one in the DHRs to App Store where you can try to download the app and install your own DHRs too instance so we've been supporting this app from the version 35, 36, 37, 38, 39, 40 so all those versions you can if you are using any of the DHRs to instance in your own country above 35 you can use it perfectly fine so I'll quickly go through the few of the things first let me just like to go through them I have already installed this app in the DHRs to instance so I'm going to see like what all the different features you have and then at the end when we have some time like I can just like show you how you can try to install the app itself in your own DHRs too. See so like here in this app like you have data entry module I'm just like going through that very quickly and like we just say like how best we are we can do the registration so now I'm just like doing a registration of the the cost of that I just have reported data reporting is today that was let's just say on a Saturday name the reporter and here you can actually just select the age or the things let me just select is in here 26 years age male not as you can just like leave these things blind none of these things are mandatory it's okay and you just say once you do that one this frame A and frame B what you're seeing it's exactly the same as you're seeing in your medical cost of death certificate so or they're in the program frame A and frame B and based on your selection by the gender and as well as age few of the things will appear in frame B okay so now like here this is the free text like whatever the clinician enters so you can enter around here let's just say very quickly I have noted down the the diseases so I just say edema and like here when you click on it so that's when like you're actually looking looking into the WHO I said according to so now like I just say so the recording is huge email so like here like you have spread that's wrong let's just say edema okay that's fine so you can see it on here these are all the different details these are nothing storing DHRs too so now what you're trying to do is like the all the list it's it is inside DHRs too but you're talking to the other website of the Irish city embedded according to you you have the list of all the different details you can see the hierarchy and everything and then like you can just select okay this is looks fine so just select this as the cost okay just one second email okay let me just select any of these things it's not right but oh yeah it's all right I just want to select the the right one because I want to also show you the how the door is to gets calculated there we go and then we select it's in let's just say five years you also have this one is hypertension just like here and then the last one which I just select is okay this one yeah so I'm not the Irish city coder or a clinician I just this is the the same example which I use so now what I've done you just have entered a few of the details so on the free text side is what the doctor has entered and on the the next side is the world by city coder is entering it is just selecting and searching all the things before we had this underlying cause of death which people were supposed to see and add by themselves but now with the door is to you can click on compute what it does it will go through the engine and these are all different full report of the door is to and then it will select the underlying cause of death for you and with the app like you also have frame b where you can try to enter all the different details that's okay I'll just like save and then in the app itself you have this a simple medical certificate of cause of death which can be printed and hand over to to the people at the clinic level so that like they can go around and this can be customized based on the other things so this is the simplest data entry form so this is already finished now what we also what we have is the analysis of the dashboard so these are all the list of all the dashboards which we have created which is embedded in the app itself so you don't have to to go somewhere else to do the analysis because usually like in pictures to what happens like we enter the data in either the track capture or a capture app and then you go for the dashboard or data visualizer to all the things so here these are all the dashboards or the charts which is already predefined people can also go to DHRs to do the further analysis but at least for most of the things you already have it around here you can if you don't like you can just select it and if you can include more data and all the things it's here you can also just save for NCD non-communicable diseases, frequent cost of doubt or by the chapters or by malariam tb and 8 combined one you can also just see on here so these are all the things which which you get embedded automatically in the app itself after you install you have all these things I will strictly show on the administration too so for example if you want to add some other attributes because every country they want to include different things for example they want to include the national ID or the addresses or the information about who is collecting the data those kind of things can be configured by the system admit itself it doesn't require any programmer or anything to customize this this application and also including the sorry and also including the material cost of certificate of death so if this is simplest one you can upload your logo on the side and then you can just see what are the different fields you want to include or not to include whether you want to include in the footer or the the body you can do that and if you if any of the people have like how to build a standard report if you already have a predefined format you can also upload that one so that's all there are the two things which can be to work work around and then is the translation so this is also something which is we want to to work on so anytime that we usually the translation is the people entering it but sometimes you want to edit your own translation for your own local implementation so here are a list of all the things you can add many different languages like just say herb and then like you can enter so these are all the keys like what you're seeing in the software so you can just change and then you can based on your user settings it will automatically will switch into different languages what you're seeing go and then other part is on the unaccord export so let me just select this this is something which takes time based on the data what you have so here is like what are the different data what you have entered it will convert everything aggregate and put it into a unaccord export file which you can download and use the unaccord analysis to so that you don't have to do the export again in different places so all these three things is in inbuilt in here so let me just quickly just show you one more things okay so just in the dashboard when you log in so when you go forward this website dhs2.world slash whocod the username password is already mentioned in the login screen when you log in you have the functionality in solution manual and the app itself if you want to try please don't do it in the production instance always we just say do it install it in the demo instance or the development instance and then like see how the app works and if you have any problem please reach out to us and then like you have all the manuals and everything so just to quickly show you how the installation works is if i can open this one yeah so here when you first time install you don't see anything you will just like see this screen which is here so it will show you like where what all the different things you can select the icd coding tool whether you want to use the the bleacher one or if you already have installed the icd coding tool in a docker container so you can actually point out to which service and then like you can just like say you have to select the all the attributes where is going default installation yeah here so in dhs2 how it works is like you already have all the patient attributes on the fields in in your dhs2 we are not making anything duplicate you can use your first name last name gender sex and date of birth if it's not there you create it if it's there you can use it sometimes like the first name is called given name last name is called family name so it's up to you in your own local installation you can include it and if you want to have any other additional information like investigator source name and other things that you can also add in your frame a and frame b the additional section you cannot remove anything which is there but you can add additional things and also patient attribute what you want to try to collect and then other part is like after doing that so you need to just say where are you collecting which hospital or health center so you select all those things and this part which most of the dhs2 implementation when you do you're not setting the roles properly so we usually any program which should have three roles one is admin that means who can change the one is capture that's who can do the data entry and one is only viewing so these are all the three groups which we already have predefined it and then you can select whichever the user who belongs to admin who belongs to capture and who belongs to view and then it will give you the review and then the installation is done so it's as simple as that one is so that's that's why like we want to try to deal with the entry and solution let's just quickly go through so this was the expert which I was just saying we imported a lot of data so that like we can show the dashboard so this format what you see here is exactly the same and you can download it as an excel and then you can try to use it so this is basically what I want you to present it's a quick demo and then you can always like when you let me just log out so with this one like you already have you can use the c already demo and then the user in password is already there you can go through it later on see how things are and like the app and everything is there we'll soon put the app in the app store so that like you can try to install it in your own dhs too yep that's basically it Rebecca so thank you so much to the presenters before we close this session which we will do just a couple minutes early to give people time to transition I just wanted to give a final reflection to my colleague Dr. Anju from WHO DDI thank you Rebecca and just a quick sum up on especially on the last presentation that linked to actually the two presentations one from Stefano on the health facility attributes and then the big one is on the mortality report things so I'm just coordinating a lot of the work within WHO on several different programs that implement the standard on routine health information system but a lot of it also support the implementation of the HHS the routine the health facility attributes assessment I think that is something that coming up but also that is something links a lot of work on the data used in the country and I really appreciate the presentation from the London school colleagues I think the data use would be something that is important to look into and then that should be what drives the data collection the item and and the the design of the data collecting the collection system for the course of death linked with the ICD-11 the loss of country have asked so far that by last click we have at least 11 countries asked for the implementation of the apps a lot of different ideas coming up but I think we need to keep on thinking it cannot be separate from the training of the first of the physicians on the ICD-11 it got to go hands in hand and that's why my colleagues Doris and Nina one is on the mortality and one is on ICD-11 and they have to work together also it's important to understand how the country designed who is doing the coding who is doing the the certification for for medical course of death and how we're going to bring the data together and what would be the final products a lot of it goes into reporting but also what would be the benefit for the facility when they look at the cause of death that is happening in the facility or in the catchment area so I think this is something we can continue to buy time over the breaks or coffees or whenever and then have a good communication but just something that you know we will be happy to be touched and see how we can support the implementation of the course of data in ICD-11 because both of them are quite new so thank you and thank you for this opportunity to present it. The final thank you to Doris and Nina who actually got up quite early this morning I believe they're in Barbados giving a training so thank you all so much with this couple housekeeping in this room we will have DHIS-2 for health emergencies next coming up in just a couple minutes there is an ongoing session in auditorium five right now on DHIS-2 and NCDs so if anyone is interested in that please feel free to move yourselves over and then the other session happening in auditorium four is the integration tech session and getting data out of DHIS-2 so thank you all for joining us for this session