 Okay, welcome everyone to this session on the choice to maturity. So we're a number of us who are going to present here today. And what we will talk about in the session is, oops, why is this not moving. We will talk a little bit about key recommendations for a solid foundation for the choice to. I think we have an old slide set here, because I had some other. No, it's here. Okay, never mind. My bad. I'll start over. And we know that's to build the details to system. My name is Anna, by the way, I work in the global implementation team here at the his center and I'm here with other colleagues as well also working at the his center. So one of what we do is to support the his network with sort of global resources for planning for assessments, etc. So we'll go through some of these resources today and many of them are around these foundational domains of DHS to some not necessarily only how to build a program but how to make your systems last over time. So we'll talk about some of the key recommendations that we give out to build a sustainable DHS to systems. And also a little bit how countries are doing we have some data from some assessments we have been doing in the past years, so we'll go through that, and also point you to some available resources, teaching materials, etc. That can be useful if you're communicating with the ministry or you want to learn more. I want to say it will set in the plenary session but in case you didn't hear that message. The Norwegian government will today at around noon test a new warning system. So there will be very loud alarms outside from the rooftops and all phones that are connected to the 4G or 5G network in Norway will sound off. There will be a loud sound, even if it is on silent or you're turned off notifications and everything so we will try to end a little bit before and it's only a test so the Russians are not coming. Hopefully. I think they're just preparing in case there are any troubles. Okay it's only a test so it will be exciting to see how loud this is. We have shown this picture before, but I think it is still a valid picture in order to have your sustainable good programs functioning over time, you need a proper foundation. So, very often we want to build quite fancy things on top like this house your nice tracker programs that are going to reach thousands of facilities etc. But if you build it on very shaky foundations. It's might just fall over and crash. So we will talk about some of these foundational domains in the session today. They are sometimes a little bit invisible, very often underfunded people don't necessarily put them in their proposals or want to pay for it. And I think everybody working with information systems in general will agree to that. What we're talking about today is not necessarily very DHS to specific, but it is very relevant for everybody working with DHS to. So in the past year, we have been doing a lot of work around what we call the DHS to maturity profile. This is a tool. Originally funded by Gabby and global funds where we wanted to learn more about how our countries doing in terms of their DHS to implementations. Very often it's easy to say, oh, things are challenging it's going well but we wanted to dig a little bit deeper to understand how are we doing on different domains. As you can see here at the bottom we have what we call foundational areas this is like the foundation of the house the pillars that DHS to is standing on. If these things are not in place. Probably your system will stop working. Pretty soon. So you might have quite a lot of funding starting up to build, for example, your, your COVID-19 immunization registry, for example, or an or an HIV tracker or just aggregate surveillance. But if you don't have things like proper governance in place. You have a system to train your end users funding and systems to handle your infrastructure. It might be a wasted investment over time. This map just shows some of the countries or the countries where this exercise has been done. Most groups have been working closely with HMIS and health programs to go through almost like a questionnaire understand a little bit where they are on a scale from from one to four. Of course, this is not exact science in any way but it's, it has served us a good conversational tools I think with the programs and the HMIS to discuss what should our priorities be going forward. So in this session here today we will focus on this foundation, the pillars of the of the of the fan city tries to health programs. What it takes to work in these different domains. As I said, we will talk a little bit about what kind of advice in general the HIST network is is giving to two countries. We will pick out some findings from this maturity profile exercise just to see in general what what countries are struggling with, and also point you to some resources so the presentation we will show today will be full of links so you can access the PowerPoint later and enter those and have a look it's all open and free and documents of course. I will start talking through two domains. So the first one is leadership and governance I think this has been mentioned by several of the presenters in the conference the importance and also yesterday in the plenary. The importance of having a country that knows what it wants and has a clear sort of governance and leadership model. Sometimes we see in countries that are not so well performing that you might just have a technical team who is working on whatever they think is the most important so if. If someone is asking for something it comes to the top of the queue while it might not be the long term priorities of the country. So in general what we recommend when we talk about governance is that a country has a two or three layered governance structure. It means that you should have some sort of governance committee that has broad representation from the stakeholders in the country so that health programs are represented. People with decision power are represented they can make some decisions on saying that this is the direction we are going in with digital health and DHS to in particular. So we go here and we believe that doing aggregate TB nationally has priority number one and this other program has priority number two and we also have to work on performance training etc. So that it's clear to everybody. Also in this governance model, we recommend that there is some sort of operational management so somebody who can sort of work day to day on planning has some oversight of activities. And then you have at the bottom, what it says the core technical team, these are the people that technicians who are doing the day to day fixing building DHS to systems. But it can be a bit vulnerable if you only have a good technical team, you also need somebody who sets some strategic direction. Another important thing around this leadership and governance topic is that you have preferably one DHS to core team across many programs. Sometimes we see that this one program has their DHS to people the other program has their DHS to people, but ideally it's good to have one team. And also that this mechanism is known to partners to stakeholders who want to invest in digital health that it's like a clear and transparent system and people are appointed and they have some roles, they are meeting regularly etc. And that there is some documentation on how they make decisions how they make priorities and references from from meetings. And of course, there is different skill sets needed at different levels of this governance mechanism. So these are some quotes I'm going I'm not going to read them out loud but these are some not quotes but but notes from from our his network when we have been conducting these maturity assessment so have been talking to the talking to the ministry is trying to understand what is the status of sort of leadership and governance in country X and taking some notes. So you can see that for some countries. There is no existing governance body in place. It's done through like random requests and requirements from specific programs. countries say or in countries it's just not sufficiently functional. They might be lack of clear vision on how to implement and maintain DHS to lack of digital strategy. So there are some programs develop tools as they would like. We see examples of governance bodies that are established but they lack representiveness from the relevant programs, not sufficiently functional, or that they do not meet regularly so there is a governance body or committee in place but there is no clear, regular meeting and meeting schedule. We have some learning resources. These are a couple of slides sets that might be interesting if you're interested in this topic to have a look at. Many of the resources we refer to in this session are Academy material that we have used to teach both online and in or in physical academies. So they're typically slides that like these, please read them, use them if you want with your with your stakeholders in your countries. Just steal and reuse the messages. That's perfectly fine. We will open up for like discussion and questions at the end so I will continue on another topic. This is strategy and investment. This is another key pillar of your the tries to house. We clearly advocate that countries should have one details to plan where possible for governing ministry we just came from this plenary talking about education so of course you would have an education details to plan and a health details to plan. But it's really beneficial if you can get people to align around one plan so that multiple stakeholders can invest in these foundational domains. For example, if you initiate a new HIV program or an API program, can you also get them to invest a little bit in security or in servers or in skill building for the core team, etc. That's beneficial for everybody. Again, avoid implementing in silos. This is related to to the point above the goal is really an integrated system, both to make it easier to manage and also to be able to use the data across programs triangulate data is useful. Also to focus on the fundamentals of the tries to for sustainability. So, again really advocate for these foundational pieces that we are talking about today. We see them come up again and again. We try when we talk to countries both in the his groups and at the his center to be have a little bit this principle of saying no to unfeasible projects. Sometimes we can be blinded by by new and shiny ideas. But if we know that they are not feasible to sustain over time either because they are lack of funds lack of skilled staff. Then maybe we should just say no, or do something else first. But at the same time, you know it's important to be flexible and adopting new opportunities so this is this balance of sometimes you should say no but of course sometimes you need to take a chance and be a bit brave. It's the same for all of us and I'm sure it's the same for everybody in the room we we sometimes see a lot of unrealistic projects that if you actually sit down and you do the math and you start to really budget. Then it's a bit. And sometimes you say yes to just stay in the game or get the project but if it's a waste of a lot of money in the beginning and it will die after a couple of years and maybe we should be like clear on saying saying no to things. And we really advocate for highlighting and budgeting for long term implementations so don't just say what it costs to to set it up. But also what will it cost to keep it running for year two, year three, year four, year five. Here also there is very interesting results from this exercise of doing the maturity assessments in 40 plus countries. Countries are saying that there's no core HMS funding. So HMS tries to play central roles in the country's health strategy but there is no costed work plan for sustainable technical assistance over time. That strategy documents are outdated that HMS is mainly donor funded. I think many of these things doesn't necessarily come as a surprise to anyone but it's, it's really documented and we can maybe do something about some of them. Again you can access all of these resources in the slide deck later. We have resources in our DHS to documentation that you can find on our website where we talk a little bit about what we believe are important aspects to think about for high level planning and budgeting. We have tools for doing actual DHS to budgets. Just Excel tools that you can download and adapt to your own use these tools. They give you some assistance to budget for a pilot projects. It gives you assistance to budget for scaling up to your whole country. And we also have some support for calculating end user training scenarios that's very often the biggest cost of scaling up something nationwide. When you suddenly have to train 10,000 health workers and some of them have to travel and the costs are very varying from country to country. So we have in this, in this template here we have, we have made the made a tool to help you do that. Also calculate how long will it take, for example, if you say that we would like to do this project in quarter one. But if you actually do the math of how long will it take to train all these health workers, maybe it is a four year training plan to get through it so I think this is a helpful tool. I have personally used it in a couple of countries and it's been a good, even again, maybe the number you get out here is not like the exact number with two lines under it it gives an idea whether it's this project matching to the project funds you have available, etc. So the link is in the slide deck and you're happy to to download it and just adapt and use it as you want. I will give the word to sure I did talking about another pillar of DHS to house security and compliance. I was thinking maybe at the end of everything. We can stop now and have a short discussion around these two topics. Yeah wait wait for the microphone so the zoom can hear you. And also others feel free to answer I mean there's a lot of expertise in this room so I don't claim to be the one with the answers here. Okay, good morning. I'm Peter Ricketts open solutions Dominica. I have a few questions. The first one being in terms of the management of the data dictionary. So I know that you were talking about your governance right, but in terms of strategy for managing that data dictionary because one of the things I have found. That was a challenge when we just got into DHS so you know you have these standard method data packages but they all use. They're not they're not synergized they did their method they met the data so you have duplicated data elements being imported. When you do that so in terms of that strategy what would you recommend. In terms of where in that stat hierarchy would the would that be managed you know is it operations. When the governance structure would you see that falling. The second question would be, is there a feasibility assessment tool so you used you mentioned. No particular projects that you may not consider sustainable over the long run so is there a feasibility let's say an assessment checklist that you could quickly assess. And then you know dovetailing from that would you recommend just saying no to the project or would you then recommend to the client. So they or what they need to do to create a sustainable or to achieve sustainability and then of course in your tool for budgeting. Does it also take into consideration the post implementation support that we've heard about in some of the previous presentations as well. Okay, thank you very much. I'll go backwards here from your last question. Yes, the budgeting tool it covers post implementation support so we. It's, I mean I can open it. My password on. Okay, I don't remember my password now I'm standing here and stressed. But anyways, yes, it does cover post implementation support so annual maintenance budgets like what does it cost year two year three. So you need to pay your core team salaries, you need to pay your security girl or guy a salary, you need to pay for the server, not just buying the server but you know going down the line so yes it covers that. The no versus no to project versus how to get there of course, I think it's a you should, of course advocate for for how to get there. Yeah. But my point was just like don't say yes sort of uncritically because people want something if you it's our role to sit and think is it a good idea or not on a multiple number of factors I mean ethics and security and many things right. In terms of feasibility assessment from the his center we have a tool that we call the readiness assessment, which will for countries that are starting up with the tries to it can be a useful tool to just go through do we have these things in place before we get started. I can add the link to that as well in the slide set afterwards. So using this you tries to maturity profile tool can serve as a form of a readiness assessment for for doing more advanced. It does not give you like a calculate a yes or no whether you should start something but it's, it gives an indication if you see that many of these foundational domains that we're talking about today is scoring quite poorly and you can also contact us if there are specific countries you would like to learn more about in terms of how they're scoring. I think you should at least carefully consider your project I mean you need to look at it in a holistic way. I'm not sure if that gave a clear answer. Do you want to talk to the data dictionary will have. You don't want to. Yeah, you want to start. No, I think in terms of exactly I think it's very important. Yeah, so I'm also part of the implementation team at his center. And I've been involved a lot in this packages to get. I think first of all in terms of what what level in the governance structure does it fit. I think, importantly keep in mind that this is sort of a very much a model of how it could be done and each country will be different in how much how this is set up. So I think the exact rules are, but I think it's important that there is the core team I think should not be the ones who are managing the dictionary in itself, but of course there will be important. For example, if you're considering this metadata packages, reviewing is this useful tool for us if we're implementing x, whether it's a tractor program or introducing a new aggregate sort of program. They will be useful in terms of reviewing what do we have what is in this and sort of making some assessments but they shouldn't be the ones who sort of make decisions on what is what our indicator should be and what the collection should be etc. And then I think something where I think Bob can talk more to that, but in terms of the metadata, we also need to keep in mind that especially as sort of the architecture in the country is advancing, you also have to deal with sort of metadata that is used across systems. That needs to be stable and there needs to be really good structure in place. Oh, sorry. Yeah, I'll leave the word to Bob. It's going to go over my head. It's a good question. And that's got lots of layers to it. Yeah, it's quite interesting, even if you go back to the literature from the 1990s, the very early implementations of DHS one. This question of coming together and building harmonized metadata between programs was central to everything they were writing about. When did it start? 1994, 1995. Many papers about it, Jermes, Triangle of Standards and things like that. I think it happened, the actual work that I think happens at two levels. I think there is a technical dimension to it. In terms of making sure that there are not inconsistencies and not contradictions and stuff in the actual metadata itself and over time, these things tend to creep in. But then I think the really important layer is the system ownership layer. System owners are the ones who own their metadata and they have to own it as a group, particularly if the metadata has been put onto the same system. So it does involve negotiation and formal kind of processes. And I think different countries have come up with different models of doing that, of reviewing and change management around the metadata. I think HISP South Africa, for example, a very long history with it. It's not an easy problem. And it's a problem that grows over time. And the WHO packages are probably a good example of us not eating our own dog food in a sense, because a lot of them were developed by different teams. And they didn't do what we're telling countries that we know that they should do. And I think that was learned over time and now a lot of the work has been around rationalizing and building common libraries on which tool kits are built upon. But yeah, I think it sits there between those two levels. And the really important thing about that middle level is not a simple level because you've got many different stakeholders on it. And you have to create the formal mechanisms by which they meet and negotiate and manage metadata change. So what we see in most places, it's unmanaged and it becomes chaotic really, really quickly. So yeah, it's a good point. Probably a session on its own. And also, of course, all of these decisions that don't have to be taken in the governance committee, of course, you could appoint someone appoint a working group who is responsible for ABC. So yeah, it's not that everything happens with the same people sitting in each layer. Michelle, we're your first. Sorry, I think you were first. Sorry. Yeah. Sorry, just a small clarification. So the budgeting, does it include cost country-wise or category of country-wise? Because even in countries, even if you talk of salaries or other implementation costs, you know, holding trainings and all can vary. So for example, PNG would be much more expensive than Myanmar. Yeah, I mean, you just adjust your local costs. Like how much does it cost to conduct a training in country X and you change the number and it would calculate. Does that make sense? No, no, you have to build it yourself. Yeah, it's not that advanced. No, no, I don't know the cost of training in PNG or in Pakistan or in so it's just an Excel tool that you work with. Michelle at the back. I think there's other budgets. I think what the key sort of usefulness of it, because it in terms of the actual calculations is very simple. But it sort of includes the categories that you need to include in a budget. I think that's maybe the main utility that you don't leave things out. What you must remember to put aside money for it. Yeah. I just wanted to actually following up when you talked about the feasibility and the readiness assessment. And I'm not sure maybe this already is the one that's aligned with some of the work that WHO has been doing but so and isn't here but I see Ryan might be able to. So they are doing work of kind of overall HMIS readiness assessment and it's particular for like TB where it's tracker. So it's including the some of the digital aspects but also the kind of programmatic aspects of are you ready to go to case surveillance for TB and and if you're not just what I what's what would you need in place and so it's not necessarily saying don't go do it but it's saying make sure you have go and plan and budget for XYZ. So maybe could also add that check in with and and add the links they're just finalizing that too under this piece and I know she's been trying to link it with with Rebecca and some of the pieces out of the maturity profile so that it is kind of a flow that it's both you can have the digital system IT system feasibility along with the programmatic feasibility of going to some of these fancier level systems. Okay. I think we need to move on we have several more topics. It's all very interesting to discuss but I think we'll move to security sure you're running with the mic. Good morning everyone so my name is a sure G data I've worked with his center as well. For security I'm speaking on behalf of our security leave Michael Markovich he couldn't be here. There was a session yesterday on security as well, where he's able to speak more to a lot of these topics. I think what we wanted to highlight though is when we're looking at maturity. There are a couple aspects of security that are quite important to security here and that's, you know, whether or not we consider this mature within various frameworks so the first thing that we're looking at is is ownership. And in particular we're looking for kind of decision making priority. So we have kind of data owners and technical owners but we don't want this muddy we want kind of a clear person or people kind of identified to make decisions about security and if that's not in place, and there might need to be some some work to make that happen. The second thing is kind of frameworks of for security. So this is kind of actual written policies or procedures that deal with specific security principles. So you can look at it from kind of two different sides one is maybe more broader policies and frameworks and the other is kind of more kind of practical. This might be a standard policy that needs to fit within your kind of overall security policy framework and aligned with what you're doing there. We also assess kind of implementation. So this is the actual kind of technical tools that are used to kind of implement those very security frameworks in place and the teams that actually implement those frameworks do they have the capacity to sustain this over a long period of time. You have the right resources in place to do this to the standard that you're supposed to. And the last aspect here is assurance. And what you're looking at is the ability to actually demonstrate kind of compliance to external parties or other interested parties that might be interested. There's many different security standards and frameworks from time to time you might need to be able to demonstrate that you can meet those security standards and not just kind of outside of this broader kind of framework of meeting various compliance and standards, but also kind of just being able to demonstrate how you've implemented certain components of your security framework to various interested parties just so they kind of know how things are working underneath the hood. So the results from the maturity assessments have been kind of aligned with a number of the categories I've just discussed. There is some challenges around kind of the availability of material for security and this is something I'll talk to you in the next slide but you can see that there are some issues around documentation around legal procedures actually like various bills and laws in the country that people could follow in order to develop their various frameworks. And, you know, this is something that we're hoping to address a little bit over time. And so pieces another area that we're hoping to kind of work on a bit more. So there are some some resources that are available when we one is this security starter kit. We're looking at revising this further throughout the year to contain some actual examples of standard operating procedure templates for example, until look at the data guides and some other types of information. There's also a reference to DHS to installation that this link on that I have on screen that shows kind of some of the best practices for implementing security within DHS to itself. So I encourage you to have a look at those if you're interested and also there's a session from yesterday where there are more materials on security itself. So what I'm going to discuss real quick is on the core team and training and users so there is a session on this later today. So I'm not going to discuss it too much. All we're trying to kind of promote here is that in the DHS to community we have been speaking about building core teams in countries for a long period of time. So we've tried to develop some guidance around this and what that might actually look like. So we've identified some different roles and identified their the rule descriptions, as well as kind of what their kind of contribution would be within a broader scheme of different actors, and we have a number of material on this and we can discuss this more in the session later on. So I'll highlight that these are roles, not not people so you don't always need a large number of people in all contexts and it is context specific. You know you might have small island countries with smaller populations, you might have larger countries that might need a little bit more. So this is always dependent on the situation. So, from the maturity assessment we had some some challenges here in terms of core team development and support, and I think this is also why we've been looking at kind of developing some structure around this to provide a bit of insight as to how this can work in practice. There's often a lot of issues with the core team kind of still dependent on on his support. This is an ongoing issue that we're trying to kind of actively address. We know that there's often not enough funding for kind of routine capacity building activities for this team right these are just kind of ad hoc okay let's get this training in or we have this specific project so let's try to do something, but longer term planning around this kind of core team capacity building is not often in place. And then areas like SOP is another areas where we're trying to develop more to provide information that those core teams can use as actual material in the countries, not just kind of more regional or global material which is the bulk of what we have right now. So the second part of this is then training end users, this is, we refer to end users as a separation between the core team, the core team is kind of part of that three layer governance framework that Anna showed at the beginning right they're responsible for kind of maintaining or managing the implementation at a national scale so they might have a lot of different activities that they're involved in these end users could be at various levels. They could be maybe people entering data or analyzing data at a district or facility level for example, or end users out in the community that are maybe be entering mobile data for example. They could vary based on the context, they could also have a lot more responsibility as well so this can vary in terms of their scope of work. But the whole idea in terms of training and users that we want to emphasize is that this is a continuous process it's not a one off process so whether it's that user in the field collecting community health data on a mobile phone or it's some type of administrator sitting in a province or a state performing more higher level functions. The idea is that the system is kind of continuously evolving. There's new requirements and of course there's different types of features and other types of priorities that they have to kind of respond to based on their context and and the system will kind of continue to update so with these trainings, just like with the core team it's not one time. It's an ongoing process and we'll discuss this more later on today. From the results of the assessment you can see here some of the information that was provided to us, and it's very similar in terms of the feedback for the core team where the training was kind of ad hoc this is a similar finding that we had for the core team. There are sometimes regular trainings organized but that's often not enough. There's not enough guidance in some cases or this training is limited in scope. It's not really kind of addressing all the needs that are there at present. So, we're hoping to kind of work on this a little bit more we have developed some resources to respond to this but we're working on a little bit more for the core team in particular we have a number of new resources that we've been working on including a training needs assessment, a number of templates job descriptions for each of these roles, and as well as just general kind of education principles for designing training, taken from a number of research and other studies that we put together based on what we've done. We also have a number of other resources, just some examples here. The first is the example of the template of the job roles and other information related to the core team. The second is this kind of learning paths for different roles that we've identified. We also have a lot of academy material and other types of resources that can be used at the country level for various purposes. So, I'll hand it over to a lot of now unless we want to take some questions first. Okay, yeah, sorry about that. Sorry, it's just stuck in a trash folder on Google. So, we can't access things on that. Okay, I'll make that publicly viewable. Yeah, thanks. Yeah, yeah, I'll fix that. No one can access it right there. Perfect. Perhaps quick, I don't know if this is relevant to what you presented, but it's about asking whether you inform countries about breaches, security breaches, etc, when they happen, and how, you know, they step forward to just Yeah. Yeah, we do now, right? Yeah, Bob, we have a process now. Yeah. Why are you going to come and ask all the difficult questions? I think what we've done with DHS to certainly over the last four or five years is made a move to become much more transparent about our vulnerability management process and like it. In the past, I guess I see last to say he wouldn't know. If there was if there was a security problem reported, we would quickly fix it and get it out as quickly as possible, as little ceremony and as little noise as can be. Of course, there are the rational reasons for doing that, but it's not really fair to leave people not knowing whether their DHS to version is vulnerable or not. So what we are very transparent about is if we get a vulnerability reported in DHS to or heaven forbid if a DHS to instantly gets hacked, we discover the reason for it. We've made a commitment that we will publish that all the level of detail might vary. But that also creates a little bit of responsibility now on implementers, because now they know that, you know, after we published on the website that there is a vulnerability inversion 238.3 for example. So you need to upgrade to 238.4. This is no longer now a little secret thing. It's public. And so that does create the need for responsible action and good communication with with implementing groups and we've tried to set up things like a security contact list. So most all of the implementations that we we could find and all the ones that we're in contact with, we tried to create a list so that we can give some early warning before the thing appears on the website. If people have implementations are responsible for implementations and they're not on that list. Please send an email to security at DHS to.org. And get yourself on there. In terms of informing people about breaches know we don't do that. When I've been responsible myself for helping a number of places with different security problems which they've had is not our job to say this particular country has had this particular disastrous things happen to them. So, in a sense, I mean, these are national systems as national concern is not up to us to say whether they've been breached or not. We will help them with the breach. And if the breach was caused as a result of some problem in DHS to which is rare but it has happened. And we'll make sure that we fix that I will publish that so that other people are aware that the vulnerability exists. Does that get to what you're asking. Yeah. Thank you. This is Adana from Ethiopia. And my company is a private company called out takes two missions. We are providing technical support for the ministry. And two main issues one is DHS to implementation and the other is data analytics as part of our technical support to the mystery. We try to evaluate the maturity level of DHS to implementation in Ethiopia. So, we try to see the DHS to maturity profiles that has been shared in the session. But the tools that we want to use was a tool which has a level of maturity. For example, level zero is for nothing to a level five is for optimist. So, while this DHS to majority profile, help us to collect qualitative data than to just to just based on that qualitative data we can guide the mystery to to improve the implementation. But our worry as a time was, is it really important to use the DHS to maturity profile or some other tool, which helps the ministry to prepare a roadmap for the coming five years, specifically in sustaining and implementing the DHS to. As a result, we moved to another tool, which is called Soshi. So she was brought up by the major and the Soshi to get us about five major domains. The Soshi, basically designed for health information system, not for digital systems. And we try to feed that to digital systems. Again, what's missed is the DHS to is able to live. So again, I mean, I'd be glad if you come up with a comprehensive tool which helps countries measure the level of DHS to. I mean, is this level zero is this level one was level zero mean was level one mean was level five mean I mean. If we able to know the exact module table of DHS to at a national level, which helps the country to put appropriate investment in a way forward and to prepare a roadmap. Okay, I can try to answer so I there are a lot of assessment tools out there. Good assessment tools. I think we created this DHS to maturity profile because we found that many of the existing tools, it did not really help us to make a good DHS to action plan. So the questions that we asked in this tool they're quite DHS to specific. Some is general of course that would be relevant for any type of any type of information system or health information system implementation. The questions are also quite DHS to specific. So, again, our, our aim of doing this exercise is not to score anyone or to rank anyone or to publish that you are green or red or yellow. But it's to have like a tool to have good conversations to make a good plan, and to also help us to say that, or help everyone to say that it's feasible to do this project that you have in your priorities now. I know I answered your question but my answer is that there are, this is not the only tool out there there are many good tools and I'm not saying this is the only one you should do to work with each eyes to but it's, it can help you to make quite DHS to specific action plans. I think maybe we should move on to the next topic. Just the final comment to add to what I said I think another thing is that it wasn't the purpose to build it sort of covering everything but to have something that can also be done relatively quickly so that you can rather do it again and see how things are progressing. So rather than have something that takes three weeks and you do it once we would rather have something that takes three days and you can do it every year and actually sort of use it again for planning not for assessing but for guiding the areas where you need to improve. Okay, so I'll talk about two areas, sort of foundational areas that are somewhat related here now. The first one is what we call the facility and population profile. I think the name is not the best. But what we're talking about here is that in general sort of as a across the whole DHS to implementation. There is a need to have an idea of the population so denominator data. There's also information about the health facilities. So some basic attributes, which is sort of the foundation when you're doing additional analysis sort of at the programmatic level you need to know how many people live in different districts the facility catchments, etc. So that population estimates. One element of that is whether in the choice you have any linkages to the CRS system. So if you have complete CRS data, are you able to link that to the choice and use that as your target population denominators for the health indicators. So far, not very often that we actually have, first of all, complete CRS data second of all that is actually linked to the choice to. But then at least there should be population estimates typically from the census data available kept updated in the choice. The second is in terms of maintaining the facility, facility information. So do we have an updated facility list in the tries to, which is again sort of a foundation for all the other stuff we're doing in the tries to if you don't have your organs in place and they're not updated. That sort of creates problem across the board. And here is around the human resource data so whether there is information on the staff working in facilities. And whether that is kept updated whether you have sort of a sensible breakdown into different stuff categories. That's sort of the last element of this facility population profile. And what what we're seeing here from the maturity work is that CRS is something a lot of countries are looking at, but it's not really happening this linkage to the CRS systems. And then there are some attempts, for example, if you're doing a child health trackers immunization trackers to try to set up a link between the births and the CRS for example to send notifications. But it's quite limited. So the population estimates more sort of from the censuses countries report it's available. In most cases, at least down to the district levels, but then there are often challenges in, especially at the facility catchments to keep that information up to date and available. So around the facilities. A lot of countries have the geographical coordinates for a lot of the facilities but there's always not always processes in place to sort of add new if there's a new facility to actually get those coordinates in etc. So often the updating and maintenance is the problem. So I think in the plenary session on maps yesterday there were a lot of examples of tools that can help in some of these areas. So for example, you have the ability now with the population layers in the maps. You can calculate catchment population and import and have them available in DHS to. For example, using this micro planning app. So there are a few examples of how CRS systems have been linked to tracker programs. So there are a few resources here that you can look at in this area. Another thing. I'm going to confuse you a bit now because you also saw in the map session yesterday is that we have this organization unit profile function now in the maps app. The facility profile that we're referring to in the. As this sort of domain. In a few minutes john will show us another app called the organization unit profile app, which is a tool that he's been on team is working on to help with the whole organization unit sort of management process. I'll leave it to John to talk about that in a couple of minutes. Another thing we sort of talked a bit about earlier the metadata governance and how that is done. Another element that we're looking at in this maturity profile is what we refer to as a metadata quality. And the org unit quality. So there we're not talking about whether the indicator list is a good one whether you're collecting the right data. Here we're talking about sort of more from the DHS to perspective is the way it's been configured and maintained. Done properly. So we're seeing this metadata quality it could be for example that you've somehow managed to end up with a configuration in DHS to that is invalid. Which can cause functionality to break for example which can prevent you from actually being able to upgrade to a new version because there are invalid configurations. So that's one reason this is important. Another reason is that from an end user perspective, if you have the tries to system with thousands of data elements, hundreds of indicators, and it's not properly sort of managed and structured. It's very difficult for the users to actually find what they're looking for and use the tries to for what they needed for. So as I come back to we have some tools to help sort of assess this stuff. Including some assessments I think this, this is kind of if you think of the government model this is typically the responsibility of the core team to make sure that the way this is done in terms of naming conventions and groupings, and making sure sort of the sharing of the tries to set up so that people have access to the relevant stuff. That's part of the work of the core team. But we see in some cases that it's, it can sort of grow out of hand and then you need sort of bigger initiative to actually try to clean it up. And so what we've heard now from the countries that have been through this maturity profile is that it varies a bit some places, they say that the quality is okay. So quite a few countries who are saying this is a problem and actually sort of hinders users of the data to do what they want, because it's difficult to find the relevant content. And that is creating problems, for example, for the upgrades, as I mentioned. So here we have, like I said, we have this data integrity. Metadata assessment tool, which is one sort of standalone tool that we made for looking at a lot of, a lot of these sort of quality measures of the meta data. And there is also, I think, underutilized function in the tries to for doing some of these data integrity checks, which actually covers a lot of the basics but I think it's not something that is used routinely in many places. And we also have some more general documentation around best practices for metadata maintenance and assessments. Next is the organization in profile app, which I'll hand over to John to talk about. Hi, hi, my name is John Lewis from his Vietnam, a part of his special hub and part of his deposit. So that's how it is. Just to give a why we needed this, why we build this organ profile was many of the countries had not one DHRs to but multiple DHRs to, and then the core team were managing the DHRs to and maintaining it, not from one place but multiple places. And sometimes what happens they create an org unit and then forget to include in a proper group, and then forget like we have the same convention of the naming convention or put together different group sets assigning all these things. It was creating lots of problem. And what people were trying to do is either create a new DHS to instance, only to keep all their organ in there, and then synchronize across all the places. We used to have build their own system or use some other system to make only the mass of the list or the organ profile and sync across all the places. And that means like you require a different hosting things you have a different team, different way of updating and all kind of other other ways. And then also to, every time when you have an organ, we also have to update the, the, the coding system, like sometimes malaria call this organ is different coding, TB call it is a different coding. We know in DHRs to we make it as an attribute but like it's not been updated in across all the different places. And that's why we just say, okay, let's just let me get an app built inside DHRs to which can talk across all the other DHS and that is possible in DHRs to version 240, so that like we can listen to the what's happening that will be changes and we can manage those things in different places. That's one particular point. The other point was also to analyze the data what we are collecting on as an organ profile. In DHRs to one stage, during the access we already have the main thing called organ profile where the first step was to address the what this organ it means, whether it is a private public NGO where it is located and those where it is a secondary field. And now most of the time like when we install in DHRs to we don't even have those groups and groups that defined. So something to give a bit of structure of how do you manage and maintain your organ. So those were the some things we should get ready to use. And also how do you analyze this data what we call semi permanent data which is infrastructure human resources and how best we can try to link together. Yeah, so that's why we fight you to build this up which can give not only the DHS to 40 but also the managers just saying how many organics we have and we classify the organ usually from the long history. One is service delivery unit, one is Arbus city unit which is province and other things and then we have offices like PHO Malaria office anti Malaria office and usually they have different different name different people so we need to combine all these things together to make a bigger place. And then during the COVID what happened like we also use the, the prison, the shopping malls and all the things for the vaccination and for COVID system, it was an organ it, but it not necessarily the organ in your national HMR system. Sometimes you will use the temporary organ it for, for your own data collection for your own health program, how do we best manages all these things up. And then we coming down to the villages like more and more DHS to are having this catchment area and collecting the data based on the village, how do we best manage all these things so that's why we were trying to use this functionality. I'll just like quickly go to the demo itself instead of like going on to the PPT. I hope we can share. I'll be fine for the online. Yep. And as in DHS to first step what we are trying to do. I had logged in, but like in DHS to we have the time out. That's good security. Usually when I present any kind of app or it thinks I'll go for the analysis to data entry, but this time I'm going from the data entry to analysis just to give a few, few details. I hope you can all see the screen. Like, this is the we have different modules one is service delivery. So this is for the end user when they're doing the data entry service delivery is any facilities which are providing services and I'm just a unit is basically province district and villages and all. And department on offices which includes all the other places and each and every type will have a different set of rules and things. All these things are our unit again. Okay, so quickly I'll just show you. This is one health center where we have the basic information where it is located and this health center is present in this isn't based from law. It's also present nature must go out loud. These are all three instances where they use and the code are the same. This is the reporting hierarchy and location hierarchy in sometimes we need to have both. This is also the same question which Nora asked me last time. For example, center level of hospital they will not report to your province, they will report directly to the national Ministry of hospital, but then I'm like that's the reporting hierarchy. So sometimes we need to have both. In this app, you can also see what all the different catchment area of this particular villages is there access details those are all different things what you have infrastructure staffing details service details what kind of services they're providing. All these different things, and this key data key data is something which we are not entering but importing pulling the data from other DHS instance. I apologize for the one thing which is called UPS assessment done by the external people and validate all the things which has different kind of scoring system for whether we have infrastructure facilities human resource management, and plus some key data like last year. How many patient care happened and all these things and these things are imported from different instance of DHS to around here. The same thing with the villages. I just want to quickly just show you one thing which is so this is the village which present in different in law there each village has different coding system and then every year the coding system will change. Last year the same village was called different things different coding with the same name but this year it will be changed. So we need to manage and maintain all those things. Because like we use this coding system to map to the different health facilities. And then like in in law there also did the this assessment like the health worker will go around to the every village and say how many family members are there. How many families are there, what kind of health insurance they have what what is the primary source of water sanitation and also ethnicity. So based on that one they can try to calculate all the things again this is imported from different DHS instance to around here. Quickly I'll go around to the analysis the first thing what we try to do it was to give an overview of what exists in DHS to with all these four different you know three different instances of DHS to see. So here. So first thing is just like how many. This is something for the manager, we just say 88% of your health center have all the equipment for essential health services and only 80% of your health center has essential drug. And what is your hospital to population ratio of the health center and the doctor that you already have. And this is the ownership, which we already used to have every time in all the DHS to whether it is owned by a government private public or PPM, and then the location urban semi urban and all these things. And then how many health center have adequate human resource. This we are pulling up from different area. So quickly, one quick thing it's about in law they did this assessment for all the province from the World Bank and from the team they the province people will go around to the health center and do the assessment in here what I'm just not showing is how many help facility have adequate equipment for essential services how many people don't have. And the gray one is the assessment has not been done. This is something in giving a full overview of what's happening. This is a filter. Like for example, one of the best way of trying to deal with it. Like this I'll take in patient daycare. Give me a list of all the facilities which has in patient daycare more than let's say 200. Okay, and then like I'll just say, okay show me more than 200. Let me just like apply this out of 5,000 2500 service delivery unit. 248 have more than the 200 in patient daycare now like what I would like to do is to include give me list of all the facilities where in patient daycare is to more than 200. And then the nurses are less than or equal to do because like we need to see like, how what's the ratio like how many people have if you have more that's one thing but we are just like I understand. Less than two. So these are all the help facilities we have less than two or two or less nurses and they have more workload. And then you can also include. Let me just show you one more thing. When you filter you get the list from here and you can also see in the map what all the different places, how many if you just see these are all the areas where the in patient daycare is not so much. And then you can just see around different places. See the hospitals or the things around here can get to all the different details. Now other things which you can try to do. Okay, this is fine but let show me list of all the places where the less human resources less but like I want 80% of essential service. Let's say get done equal to some any facility which has adequate equipment and drugs, but less nurse and then they are for the in patient daycare is 200. So if I just see the list. Now only 70. That means if we can focus on this 70 facility and increase or have allocate the proper human resources, we can try to get the good quality of the data so one of the whole point of the organ profile is not only about managing the DHS to coat it but also giving the access to the to the admin level people to just say where which area they can try to do it and then we can get the data from multiple DHS and multiple programs so that like we can try to do the analysis from here. So like I just bought this go to the map. And just see these are all the areas are good. So here this is the north and the south. So where you can probably just see all the different details. And when you come down you can just see the all the access to the facilities and everything, whatever you have here. So we also have what kind of road they have how much far from the district, what kind of infrastructures they have. And whether the internet is there, how many things are there stuffing details and all the things and what kind of services they provide. You can also filter by just give me the list of all the hospitals which are providing mch only. So you can get the other filtering. That's basically what I wanted to present. Yeah, so we have a very one box in our maturity profile. Maybe I'll do that quickly and I'm sure there are questions for you, John. So then we can take the rest of the time for the. I know that's just very briefly on this sort of the. The last thing is the infrastructure and if we're thinking of the chest infrastructure is sort of two things one is the central the server basically which could be one or more. One or more servers. The other is the infrastructure for the end users, which could be laptops desktops or it could be phones tablets depending a bit on the implementations and the purpose. I think, in general, what we recommend is to always sort of build on what is already there that's sort of one one recommendation we have in general when when planning the implementation. And not sort of be too fixated on okay it's a new implementation we need to buy a new devices for everyone. Maybe some of it is there maybe it can be sort of a hybrid where computers use that facilities with computer use those sort of thinking of how what is already in place can be leveraged in implementation. Another key thing in terms of infrastructure is to actually plan for the maintenance that's something we see that is often forgotten we have a project, we're going to buy 2000 devices. But we have no budget to replace maybe you need to replace the third of them every year from the implementation. That's not part of the plan. So, and servers needs to be paid every month every year. If you have a physical server that also also needs to be replaced at some point. So thinking long term and not just sort of we're going to roll out this in one year and then that's as far as we think. Yeah, also the infrastructure of course sort of obvious but sometimes forgotten that as you're sort of extending the details to into new areas maybe you're moving from aggregate to also introduce using tracker that actually has implications for infrastructure. So you might have a server that has been working well for five years for aggregate, but if you're then using the same server to roll out the tracker program which has far more data for more users. That has implications also for your hosting so you need to plan for that as well. Specifically also in terms of the central infrastructure the server. That is in practice not only about sort of your decision do I want to use the cloud do I want to have it in country in the data center it's also about what skills are needed and what are the costs of those sort of a complex assessment that needs to be done with different implications so what is the cheapest is not necessarily feasible unless you have the people who can maintain the cloud infrastructure for example so maybe you need to outsource more. Even if it's more costly if you don't have the right skills in the ministry to maintain it in the core team. So I think no big surprises in terms of what we're seeing in countries here there are course variations. One thing I would like to highlight in here is perhaps this idea of it support that's something that many countries I think I've reported. There may not be devices but there is no structure in place for actually helping the users out in the districts in the facilities if something happens to their device or their account. So maybe one thing that standing out a bit from this assessment was sort of a lack of a proper structure for providing support on the infrastructure. Quite a few relatively new academies presentations sessions on this. This is linking to presentations but we will also have recordings of the actual sessions on some of this. And then we have in the Android implementation guides. Something around the Android specifically in terms of the what you need to consider and also specifically on server hosting. Okay then I think we have five minutes maximum for a couple of questions. So about the org unit profile app. First of all you had me at I can manage multiple instances with one app I am in love that is brilliant. But I'm really interested in how the data is stored because I can tell some of it's being pulled from data sets some of it uses org unit groups. And some of it I'm guessing is custom attributes and one of the issues with those things or I'm guessing you probably come across this problem. So how you've addressed it is then being able to track things over time. So if I've got my staff in my facility right now and I want to do some sort of analysis can I also see what my facility staffing was two years ago and do a comparison. Yeah so actually like it's them as you said suggested it's exactly the same. It could happen. Thanks to like all the DHS to platform team. They're made in 240 so that like you can link multiple DHS instance or just you can pull. So one of your DHS instance should be 240 restaurant or things if it is above 36 is more than fine. So what we try to do it was is for the key main things when we create an organic profile. These are mandatory field. Like for example, the groups like the location, the other things at least has to be seen there. It's an it's filled fields will be there but it's optional for you to select whether it is urban ruler or things that you can try to deal with it. And some attributes. What we are trying to store we are not storing as a data element attribute but we are storing in the in the data store itself of the app so to try to use the things because if you create because this is an app which will be which can be installed in any DHS to the later we want to make it as generic as possible so that like people can install this app and then they can point to different DHS instance where they are using up and then they can synchronize the data so you can you can use it as a organ synchronization not only that one but also organ group and groups that about pulling the data you can what we've been thinking is to you can specify the data. For like okay give me only last year data or last six months data because we don't want to we want to encourage people to use the DHS to analytical tool. But this is just to follow your comparison to try to get the data in and when you specify those data whether it is from this in this instance use take program indicator. In this instance use take aggregate data value, and then we we pull it and store it in as a tracker. So the behind the argument profile app is a tracker data model. And that's how we can try to manage and then we can also analyze the data by two different hierarchy, one based on the reporting hierarchy one based on the location hierarchy. Okay. So that means because it's tracker data, it's not just available through your own analytics available through any, any, any place to dashboard. Yeah, and then you can like any kind of changes you can also monitor over time and also audit. There's one there. Just a quick question and it's applicable to your app fantastic app by the way but just in generally apps in the app store. It does DHS to us on UIO do an assessment of the long term sustainability of these apps so let's say you adopt one of these apps and but they're being managed by, you know, other organizations. How will, you know, countries know or be able to trust that this would persist over the long term over the long term. This is the app, which is a collaborative effort by his special hub. That includes his India his Indonesia Bangladesh Pakistan as a whole. And one of the, this was from the global fund and the Gavi things we have created this is special hub. And this will be the one of the product which will be going to manage and maintain for all the different releases. So that's one of the ownership which we will take. That's that's in already in the plan. Just, but I think you also said sort of in general, and I think that's a good question that we are discussing how sort of how to improve the maintenance sort of transparency into the apps and what is the plans what is the roadmap, etc. So it's a good question and there are some apps there that has been maintained for many years there are some that have sort of been released and then not really without having a clear roadmap. So it's something we're discussing how to make more transparent. Thank you very much. This question relates to the org unit profile, or two questions. One is how would this relate to a country's master facility registry because it does look like a master facility registry. The first question. The second question is, we know that for planning various population data is used so there might be for reporting it might be the statistical data from the statistics office. For campaign planning, it might be different population data that's used again for ordering drugs. It's again different data that's used. So how would that be accounted for in the profile. Thank you. So for your first question, like it's about it is master facilities but the problem is in DH as to what we call as an org unit which also includes province district which are not part of Ministry of Health. So what we have done is to write down a bit of manual just saying what all different classification what do you think. So there is already a master facility list is perfectly fine for the for many countries where you don't really have master facility list and all things and you're trying to maintain that when a different instance and merging together. So that's when like we just just say this can be a helpful tool to manage all the other things from your side. And also time, especially they talked about the linkages. So in master facility list, usually they talk about location based reporting, but not the, the how we are reporting how the people are reporting to the hierarchy. And that's also very, very different most of the places we haven't seen that, and in DH as to it is reporting based hierarchy, not only location based hierarchy about the second point about the population. We still use the normal DH as to for doing all the analysis and all. What this have is this in terms of your organ profile what data is key to include it. So it is other way around of thinking if the, the campaign managers want to try to assess the data so they have their own DHS done one way of trying to deal with it that's fine. But like what data from there, it's more relevant for the organic management. And like you try to come around here. And then what you want to try to quickly show okay in my facility, 80% of the people have been immunized that can be pulled directly from the statistics. So here that's why like we want to just say, these data is pulled from your routine health information system. It cannot be modified in the argued app. Just a reminder that there will be lots of noise in a few seconds I guess. Yep. Perfect. So that was the lunch signal I guess. Thanks.