 When we log in, you'll see the first thing is the data quality dashboard. This is the dashboard that Bob was actually talking about yesterday. We can even see here the same chart that he was talking about where you can see the clear outliers. Actually, tomorrow we're going to look at this chart in more detail. We're going to go through how to actually drill down into this data to identify where these outliers are coming from. But for now, to get to the WHO Data Quality app, we just go to the apps menu, the WHO Data Quality tool. It's going to take just a second to load. And what Bob was just pointing out is the outliers tabs. You see we have four tabs at the top. The top one is the outliers. And just to reiterate, this is probably the most useful functionality of this application, to be completely honest with you. This data is coming in every month. It's something that you can continuously look at. You can constantly keep an eye on this. And as new data comes in, as new outliers come in, you'll be able to quickly identify them here and drill down into them and take the appropriate steps to correct them. So to show you how we drill down, just quite quickly here. If we look at September 20th, we see that we have two red highlighted cells here. These are telling me that these are outliers, that these values are very different than the average. And you can look and see that the average over the last several months is around maybe 30,000. And then in August, we jumped to 59. And in September, we jumped to 88,000. Clearly something's going on here, especially for HIV test done. It could be maybe an outreach campaign. There could be a real reason why you have this big jump. But more often than not, it's probably going to be an outlier, or maybe potentially a data quality check. So if I just click this expansion menu here on the far right, click Visualize, then you can see the data. And you can see that, yeah, August and especially September are very far outside the norm. So what I can do now is just, again, that expansion menu, click Drill Down. And it looks like I'm actually already as far down as I can go. This is the regional data. So then if I click Contact, it's taking a second to load here. Let me just refresh. It might have been just because of all of you. Yeah, okay, I timed out. Let me just get back in here. Come back to outliers. Here we are. Gonna drill down. Yeah, okay, now we're drilling down. All right, so I've drilled down from, we were in C region. Now we're in seeing the districts within C region. Again, we have our HIV test done, and it looks like District C4 is the major culprit here in September of this year, 44,000. Let's just visualize that again. Yeah, that's looking like a pretty clear outlier, isn't it? Let's drill down one more time. All right, now we've drilled down to the individual health facilities, and we see that facility 365 recorded a 33,000. Most months, they're recording around 150 to 200 HIV tests. One month, they recorded 33,000. Again, if we visualize this, boom, that's clearly an outlier. There's no way that that health facility reported or conducted 33,000 additional HIV tests that month. If we, can we drill down again? Nope, we cannot drill down again. We're already at the lowest level. So what can we do? Let's contact, right? So here it's saying that I can send a message to the users that are associated with this facility. So what does this message do? But I don't think Bob actually explained it. If I send this message and I say this, then it will send this message to the contact details of the organizational unit, well, the contact details of the organization unit. So if that's a, you can also select a various user groups to send the message to as well. And what this again does is it will, the message will go to the DHIS Tube Messenger app. And if you configured it, it can also be pushed out through email as well as SMS. Those are different configurations. And we'll talk about how to do some basic configuration for email alert notification pushes tomorrow. But that's essentially what happens here. So it's a real quick and easy way to communicate to the folks who are responsible for correcting this outlier. All right. Nora, are there any questions coming through? Got to make sure I'm not talking to myself. Yes, there's been quite a lot of questions on the Slack channel. There was someone asking about explaining what is a Z score and a standard score. I said, we'll get back to them. I'm not confident enough to just let it go. Just straight feedback. Then there were questions about, can you set this up to provide information for HIV or TB program? And I responded that you can set it up to analyze whatever data elements or indicators you want. And then there's a question about, does this work on indicators? And I'm trying to say it works based on count indicators. Will you sum up an age or gender desegregation? Yeah, absolutely. I would recommend probably not actually using it on indicators because indicators are calculated value. Your outlier could then be dependent upon the numerator or denominator or any individual component of that indicator. I would much more recommend using outlier analysis on just raw data values. Be more effective that way. That's right. Because if it's a calculated indicator, you just don't know what's underneath it and you could go off on a wrong track. Then there was a question about, if we give this data quality to access to at provincial level, would that provincial level be able to see the whole country or just only their user or unit? I explained it is only their user or unit. The one that they have access to now is what they will get. Great. Okay, hopefully that's clear to you all. And if you want to know how to configure this tool, that all of the content for how to configure the tool is in the asynchronous session number one. I'll show you that now. So I'm going to get out of this video. I am going to go back to the broader course. Content, scroll all the way down, and then we have asynchronous session. Oops, sorry, went too far. Asynchronous session one, WHO data quality. These videos and PowerPoint slides here tell you how to configure this application. So if you're a system administrator, these are the videos that you're going to want to watch. We can also start to cover some of these today as well if need be. There is also a, yeah, sorry, go ahead, Martin. Sorry, you're watching this in studio. Could you watch it in the live version please? Yeah, thank you. Right, okay. Yeah, so asynchronous session number one here. These are all of the, oh, sorry, this is not it. Yeah, these are all of the sessions that tell you how to configure this tool. But also keep in mind that in our today's presentations, we also have reference here. One click away from you to be able to go in and find the configuration documentation guide. We have a brand new configuration guide for this application just rewritten in the last couple of months. It's very useful. Please use this one. There is an old one circulating around that was written by statistics Norway. If you see that one, please do not use it. It is not a useful tool. It's not a useful guide. Use this one here. Okay, all right. I am now then going to keep on plugging along here. And we will now start watching the next session. Long as Bob went through the analysis functionalities and menu of the WSA data quality app. Nora, how are we doing on the questions on the community of practice? Or the slide channel, sorry. I think that there's quite a lot of questions about what can you see and what can't you see and having to explain that you have to set up the WHO DQ tool in order to see what you want to see. Doesn't come in inverted commas preloaded. You have to do that. I think there is one debris, Derevi is saying that he can't get the data elements drop down list to open. I'm trying to troubleshoot that one and see if I can figure out what's going on. But that's, I think it's mainly about can you see anything else besides what Bob is showing us? Can you see other data elements or count indicators? And saying, yes, you can. You back in. Yeah, you just have to, it's actually part of the configuration process that you tell this application what data elements you want to see to be able to use in this application. You can turn on any data element that you have in your system to be used in this application. You can turn on whole data element groups, all the data elements in any dataset if you want. That's completely up to you. This does not come pre-configured. It's not going to provide you with anything is when you install it for the first time. So you have to do the configuration and the mapping just like you do with everything else in DHIs too. It makes it a very flexible tool, but it does mean that there's some administration that's necessary on your part. Let's just take a look at this particular question on not being able to select the data items. So now I'm in the analysis tool and I'm going to choose, I'm going to leave my analysis type as between indicators and overall results. Then I need to go down and select my data items. Here I'm just going to look at ANC first visits and let's just look at another one, ANC fourth visits. Then just like you're using the data visualizer app or the pivot table app, you have to select your periods. So I'm going to say years and I'm going to leave it 2020. And then as always, we have to select our org units. So right now it's just set to national. When I click analyze, this is what I get. You see that it has made the ratio between ANC one and ANC four, making that comparison simple scatter plot and a data table to associate that. If I go in and say edit my org units and say instead of regions, I want to look at facilities. Click analyze again. It takes a little bit longer because now we've just looked at the scatter plot. We've turned on all of the facilities and we can see that, yeah, there's a few outliers here that are way off. For example, this facility 147, they reported 39,281 ANC one visits, but only 268 ANC four visits. Now that's definitely not correct. So you can see that this ANC one visits data for facility 147. Probably it's not accurate, probably a data entry error there. What other issues? The question about username and password, the username and password is provided on the login screen of the instance. It's demo and the password is district one, hashtag capital D in district. I can have, yeah, Nora, something you want to say? No, there's nothing I want to say. I think that the other thing to emphasize about this consistency analysis allows you to look at the relationship between two linked data elements. Sometimes an outline analysis will not show you a problem. The data looks normal, but when you look at this, look at this relationship between two linked data elements, you may easily pick up, but something is not right. And so use this consistency tool to look at relationships between data elements. An example could be malaria or DT, tested malaria or DT confirmed, would give you the relationship. Right, yeah, exactly. That's really what we're doing is we're comparing data values that have something that's related between them. So maybe in this example here, I've just changed my analysis type to look at dropouts. So you see I changed dropouts. And now I've turned on the OPV1 given and OPV3 given. So what's the dropout rate between OPV1 and OPV3? So if I leave it on facilities, I look at the data across the entire year, you can see very easily those health facilities that have a negative dropout rate, meaning that they have higher OPV3 than OPV1. Well, certainly if you're looking at cohort analysis, this could technically be possible. But for most data systems, most countries, it's going to be very, very unlikely that you have more OPV3 than OPV1. So could there be data quality issues? Absolutely. Do you need to investigate them? Yeah, absolutely. Okay, all right. So let's just keep on plugging away. We've got, sorry, Nora, one thing. Yes, a question. Is there any use of the WHO Data Quality App when collecting case-based data? Or does this app only work for aggregated type data? The data that you are only able to put into this application is aggregated data. Now, that does not mean, though, that you can't aggregate case data and use it in this application. That's a very common practice. Most countries that are having high-volume case data for something like HIV monitoring, maternal and child health, immunization monitoring, that kind of stuff, disease surveillance, monitoring individual malaria or COVID patients, that kind of thing. You would need to have that data aggregated into an aggregate data element, and then you can use it in this app as well. This app does not use tracker data elements. We can't really do this kind of analysis on individual patient data, like if we're looking at individual patient level. We can only look at it at more aggregated levels, like facility or district or region, and more aggregated over time, like months, quarters, years. Okay, any other questions, Nora? We've just got one user who's struggling to get the sequence activated, and I'm trying to see how I can help. Is this Derabri? Yes, yeah, Derabri, yeah. Yeah, I was trying to also demonstrate. Yeah, I have to go offline to help. Yeah, it could be a connectivity issue as well. Yeah, so there was a question that just came in. It said, just for clarification, does this mean you need to configure program indicators to be used in the data quality app or event or tracker data elements? No, that is not the case, actually. What you need to do, this app will only use aggregate data elements and indicators. So that means you need to convert your tracker data into aggregate data. You know, if you capture your data as tracker data elements, you would need then to have some mechanism, and there's various apps out there to do this, but you need to have some mechanism to convert that tracker data into an aggregate data set. And if you're able to do that, then you can use it in this application.