 All right, so today our first session before our break is going over DHS2 data quality validation rules and notifications, okay? Again, this is gonna be fairly technical. I'm going to have to go quite quickly through this. So if you fall behind, please ask a question on Slack. List video will be posted later today, hopefully, maybe tomorrow. And so you are certainly invited to rewatch it. You can slow it down if you'd like, and make sure that you're capturing everything if this is something that you wanna implement in your country or project. What are the objectives of the presentation? Well, essentially, we wanna understand the logic behind validation rules. We also wanna give you a quick overview on how to build validation rules. This may be kind of a recap for some of you. I know many countries already have a lot of validation rules. I think we're gonna expand your appreciation of what validation rules could actually be used for, but the key message is that we need to have data quality coming to the user. So right now, all of you are using your email more than you're using DHS2, right? I certainly do, and I'm one of the DHS2 product managers. I'm looking at my email every day, almost all day, and we need to start to build in processes where data quality and even just alerts for data start to come to people's emails. That's where they're spending their time. They're not spending their time so much on DHS2 dashboards. And there's just two little jokes here. You can look at these yourself, but the whole point being is we need to start sending data to where people are spending their time. Certainly, we need to start sending notifications for any kind of major data quality problems to where people will actually see them quickly, which is probably going to be their email. Again, send data quality issues to the user. Over the last couple of days, we've been looking at the various data quality apps, the WHO data quality app in particular, and we appreciate that there's a lot of functionality there if you go into the application, right? But we also have to appreciate the fact that many people are not going to go into that application very often. Many people are going to not feel compelled to dig through the data to find the data quality issues. This is where we as system administrators, as DHS2 managers, as health information system managers, have to be proactive. We have to set up this system to be able to push data quality alerts and notifications to where people, again, are spending their time, which is probably their email. You can even set it up to send it via SMS. We know that data use is low, and we talked about on Monday how one of the driving reasons for this is because data trust is low. There's actually a whole lot of research out there that makes this point very, very clear. That people just don't trust the data because they're not able to see the data quality issues, right? And what are some things that we can do? Well, again, data quality issues are not a failure of the system. They're a feature of the system. Any kind of large project that is using DHS2 will have data quality issues. The failure is not being able to do something about it. The failure is to keep people unable to see the data quality issues, right? And so how do we do that? Again, probably for the fifth time already this morning, we need to start sending the data quality alerts and notifications directly to the folks. DHS2 is smart enough to actually send people alerts and notifications directly to where they're spending their time, email, SMS, even WhatsApp. And the point of this presentation today is to start to train you on how to configure DHS2 to actually do this and also introduce some best practices. The final point here to make is that the alerts and notifications that you actually send to people need to be based upon some standard operating procedures. We saw in the case that Pamoud presented in Sri Lanka, they have very clearly defined standard operating procedures. On Monday, Andrew from his Rwanda is going to present on Rwandan standard operating procedures. Many countries have very good defined standard operating procedures. Many countries also have very poorly defined or don't even have data quality standard operating procedures. You need to have these standard operating procedures in place and the notifications and alerts that you start to send from the system need to be in agreement or in alignment with your existing standard operating procedures. You don't need to have people guess what they need to do when they receive an alert. The alert should tell them what they need to do. And if they're confused, then they can refer back to their standard operating procedures that should make everything very clear. So how do we start to send the alerts? Well, the best way to do it is through validation rules. So quick intro to validation rules. Validation rules are a way to assess the data that has been entered in DHS2. And it's very base sense. They are based on predefined rules. You define the rules in the system and we're going to show you how to do that just a minute. Validation rules though, do not fully allow you to check whether the data that is reported is complete or accurate, right? It's just going to be base, it's just going to tell you, does it fit this rule that you have defined, right? So the data could still be wrong and not trigger a validation rule just to make that point very clear. But we are asked quite often to do various HMI assessments. And one of the first things that we usually look at is do the countries or do the projects have clearly defined validation rules? If they do that, and are they being used routinely? And if they are and they do, then typically what that means is the country is taking data quality very seriously. If they're doing that, then we can hopefully make a little bit of an assumption that there is some data use or data use is actually happening in the country. If countries or projects do not have clearly defined validation rules, if they are not using them routinely or they just don't have them at all, there's probably a fair, it's fairly safe to say that they are not taking data quality very seriously and that they do not probably have very much data use going on. And again, why is that? Because people are not able to see the data quality issues. There's no transparency. There's no data trust. And these are things that the validation rules can really help with. All right, let's get in a little more technical here. What exactly is a validation rule? It's an expression that defines a relationship between a number of data elements, essentially. The expression has a left side, a right side, and an operator in between. We can kind of think back to our grade school math days here. We have a left side, we have a right side, and then we have an equals greater than or less than in between those, okay? You create validation rules based on what you know to be true. This is very, very important and I'll say it again. You create validation rules based upon what you know to be true. The reason that you need to appreciate this in DHIS too is because in other statistical analysis software that maybe you're using, this is the opposite where you make validation rules based upon what you know to not be true. But in DHIS too, you define validation rules, what you know to be true, what should be happening, okay? The example here is malaria-RDT positive, the number of positive malaria tests should always be less than the number of malaria-RDT tested done, right? How can you have more positive tests than you have total number of tests completed, right? If you have more positive tests, then you actually have total number of tested, there's an issue, there's a problem there. It's either a problem with the malaria positive, the number of positive tests, or an issue with the number of malaria tested. But clearly there's an issue because you know that you can't have more positives than you have tested. So this is a very simple example. When to use validation rules? Now Nora is also gonna talk about this a little bit later, I believe either tomorrow or next week. But validation rules can essentially be used in three different times. They can be used during aggregate data entry, so data set data entry. I think we'll be able to show you an example of this a little bit later in the week, but validation rules can be triggered while data is being entered. So for example, if I enter more malaria positives than malaria tested, during, you know, if I'm at a clinic and I'm putting that into my data set, a validation rule can be triggered and saying, hey, you've entered data that should not be possible. You should go back and check this, right? They can also be used to block data entry altogether. So we can say that if they enter more positive than tested, don't allow them to submit that data. Now that can be a good thing. It can also be a bad thing. You know, the general rule of thumb is allow users to get their data into the system. Send them alerts and notifications during data entry, if necessary, but typically you want people to be able to put their data in the system. Only for probably the most highly trained users, maybe folks at regional or national level should you disallow data to be entered that does not pass. That's general, the best practice for rule of thumb. Validation rules, second bullet point here, validation rules can also be run manually. So you can run data validation rule analysis at any point. You can do this before you do a supervision visit for an individual health facility or a group of health facilities or for a group of community health workers. When you run it manually, it can also trigger emails or SMS alerts that can be sent out to those who are assigned to the organizational unit that has prompted or resulted in the validation alert. So we'll cover that a little bit today as well. The last point is that validation rules can be scheduled to run automatically and this is probably the most useful functionality of validation rules is that you can schedule validation rules to run periodically, say weekly, monthly, and then those can send again alerts, emails, SMS messages to the various users who can actually go in and correct it. All right, we, sorry, go ahead, was there a question? Nope, just an unmute a microphone for a second. All right, so let's talk about how we can make a simple validation rule. Now I'm gonna show you this through PowerPoint slides, but you can in the exercise go in and do it yourself. This PowerPoint slides are just direct screenshots from DHIS too, okay? So let's make a simple validation rule. Here we are looking at a bar chart that's showing A and C one visits, A and C two visits, that's blue, A and C three visits, that's red, and A and C four visits, that's orange, okay? And if you're not familiar with anti-natal care, then essentially there are four visits to the clinic that every pregnant woman should make and these are to receive different kind of clinical services, vitamins, minerals, checkups, that kind of stuff while the woman is pregnant. And typically we have relatively high A and C one visits. Most women go to their first A and C one visit, but over time throughout their pregnancy, they go to fewer and fewer of the visits and by the point we get to A and C four, we typically see much fewer women going to the fourth visit than went to the first visit, okay? Now some of you folks out there who work in maternal and child health, you're saying, wait a minute, these are different cohorts. Well, yes, that is true, but the model does play itself out fairly consistently and only in certain countries do you have which may have like more like seasonal pregnancies. Does this not actually, model doesn't hold true, but for the most part we almost always see that there are fewer A and C four visits than there are A and C one visits. So let's make a simple validation rule that we'll check for this. So we're gonna say that A and C four should be less than or equal to A and C one, okay? So what are the steps to build this validation rule? And more importantly, what are the steps to actually get this validation rule built into a notification and pushed out to people? Well, there's essentially six steps, one of which is optional though. The first one is we need to make the validation rule. I'm gonna show you how to do that just now. Then we need to assign the validation rule to a validation rule group, just kind of like we assigned data elements or indicators to various groups. We need to of course test the validation rule. Testing is so important. There I've seen several countries that did not adequately test their validation rules and it was a disaster. You need to make sure that you're testing these before you actually start to push them out to everyone. You need to make the validation rule notification. Step four, make the validation rule notification. Step five, which is again optional but highly encouraged is you need to schedule the validation rule group to run automatically. This is done in the scheduler app. We're gonna show you how to do that. The step six, which is not optional, you definitely need to do this is you need to write a protocol and guidelines on how to use the validation rule group. You need to make sure that every step of the process of performing these data quality checks, how to run them, when to run them, who receives what notifications you've defined all of that. So that if you have maybe turnover in the organization, whoever comes after the person who made the validation rules is able to update, modify and follow what's been done previously. And you also need to have control over this process. You don't want to have just dozens of validation rules that no one is using that are completely useless that are not tested. You need to, as system administrators, as DHIS2 managers or HMIS managers, you need to have a fair amount of control and very clear documentation of the validation rules. All right, if you want to start making validation rules, again, I'm gonna go through the slides, but the slides I'm gonna show you are just step-by-step screenshots of the actual process. But if you did wanna go in and make validation rules yourself, this is the link. It's a separate instance than what you've been working in so far, and you need to go. So it's academy.demos.dhis2.org backslash dqapp-config. Okay, so you can copy and paste this link directly into your web browser. We recommend that you use Chrome and you'll be able to start building your validation rules. All right, so what's the first step to making a validation rule? Well, we're gonna go to the maintenance app. And in the maintenance app, we can click on the validation on the top header bar. So you see that's underlined in orange at the top of the screenshot. If we wanna make a new one, we just go down to the add new button, which is that blue button right at the bottom right corner of the screen. Okay, let's go step-by-step here. You see the steps are numbered on the right. All right, so follow with me as I speak through the steps. If you're following along, you're actually building this validation rule as I'm going through it right now, what you need to do is you need to give the validation rule a very clear name, okay? In this example, we're calling it A and C1 greater than A and C4, okay? And then if you are building this validation rule in the demo database, recommend that you put hyphen your name so that you can go in and you can remember which validation rule is yours, okay? Then we, it's not necessary, it's not compulsory, but we recommend that you do give it a short name. And then of course we do recommend that all data is coded or all metadata is coded. So that's the third step here. Not again necessary, but it is strongly recommended that you have some kind of coding procedure or coding methodology for all of your metadata items so that it helps keep your metadata more organized and allows you to certainly system administrators to query the database more quickly and easily to pull up any kinds of issues that you may have. Anyways, not compulsory, but we do recommend short names and codes. Then the description, now the description is very important. This is step two, you need to provide a clear description of what it is, what they're actually looking at. So in this one it says A and C4 should not be greater than A and C1, okay? Then after the description, we can provide some instructions. Again, not compulsory, but highly recommended that you put in some clear instructions. You see in this one we have A and C4 should be greater than A and C1. Please check these values, these two values are incorrect. So imagine the situation where this validation is triggered. Someone has put in that A and C4 is greater than A and C1. What should you tell them to do? Then you need to provide them very clear instructions. And then that's the place to put it here in the instructions. Again, don't make people guess what they need to do about the problems. Tell them specifically. Step four is to assign an importance, okay? And let me tell you a little bit about what this means. So this is completely up to you. We have a high, a medium, and a low option for importance. And the default is medium, but you could use this importance for a couple of different reasons. And I'll kind of tell you two different examples right now. There are some countries and projects that give validation rules a high importance if that validation triggers an alert for something that they know to be wrong. For example, if you have more tested than treated, you know that one of those numbers is wrong. There is no way physically that you can have more, if you're monitoring just those who are treated after receiving tests, there is no way that you can have more treated than tested. Now, I know some of you epidemiologists are out there saying, well, what about clinical malaria, people who aren't tested? But if you just bear with me, what I'm saying is those who are treated after tested, if you have more treated who were tested, you know that's not something that can actually possibly happen. So you give that validation rule a high importance because you know one of those numbers is wrong. And then whoever receives that notification knows that, hey, this is high priority, one of these numbers is wrong, something needs to be corrected. That's one way of doing it. The other way of doing it is setting the high priority validation rules for disease outbreak monitoring. Now, this is a little bit of a different concept to what we've been talking about. You can use validation rules for disease surveillance. For example, you could say the number of Ebola cases should always be equal to zero, right? What happens? As soon as one health facility reports a single case of Ebola, it will trigger a validation alert. People receive SMS messages, just people receive emails, right? And then all of a sudden everybody knows, hey, there was a case of Ebola reported, something we need to go out and do something about it. Several countries in West Africa use validation rules and high importance validation rules, specifically for disease outbreak and monitoring in this way. Then you need to define your period. And your period, step five here, should be set to at least the same as the frequency in which the data is captured. So if your data is captured monthly, then you should set your period to at least monthly. You can't set it to weekly, you can't set it to daily, that would not work. You need to at least set it to monthly. However, you could set it to like quarterly or annually, and what would happen is that this validation rule would aggregate that data from those months to that quarter in which you're doing the analysis, if that makes sense. But the general rule of thumb is that you just set the validation rule to the same period at which you're capturing the data. After you go through those steps, you need to define the left side and the right side. Here, I'm going to just quickly show you, this is the left side generator, okay? And how about this? I think I've stretched that a lot. Let me just go in and I'm just gonna show you this. It's a little bit easier to show it to you than to present it on a slide, the left side and the right side. So I've gone into the maintenance app. I'm gonna go to my validations. I'm going to list my validation rules. And yep, great, it looks like some folks are already in here doing this. I'm gonna click on one validation rule here, which is A and C2 should be less than or equal to A and C1. And I'm going to, so I've defined my period. I've defined my importance. Now I'm gonna go into my left side. I'm gonna zoom in a little bit for you here. That's a little bit bigger for me. I go into my left side. Then what you have, the first thing is you need to find is your value missed, I'm sorry, excuse me, missing value strategy. So you have three options. You can say skip if any value is missing, skip if all values are missing or never skip, all right? This depends upon which one you choose depends upon the different data elements that you've included in your expression. Typically, I would say more often than not, you want to probably have skip if all values are missing or skip if any values are missing. You don't want validation rules to be triggered if there's no values there. It's not usually good practice. If they're just missing values, some DHIS2 will stand in a zero value, okay? So I'm actually gonna change this. I'm gonna say skip if all values are missing, okay? Cause why did I do that? Well, I actually have two different data elements here. So I have A and C second visits fixed plus A and C second visits outreach, okay? So I'm gonna say skip if all values are missing for these two here. Okay, the next thing that I want to point out is one second. This sliding window, what does sliding window mean? Well, the great thing about DHIS2 these days is that we have a lot of user documentation out there. There's a lot of resources out there. So what I'm gonna do just to kind of show you the processes, I'm just gonna Google DHIS2 sliding window. You can see I've Googled this before, right? And so what's my first link? It's a link to the master user documentation. Gonna click on that, see where that takes me. Well, I think I found my answer right here about sliding windows, okay? I'll just give you a quick summary of what this is. Now, this is our good functionality and primarily intended for disease surveillance, okay? So the typical use case for this is if you have a threshold and that threshold for, again, outbreak threshold and that threshold for outbreak is calculated on, I say, a weekly or monthly basis, right? But you have maybe daily data coming into the system. And what the sliding window will do is as you move forward in time, as you go from week to week, and make this validation roll analysis, the days will kind of follow you incrementally as you move forward in time. And that's actually what you see this little diagram that's playing out. You see that we're going from week one to week two to week three, right? And our threshold for week one is 10, our threshold for week two is nine and our threshold for week three is eight, okay? And you can see that the number of cases is, and that threshold is calculated on a weekly basis. But as we move forward in time, we're getting more and more cases reported in. And so you see that our number of cases is changing every day. And that sliding window allows you to go kind of period by period or day by day as new data comes in, right? So instead of the alternative would be that on week two, the validation would only run for, and say you're on Monday of week two, right? First day of week two. The validation would only count the data that's entered from for Monday against the weekly threshold. That doesn't actually really work well in disease surveillance. We want this sliding window, we want the data to be aggregated as we move through time day after day. I appreciate this is a bit of a nuanced thing. Those epidemiologists out there, hopefully you appreciate what we're trying to go for here. Again, sliding windows are principally used for disease surveillance and using validation rules for like outbreak detection or against say like a threshold, an outbreak threshold. If you have more specific questions on that, I'm happy to go into it in more detail. But for now, I think we'll just have to keep moving on. Okay, so the next step is that we actually need to build our expression. So here you see we have A and C first visits fix, A and C second visits fix, A and C second visits outreach. Just for the sake of showing you, I'm gonna delete those. And you see that when I deleted those and I deleted those in this expression builder field here, it says the expression is empty. See the translator down at the bottom is saying, hey, you've got nothing here. So how am I gonna build this? I am gonna come over to the right side. I'm gonna leave, I'm gonna select data elements and I'm gonna search for my A and C two visits. So I'm gonna search A and C two and here they are. And I have fixed and outreach. I'm going to double click on fixed. You see that it comes here. And this thing that you see here, this hashtag and then brackets and a seemingly random series of letters and numbers is the unique identifier of this data element. Okay, this is kind of how DHIs too stores this data element in the backend. All right, so if you're curious on how to query the API for different data metadata items, the easiest way to do it is through the UID of them. Anyway, you still have to deal with it here as you're building the validation rules. So I'm gonna say A and C second visit plus or fixed plus A and C second visit outreach. So here I just click the plus button and I click outreach. And then here you go. We're back to our original expression. So just this plus this. I will go over all of these various other operators. There is clear documentation of what these do. Hopefully most of these are quite clear, like multiplication, division, minus addition. Days counts the number of days in the org unit, in the period that you are running the validation for. So if you have a monthly validation and you are running that validation for October, it would say that there's 31 days in October. So that would substitute a value of the number of days in the period at which you're running the validation for. All right, so I'm gonna save this. After that, you have to select an operator. You see that we have quite a number of different operators equals to not equal to greater than, greater than or equal to less than compulsory pair. Compulsory pair means that both values have to be there. That one value cannot be missing. That the left side has to be there and the right side has to be there. You also have exclusive pairs, which means that one side has to be there. If the left side is there, then the right side cannot be there. Okay, the last, okay, and then we have to define our right side. Defining the right side is just like defining the left side. Then we have to assign, well, you don't actually have to do this, but you can assign the org unit level for which you want the validation rule to run. If you leave this blank, if I don't assign any of these, then what it'll do is it'll run the validation rule at the level at which the data is captured that triggers that validation rule. So this data is captured at facility level. So it would provide the validation at facility level. If I turned on say district level, then what it'll actually do is run the validation at district level, not run it at facility level, and it will aggregate the data to the district. So all the facility level data will be aggregated to the district. In this case, all of the data for A and C2 across all the facilities will be aggregated to the district. All the data for A and C1 across all the facilities will be aggregated to the district. And if the validation doesn't pass at the district, then you'll receive an alert for the district. The last tick that we can say is skip the rule during form validation. Now this allows, by default, the validation rules will produce alerts during data entry in the data set. If you don't want that to happen, then you tick this box, okay? But by default, the box is not ticked. If we go into a data set, into a data entry form, we start entering data and we say that A and C2 is greater than A and C1, then you'll see an alert pop up. If I tick this box, then the alert will not pop up. Okay, let's get back into the PowerPoint. I think we can actually skip ahead a bit. We've covered all of this already. Okay, now you need, the next step is you need to make a validation rule group. And the validation rule groups, making these is very simple. Again, in the maintenance, you go to validation rule group and you just group your validation rules together. So if you have all of your A and C validation rules, you can put those into one group. So the next step is that we need to cover before we're able to break is going through building validation rule notifications, okay? So because we're actually running a little bit of short on time, I'm gonna skip through running the validation rules. I believe that we'll be able to come back to that later in the week. So I'm just gonna show you how to build the validation rule notifications now, okay? So let's again, do this one together. So I'm back in the maintenance app. I'm gonna go to validation rule notification. You see that on the left side of the screen. We see that Robert, Robert, you're winning the prize today. And Wei Wang, you're also doing a great job. So you're already a bit ahead of the pack here. Let's just add one together though. So I'm going to click the plus button in the bottom right corner, like we always do. And I am going to give this a name. I am just gonna call it ANC four less than ANC one. Give it my name, hyphen scott, just so everyone, I can find it later if I want. Obviously, if you're making these in your system, in your actual DHIS two instance, you don't give it, don't put hyphen your name, right? I'm just doing it now because we're all building and sharing inside of one database. All right, and then we need to find our validation rules that we want to use. So I am just gonna randomly six because Robert was the first one I saw this morning. I'm gonna choose Roberts. Hopefully Robert, your validation rule is working properly. We'll find out in a minute. And then we need to define our recipient groups. Now we can send this to different user groups. This is not necessary. You could have it sent to notify users in hierarchy only. That's also an option. So what does that mean? That means that anyone who is assigned to this org unit would receive this notification. But it's also generally best practice that you have different user groups. Maybe you have a user group for disease surveillance or disease response. And any validation rules that are associated with disease surveillance, they should maybe be sent to this user group. Even if the validations are coming from places that that user is not really associated with, okay? So this is an option here. It's completely up to you to decide on how best to use it. But like I said, it's useful to make various user groups based upon different kinds of associations or projects or different roles within the system. I am going to say though, I wanna notify users in the hierarchy. And you can see that you can do both. You can say I wanna send it to this user group and notify users in the hierarchy, for example. And then it's important. The next step is you have to define how you send the notifications. Do you wanna send it as a collective summary or as a single notification? I'll show you an example of this in a minute, but essentially a collective summary will group all of the validation rules into a single message. So for example, if you have 20 validation alerts triggered, then it will send it in a single email. If you say single notifications, then it will send those 20 different validation alerts as separate emails. So instead of you receiving one email with 20, you receive 20 emails with one. Again, it depends on the different strategy and role and purpose of the validation rule. I would say the general rule of thumb though is try to minimize the number of emails that people receive. People receive too many emails. It just becomes noise. They start paying attention to it. Most countries, most projects I've seen have found it more useful to use collective summary. Then we have to define a message. And this message you'll find can be pretty, can be relative to the validation rule. So you see we have our template variables over here on the right side. So we can use these, drop these into our message template and subject template, and it will automatically fill in these for us. So I'm gonna say rule name has been triggered for, let's say organizational unit. So whatever facility on actually for the period. So what month did the validation rule get triggered for? And then I'll say on the current date. So the date at which the validation rule was, the validation rule analysis actually took place. So it'll say, in this case the translation for this would be A&C one minus A&C four, Robert has been triggered for, let's say like facility 146 for October 2020 on the 22nd of October. So that's the subject, short but sweet, but it gives a lot of good information. And for the subject, the message template, we can make this even a little bit longer. We could say something similar, like validation rule name with validation rule description has a left side description of left side value and right side description of right side value. And then maybe we drop in at organizational unit on the period. This is incorrect, please follow the data quality SOPs, something like that, right? So it's just a longer message is giving the user more information about what exactly, what's the validation rule? What is, why is, you know, what was the description of the validation rule? What are the actual values that are triggering the validation rule? Where is it at? When was it? And then giving them maybe some more specific instructions about how to actually correct it. Again, if you remember earlier, we were talking about it's very important to give specific instructions on how to correct the validation rules in the validation rule notifications. So if this happens to someone, they receive this notification, what specifically should they do about it? This one is quite vague. Please follow the data quality SOPs. You could say review during monthly data quality verification meeting or something like that. You can make it specific to whatever actually is your standard operating procedures or your processes in your project or country. All right, so that is making the validation rule notification. I'm just gonna pop back over to my PowerPoint. I'm gonna skip how to run the validation rules. We'll be able to come back to this. I'm gonna skip how to run the validation rules. We'll be able to come back to this. Right, okay. So when to run your validation rules? Now there's a point here I do wanna make this quite clear and I made reference, kind of alluded to this earlier. When you're actually configuring your data set, when you're configuring your data set, there is a tick box. And so this is what this, and that's what you see on the right side of the screen here. This is a screenshot of configuring a data set. The data set is reproductive health. And you see where the red box is. It says complete allowed only if validation passes. You see that that box is ticked. Now what does that actually do? That means that any validation rules that are associated with the data that's being entered for that data set, if there is a validation notification that is triggered, what happens? It means that that validation, that data set will not be allowed to be marked complete, which means it won't be, the data that was entered for that data set would still be in the system, but that means it won't show up in your reporting rate analysis. Data sets have to be marked complete in order to show up for reporting rate analysis. Now this can be a good thing. If you have highly trained staff who know how to correct the data quality issues, and you have very clearly defined validation rules with good descriptions and good instructions that tells them exactly what they need to do. In most cases though, in my experience, this has caused a lot of problems. The reason this caused a lot of problems is because people have not clearly defined their validation rules. They've not clearly defined their validation rule instructions. Their district staff, the people who are putting the data in or facility staff are not highly trained. And it just means that people are trying to put the data in the best they know how, and they're being prevented from being able to do that. There is one country I worked in who had one validation rule that was configured that was not possible of being passed. It was no way to pass the validation rule. And this was the main data set that they had for the country. It was the monthly facility reporting data set, the major data set. And it prevented people from the district and the city levels from being able to mark that data set complete for a couple of years before they noticed the problem. And that means that they had no reporting rate data for years. So they asked what we could do. All we did was go in and untick this box. That was it, that was the easiest fix I think I've ever had for DHI, so to be honest with you. And what did that then do? It allowed them to put the data in and we reinforced the data quality checks post data entry. All right, now I wanna show you quickly before we break in a few minutes on how to schedule validation rules. Okay, so we talked about, you can actually schedule the validation rules and they can send out automatic notifications. So we've talked about how to make the validation rules. We've shown you how to make the validation rule notification. Now we're gonna show you the last part of the job which is scheduling the validation rules to run automatically. And so there's an app for this. The app is called the scheduler app. This app is again, an app that should only be accessible to system administrators. This is not an app that you want anyone to be able to go in and do stuff with. But if you're a system administrator, you should be able to use this application, the scheduler app. And what we're gonna do is we're actually go in and we're gonna make a monitoring job. And so I'm gonna walk through the screenshots here, okay? When you go in to make a new job in the scheduler app, you first have to define, you have to of course give it a name just like we do with everything else. Give it a name. In this case, we're gonna call it A and C. And then you have to schedule when the job will execute, all right? You have three options for this. On the first option, which is that first orange box on the far left, you see that you can use a predefined frequency. So you see that we have five predefined frequencies there every hour, every day at midnight, every day at 3 a.m., every day at noon, every week, okay? So these are your options for predefined. If you want a different frequency, say you want the first day of every month, then you have to put in a chron expression. Now, if you are a server administrator or anyone with a bit of background in software development, you know what a chron expression is. But if you're not familiar with chron expressions, all I can say is you should just Google it, there are a lot of great chron expression builders out there in the world. You don't necessarily have to know anything coding or anything, you just have to be able to say, I want something to run at the first day of the month at 1 a.m. And then there's chron expression builders out there that will kind of give you the syntax for that and you can copy and paste it here into DHIS. The other alternative is to have continuous execution. Continuous execution means that it will constantly run. As soon as it finishes, it will run again. Now, this may be good for a few situations. The first one is disease surveillance. If you are monitoring just one or two validation rules that are used for disease surveillance, you can have this continuously running, right? That's not going to tax your system very much. But if you're using, say you want to make an expression or a job that will check maybe 10 validation rules against a monthly dataset, right? You do not want that to run continuously. The reason is, is because that will be very taxing to your server. It'll be very difficult for your server to maintain that because the validation rules require a lot of processing to run and if you're running it continuously you're going to really slow down the performance of your server. So again, this is useful for disease surveillance if you're just running for one or two validation rules but I highly, highly, highly recommend that you do not use it if you are running maybe three validation rules or more. You'll really tax the performance of your system. All right, the next thing that we need to do is to find a relative and a relative start and a relative end. Now this may sound a little bit confusing to you and essentially what this is is you enter a number here. You enter either a negative number or a positive number and you see here we've entered a negative one for our relative start. What this means is this is relative to today. So and it is counting the number of days. So a negative one here means yesterday. Yesterday was negative one day from today. You see that our relative end is positive one. That is tomorrow. So what does that mean? That means that it will run any of the validation rules that are included in this job. We'll check any data that was entered between yesterday and tomorrow. So that's includes three days, right? But these three days cover these three days fall within October, which is one month. October falls into one quarter, quarter three and these three days follow within one year. So for example, if I have this job looking at just the last three days, then it will also trigger any validation rules that are set to monthly and run those for October as well. If we're sitting in, if we kind of fast forward and say that we're sitting in October 31st today, today is October 31st, let's just imagine that. And we have our relative start to negative one and our relative end to one. That means that tomorrow will be November. And so it would run the validation rules for both October and November, right? Because these three days straddle two months. Hopefully that makes sense to you. I know it can be a little bit confusing. What's a good rule of thumb? Well, I actually kind of recommend that you use a pretty large number for your relative start. So something like negative 90 or negative 60 or something like that. And excuse me, this PowerPoint is not updated. So negative 90 days is not March 24th. Sorry, I'm recycling a PowerPoint presentation here. But you say relative negative 90 days, relative end maybe just plus one, which is tomorrow, you don't have to put in the plus, you can just put in the number one. And negative 90 days to today, well, that covers 91 days. That will cover about four months. I think in this situation, it will cover two quarters in one year. So any data that was entered in the last 91 days in the last four months, the last two quarters or the last year would be included in this validation check. The second to last step is that you have to choose the validation rule group and you have to, or you can add your validation rules one at a time. You can also say if you want to send notifications. So again, notifications, where does this come from? This means that the, that validation rule notification that you configured earlier would be sent out. And notifications can be sent via email or SMS or both. If notification is, if the notification is configured and you configured an email server to your DHIS2, you configured an SMS gateway to your DHIS2, and the user who is receiving that notification is configured to receive emails and SMS. You have to do all four of these in order for anyone to receive any notifications from DHIS2. There is documentation online about configuring email servers and SMS gateways. So we won't get into that today, but if you are struggling with that, please send us a message and we can point you to the documentation and support material for that. Or of course, answer your questions. The last one is persist results. Now, if you tick this persist results, what it will do is it will save any of the validation rules that are triggered. And when this job runs automatically, in this case, it's running every day at midnight. When this runs automatically, any validations that have already been triggered will not be shown. It will only show the new ones. Why is that? It's because it's persisting the results of the old one. Now, if you are a DHIS2 expert and you know how to query the API, these validation rule results are actually saved in a data table. So you can actually start to go to analytics around these if you want. That's just for the folks who are DHIS2 experts and are familiar with the API. If you're not, you just need to remember that the functionality here is saving the validation rule results. And when that validation job runs again, it will only show you the new validation rules that have not been shown in the past. One other consideration is that if you're having a lot of validation rule notifications triggered, these are being saved onto your server. It could take up some server space. All right, I'm going to skip this for now and I'm going to show you some quickly some examples here. I know we're running a little bit over time, but let's take a look at this. So this is an example that I made for our demo database. And you can see that I made the ANC validation notifications run for the whole country. This was just two validation rules. And what did it do? Well, it actually ended up sending me 19 emails. And in those emails contained 2,000 different validation rule alerts. Now, is that useful? Well, certainly receiving 19 automatic emails is not useful. And receiving 2,000 alerts is completely unuseful. There's no way that anyone would be able to respond to this or do anything about it. What do we need to do? We need to make sure that our messages are clear and concise and are in a manageable amount. Now, here's a look at what the actual email said, right? So looking in the email itself, looking in the email itself, it says that this health facility has produced an alert for ANC 2, which is less than or equal to ANC 1 for this month. ANC 2 cannot be greater than ANC 1, right? So the message is useful, but I received 2,000 of them. Now, can I do anything about that? Really, no, you cannot. So we need to ask ourselves, we have to be very careful about this. How and whom can automatic validation emails and alerts be useful? Here is a good example from Rwanda. Now, Rwanda is using DHIS 2 validation notifications for outlier detection. And we're gonna cover how to do that in the next session after our break. But essentially what they've done here, and you can see this is an actual email from the Rwandan demo database. This is not real data, but this is their demo database. You can see that this person received 23 high-priority validation rules in their email. And there is a clear message. So a validation for pin to three outlier, data validation rule has been detected at this health facility on this date for this month or for that year. Please confirm the value is correct for pin to three. And they only received 20 notifications. Now, this person is at district level. These health facilities that are producing the alerts are from their health facilities. This is a amount of notifications that they can do something about. Also the message is quite clear, it's telling them exactly where and when the problem is. So if you're a district health officer, you're able to receive this email and you say, okay, I need to go in and I need to check these 23 values. How long does that take? I don't know, maybe a few hours, maybe it could take a day, but you need to go in and correct these values. These are suspicious values that need to be corrected. This is a great example from what Rwanda has set up. Here is another one from Cameroon. Cameroon is still experimenting with this, but they have set up some basic alerts for disease surveillance. And so in this case, you see that I won't translate it for you. Those of you who are French, I think there's a bunch of Cameroonians on the call as well. This is just a test. This is not, I don't believe that this is in production yet, but you have at least tested using validation rules for disease surveillance. So you are able to send alerts and notifications to those who are on disease surveillance response teams when there is some kind of outbreak or number of cases that needs to be investigated. So another really good example of how this can be used from Cameroon. And again, the person's only receiving four notifications. That's an actionable number. That's not 2,000 notifications. So the last point to be made is that, again, these notifications and everything need to be built into your standard operating procedures and you need to test them thoroughly. So that is the end of.