 Okay, and at this point I think we will get started. Thank you everybody for joining us to view the adverse events following immunization module. I'm Mike Frost from the University of Oslo. I'm the product manager for Tracker, the individual longitudinal component of DHS-2. There's been a whole team behind this work I'm presenting and we'll have some of the colleagues that worked on it. This has been a work in progress for over a year working with WHO, with Gaby and getting input from any number of countries and worldwide global organizations. I wanted to give our colleague from WHO, Madhav, a moment to provide some context and information about this package and what the goals were. Madhav, over to you for a moment. Yeah, thanks Mike. Hello everyone, good morning, good afternoon, good evening to our colleagues, friends worldwide. First of all, I wish to thank all of you for taking this time and coming and learning more about this package of, developed by the University of Oslo on EFI reporting as a part of the immunization tracker. Now, just to give you a quick background, I will not take up much time. I just wanted to inform you that the work with reporting adverse events following immunization from WHO, we have been under a lot of pressure, I should say, from a lot of countries asking to find digital solutions for this. So we started working with, actually the first work was done with the International Vaccine Institute in Korea where we had originally prototyped and developed a very primitive Fox Pro based version of reporting adverse events. And when we saw it successful because we wanted to get case-based reports on it. And that's the reason when we saw that it was successful, we came back to the drawing board and then we also collaborated with Gavi. And at that particular point in time, Gavi expressed interest in working with us together with the University of Oslo and coming up with this package on DHIS2. Now, the great advantage of this particular package, we've already piloted, by the way, just to fire information. We had, when we had originally conceptualized through this work, when WHO had done it, we had originally done a work where we had collected case-based data with a particular package called Harmonia, which is still available online for conducting data management trainings. So this will also work. And this particular package, which was developed by University of Oslo has been developed based on that particular package. I mean, it was been a very great, I mean, learning experience for all of us. There has been severe good contributions, particularly from the field, I should say, and colleagues from Afro particularly have contributed quite significantly to the development of this package. I could point out two people. We have done quite a bit of work. One is Sujit Jain and the other one is Edinam, who are also on the call. Now, that said, I want to also appreciate the University of Oslo, as well as the HIST India colleagues who have been really the pillar behind the work of this particular work. They have really made a very, they have put in a lot of effort in making this particular package. And they have tried their best to customize it from the field level perspective. So I won't be taking up much time. I'm handing it to Mike and let's hope it's going to be a very useful call. Thank you. Great. Thank you so much, Mod Ops. So I did want to start giving a little bit of context of what this package falls under. The University of Oslo, the DHS2 team is a collaborating center with the World Health Organization. And for kind of four or five years at this point, we have had a role in creating specific configurations of DHS2 that are meant to already adhere to global recommendations, standard guidelines, WHO recommendations. So there are quite a number of these packages that have now been published. They're available both through the DHS2 website and through the WHO website. We will be sharing these slides after the presentation and you will find links there to follow, to see some of the other packages as well and more information on this package. The idea being of course that DHS2, anybody can go and download it now from GitHub or from our website, but it comes as an empty system that must be configured to adhere to your use case. But with these packages, the idea is that it's preconfigured, that it already covers the kind of data that you would like to be collecting, that it matches a workflow that makes sense at least for a majority of countries, that it provides the kind of analytics and indicators that you would want. So we've been working with this for quite some time. I'm just showing a screenshot here of our current map of where these many different WHO packages have been adopted. Some 40 countries worldwide that are already using one or another package and have integrated this into their standard HMIS, into their individual reporting systems. So you can see many of the countries that you on the call are representing probably already have one of the WHO packages at least. There's a larger map showing the DHS2 adoption, but quickly the countries that have DHS2 as they proceed into new areas, they are likely to start from these WHO packages. So the adverse events following immunization is one of the many that we have been working on and we have this long-standing support working with other groups. WHO is the leading organization, but Global Fund is a part of this initiative. Gavi is UNICEF. Many of the global organizations feed into this process for defining the requirements for these packages. I know this one is a little bit difficult to see on your screen. There will be a link again that you can follow. What I really wanted to show is that this package is also just one of the many immunization packages that we have worked on. So there's quite a bit available if what you're looking for is a number of different ways of approaching immunization programs in your country, everything from a basic EPI dashboard in analytics to bottleneck analysis, scorecards, and then of course the one that I have highlighted here, the adverse events following immunization. During the demo, I will show a little bit about how these packages can be integrated. That's a guiding principle for us as we design the packages that they can live together in the same instance, the same system and provide all of the data that are required at whatever level that the package is designed for, but while also feeding into national program indicators to global reporting and any number of things. So then specifically on the adverse events following immunization, this has been a part of the work from WHO's Global Vaccine Safety Initiative, meaning that we took the materials and requirements that WHO had already developed and tried to translate those into this DHS2 configuration, paying special attention to the 25 core variables that are recommended. One of the reasons that right now I assume that many of you are on is because of course COVID-19 is about to be releasing vaccines worldwide. And of course these are newer vaccines, it's doubly important to be able to collect information about any adverse events that occur. So just a word to say on that, this package was designed originally over the last year plus, focusing on standard routine immunizations, particularly those for the age zero to 18 months. But the process for reporting adverse events is essentially the same across the vaccine. Meaning we are a part of many ongoing conversations about the requirements for the COVID-19 immunizations that will be coming out, not just for adverse events, but for also registries for aggregate reporting for a number of things. And we will be adding any new requirements to this package and the others that we have around COVID-19 as specific requirements are formalized. But at the moment, we believe that the work covered in this package actually is very applicable to COVID-19. It's a very firm starting point for collecting any adverse events associated with the COVID-19 vaccines. Just showing you a little bit the 25 core variables where they come from on the WHO website, you can of course follow these links as well and get more information about the requirements that were used to create the package. And then for those of you that already work in adverse events following immunization, hopefully you recognize this form on the left. This is the standardized WHO form for reporting and collecting the information that's associated with the adverse event. We essentially have taken this data collection form and recreated it in DHS2 using a standardized custom form that collects all of the necessary data. So it was almost a one-to-one matching of the data elements from this form that are used in DHS2 inside of Tracker and that then feed into the recommended indicators and analytics associated with adverse event program monitoring and follow-up. We in the last six weeks have been working with a group that is responsible for Vigibase. Vigibase is the global repository for reporting adverse events following immunization. The group behind that is also a collaborating center with the World Health Organization. Many, many countries are already reporting into that global repository. There's a piece of software called Vigiflow that they use at the country level to report the necessary data elements to Vigibase. We have been working with that team to integrate reporting for Vigibase so that the data can be taken from DHS2 and directly transposed and sent to the global repository. So we have at this point a working integration. It's something we're still refining but it's available already and can be a part of the package from day one. Again, in the COVID-19 times, this seems especially important for countries to be able to contribute any adverse event that they find with the COVID vaccines to the global repository where they will be, oversight globally about how those vaccines are performing. Here, we're showing you the cycle behind the AEFI surveillance. This is again, what led to the design that we have in DHS2 for this. I would pay particular attention to the starting at the AEFI detection, notification, reporting, investigation, analysis, kind of going around the cycle. What we don't fully cover in the package is the causality assessment. This piece is a more extended and complex piece that happens at the country level but all of the previous cycle is meant to be modeled in the DHS2 package as well as information that is used for feedback and for the analytics. And I'll go into kind of these different stages specifically about how the design works and then we'll have a demo from Kitika that will show us a bit more of what the package looks like live. So just showing you the different intended users for this package. The package is essentially a small three-stage kind of program, the first focusing on the staff that are responsible for entering the data into that WHO form that I showed previously, minimal kind of data that is being collected to start the investigation and provides the full background to where the person was immunized, what vaccine that they received, what are the symptoms that they are experiencing. This is the information that is then used at the first decision making level to decide whether a further causality analysis and investigation would be conducted. And then finally at the national level to review the data that has come in, perhaps assist in the causality assessment and provide a more in-depth review of the adverse event. So these are the three kind of intended user groups and the three stages within the program as well. So just to show that a little bit, I will model it here for you. One word again on the topic of integration, we have a immunization registry package which is separate from this adverse event. It's a standalone module that is meant to be used at the point of care, particularly for those vaccines covering the period zero to 18 months. This group of users may be some of the first to identify that an adverse event is occurring. And because of that, then there is in that immunization registry a simple data form that they can provide some information to that could be used for a notification to actually switch over to the adverse events program. So this again could be a clinical person filling this out. They simply put in the very basic information associated with the adverse event that they are observing and this can send a notification to the district level or the higher level that is responsible for actually performing the adverse event investigation and the first stage of filling out more complete information. Just a word on how the configuration happens in these packages, these are meant to be adapted to your context. So if the clinical person does not have the authority to be doing this kind of a notification, then of course that stage can be removed. This notification can, should be sent to the appropriate person within the workflow in your country. It can be sent either as an email or as an SMS or it can be sent even through the DHS2 notification system if they are a user. And it would be populated by these, you can see the get name, get unique ID. These would be pulled from the data itself. Again, these are features that are available in the system but if your country doesn't want to use those, that's completely configurable, it can be removed. So then we have the actual adverse event program itself. Again, following from whoever in the country is responsible for first completing this next form here. So it might be at the district level, it might be at a lower level, it depends very much on the work processes for your country but whoever is responsible for filling out this form, this is the first step. This form then captures the information on who is doing the reporting, what is their designation and department, where they are based, et cetera. And then again, these fields should be familiar for those of you that know the WHO form, going through the vaccines that were received just prior to the adverse event, capturing all of the information associated with it such as the date, time, batch lot number, X-pre, alluding, all of these different pieces of information that help inform the investigation going forward, including the information about what is the adverse event, what are the symptoms that they're experiencing, is it a serious adverse event, what was the outcome? So all of this information can be collected in this form. This form at times and in certain countries is filled out by more than one person, which is fine. So in the system, it can be completed by multiple people and the information is then used to start the investigation or not, depending on what the determination is by the people responsible. So this form is completed and again, a notification can go out that confirms that the stage has been completed and can support whatever the next step is in your system. So this notification can be tailored. Right now it's the very generic, please take the next steps to get the form registered by the authorized personnel, but that wording is very easy to change to match whatever the workflow is in your country. We then have the first decision-making level modeled in a very simple stage. Again, by maybe the state province EPI officer, the state province AEFI committee, they simply are reviewing the information that was entered in that first form and making the determination about whether an investigated is needed and when that investigation may be planned. So again, a very simple form only contains a couple of data elements, but it's meant to follow the authorization process for AEFI. And again, can send out notifications that says that the investigation has been requested from this first notification level and gives the information about what are the next steps to be authorized and complete and perhaps continue with the causality analysis. Again, the national level, once that first decision has been made, there may be a national EPI manager or the AEFI committee that are the ones that are reviewing this request coming from the first decision level. And they again make a simple response here about the date this report was received at the national level and the unique ID associated with it. This data is very useful in terms of monitoring the program for AEFI and determining the time between reporting and the time that an investigation occurs or that the causality analysis is completed. But again, a very simple stage with the data that is necessary to support the authorization and work process. And again, this can also have a notification associated with it telling them to please take the next steps to upload the case data to the global database. So back to the Vigibase integration, there's probably a process in your country about global reporting. We did a look at the countries that are using this Vigibase and also DHS2. There were some 30 countries that had an overlap between both systems. So if they're adopting this AEFI module, then this notification would help prompt them to take the steps to actually report to the global repository in Vigibase. Just taking a quick look at some of the analytics and we'll look at them more as a demo, but this can, we've set up some standardized dashboards with the information that would be necessary to follow up on the AEFI investigation. One of those here is an AEFI line listing so that as each adverse event is, this first stage is completed, the data can be sent to a dashboard for the people that need to follow up. They can see each case and the accompanying data very quickly and be able to then perform the next steps. They can also follow directly from there to the system and see the full completed form for the AEFI. And again, we'll take some more look at this just to let you know that we have set up specific analytics associated with the data that are collected that are corresponding also to the recommended WHO indicators for AEFI program monitoring. And then maybe the last thing that I will say here before we take a look at the demo itself, DHIS2 has a native Android application that is a part of our ecosystem. The DHIS2 capture app, the AEFI form is meant to work seamlessly whether they are using the browser for data entry or review or whether they're looking at Android. So you can see on the left in this image, the user happens to have a number of different programs that they are able to report into or for. They can simply click on the adverse events following immunization link. They can do a search in the second image and identify the case or register a new case. And opening up that case, they would be able to go through and see the data that have been entered or to indeed enter data itself. So again, this is the solution for those levels of the system that do not have consistent network connectivity or that don't have available computers where they would be doing entry through the web browser. Android is our offline and kind of intermittent connectivity solution for those scenarios because the data are collected on the device, but the device does not need to be connected to the internet until it is time to sync with the database. So if they're having a health facility or a district level facility that aren't connected to the network with any reliability, Android is the solution for them. They can collect data there directly on the device and when they can get a connection, then they're able to sync that data back to the national server. So with that, I think we will turn over to Gitga. I will stop sharing my screen and she can share a screen to give us a live demo. And again, we please do add your questions while we're working on this to the community of practice link that has been shared. If you have any trouble with the community of practice link, of course, you can also ask a question in the chat. After the demo, then we'll move to questions. We'll try to answer those questions that are written as they come through on the community of practice and in the chat. And if there's time, we may even open up mics for people that want to ask a question out loud, but we'll try first from the written questions. So with that, I will stop talking for a moment and let the Gitga walk us through a live demo. Hello, everyone. This is Gitga and I'm working with the team on configuration of the AEFI package in DHIS2. I hope all of you can see my screen, which is the DHIS2 application where we have configured the AEFI program, part of which you had seen in the slides that Mike presented. So each of the user will be given a login to access the application. And once they log in into the application, they will be able to see the dashboards based on their user access. So if a particular facility user logs in, they will be able to see dashboards based on their specific facility only. If a national level user logs in, you will be able to see the data for the entire country. I'll start with presenting how we enter the data for the AEFI format in the application. And then I'll come back to the dashboards to show you the various visualizations which are available. So to enter data into the AEFI program, there's an app called Tracker Capture. Gitga, just there's a request from the chats. If you're able to speak louder, I think some people are probably on smaller kind of speakers. And yes, if you're able to speak louder, that would be great. Sure. So there's an app called Tracker Capture. To enter the data, I'll click on this app, Tracker Capture. And on my left side, I will be able to see the facility or the district or the province or the country, whatever is assigned to my user. So right now, I've logged in, say, as a national level user, and I can see all the hierarchy here on the left side. For a facility user, say a facility user with Crow Health Center, they will be able to see only the Crow Health Center here. If a district user is entering data for all the facilities under his district, he will have access to district and he will be able to see all the facilities under that district. So now let's take a use case where I have to report an adverse event for a child who was given vaccine at Crow Health Center and who came back with adverse events. So I will select the health center or the facility or the clinic where the vaccine was given to the child, which is the Crow Health Center. After selecting the facility, I will be able to see the list of already registered children. So here is a list. You see there is a list of programs available here. And in the adverse events following immunization, this list shows the list of children who are already diagnosed or already registered with an adverse event. If the case, if my facility was also reporting the vaccines, then I can search for the child if he's already available in the system. Otherwise, I can click on register to add the child in the system. So I'll click on register to add this child and this is my registration form. So as you see the enrolling organization unit, Crow Health Center, this is automatically made available here. Then I need to select the date when the patient notified the event to health system. So by default, it takes the two days date. So considering that I'm doing the entry when the child with the parent visited my clinic, otherwise, if I'm doing a back date entry, then I can select say 7th of December. Was the date when the patient notified the event to the health system. Then are the details on the demographics. So this unique system identifier, this is auto-generated for each case. If the facility is also reporting vaccines given to the child, then the same ID can be used for the vaccination, the immunization program as well as the AFI program. Then we have another option for AFI case ID, which is a text field for the countries who are doing entries from the papers and they have an ID already mentioned on their paper formats. They can enter that case ID here. So say my case ID is case ID one, two, three, four, five. Then the given name and family name. So let me just put some name here. AFI case. Then date of birth of the child. So say I'll put in some back date as 1st of January, sex, male, address, then the contact number of the mother or the caregiver. And so we can logics based on the country as to the, I have drop downs for the address field. So we can have the logics for the number of digits in the contact number. Now I save and continue. This is the registration details of the demographics for the child. And based on which this new page which opens is the dashboard for that particular child. So here on this page, I'll be able to see this AFI event already created. Now, if I scroll down, you see that there is a date called report compilation date. So this is the date when I've compiled the entire details and now I'm entering the data into the system. So if I'm doing a back date entry, I could either select a back date or today's date. Just say I'm selecting 8th of December here. And then this is the form for the adverse event details. We have a section on the reporter details, then the vaccine details, then the details of the signs and symptoms for the adverse event, and then the outcome for the event. So I will go into each of these sections one by one. Let me just zoom in a little, okay. So the reporter's name here, this section is for the details where the adverse event was reported. So mostly as you understand, the child is brought with the symptoms to the same facility where the vaccine was given. So the same details can be filled in here, or if the child is taken with symptoms to another facility. So that facility where the child was reported with symptoms, that details need to be reported here. So the reporter's name, designation and department. So say, something, organization head. And the reporting health facility address. So here we have the same list as the hierarchy. So from my hierarchy, I need to select the health facility where the child reporter with symptoms. So if I select same as vaccine, if I am considering my use case, and then I'll select pro health center here. Select. And then are the contact number and email for the reporter. Then after the section, I need to enter the details for the vaccines. So usually we see that for the initial vaccines, given to the children, usually three vaccines are given together during one visit. So we need to enter the details for all those three vaccines when an adverse event gets reported. So the system allows you to give details for a maximum of four vaccines, which can be given to a child at one point of time. And once I start entering the details, I can keep adding more. So here, if I select for the first vaccine, I'll have a list of vaccines available here. As you see, BCG, diphtheria, hepatitis. So all of the vaccines are here in the list. I could either scroll or I could search from the list. Say the child was given diphtheria. Then I can enter the details for the brand name, the date of vaccination. If I say the child reported to me on seven, so maybe he was vaccinated on fourth. Time of vaccination, something in the morning, 10 o'clock. Then which dose of the vaccine was given to the child. So if I say first dose, then the batch. So because of the reporting of the adverse event, it is important to have the batch details, the expiry date details for the vaccines. So as you see, these fields are mandatory as they are marked with ashtray. And then I'll give the details on the batch number. Expiry date for the vaccine, which allows you to enter future dates, say something in 21. Then it is also important to have the details of the diluent given for the vaccine. So for the vaccines where we are giving diluents, those details can be added here. What was the name of the diluent, then the batch number of the diluent, then the expiry, date of reconstitution, time of reconstitutions. We have few validations built in into the system. For example, you cannot have a date of reconstitution after you are giving the vaccine to the child. So this date must be on the same as or before the date of vaccination. So right now if I say it's the same as 4th of December. And similarly, if the child was given another vaccine during the same visit, those details can be entered here and so on till four max vaccines. After this, we have a section on the adverse event wherein we need to report the signs and symptoms. So if I say there was a severe local reaction, I need to give further details on extending beyond the nearest joint if it's more than three days. Then if I say scissors, then are they fibril or if it's fibril, then I can have sepsis, abscess, and cephalopathy. So this is the same reporting format which we had seen right now in the presentation. We have some other minor symptoms available here in this table wherein I can select the ones which are very common with the vaccine. And then we have to give details on the date when the AFI started. So maybe the child reported with symptoms on seven but the AFI had started on six. So I can add that date here and time may be in the night. And if the person wants to give some details about the AFI, the sign symptoms, our text box is made available here for adding those details. After this details, we have details on whether this was a serious event. If yes, then death, life threatening, disability, hospitalization required, or congenital anomaly. If I say death, then I need to give the details on the date of death. And I can also say life threatening or death. And then we have a dropdown for the outcomes. So say recovering, recovered, recovered, resolved with sequel, not recovered, unknown, died, autopsy done. And then we have option to enter the past medical history to know if there was some known history for the patient which might have led to this adverse event for the child. So then we can add those details here. And once we complete all of these sections, then we click on complete. Now this form allows you to do entries, entry into the form based on as and when the data is made available. So at the first moment, maybe all of this information such as outcome or severity might not be available with the person who's entering the data. Some of the symptoms might come in a little later in next day and so on. So accordingly, we could reopen the form and keep adding the details as and when they are made available. And once we have completed everything, that is the time when we click on this complete button at the bottom. Once I'll click on this complete button here. So this sends a notification to the next level, next decision-making level, informing that an adverse event has been reported at Crow Health Centre. And please look into this and verify the record. So this is important because it prevents the delay in notifying the adverse event from the facility level to a higher level. So if I again go back to the dashboard. And one is we can send the messages on the mobile phones of the person for each type of facility. And then we can send SMS messages on the DHIs to itself. So here on this message on the dashboard. I can see that a minute ago, a message was sent that AFI stage has been completed for case ID 12345 at Crow Health Centre. And please take the next steps to get the form verified. So then the next person would know that, okay, now I need to look at an adverse event has been reported and I'll check and review the form. Coming back to the form. So what the next decision-making level could do is to go to the tracker capture and search for this patient here is ID 12345. They'll be able to see this record here and they will be able to see all the information that was filled in for this AFI event for the child. And what they need to do is click on this plus icon here. Once I click here, I see this stage first decision-making level. So maybe I'm just taking decision on the same day today. And once I open this, I have a very small form just to say that based on the details that you see here, do you think an investigation is needed or do you think it's fine and no investigation is needed? Now, again, when I say yes, investigation is needed and when I can plan for investigation here, another message will be sent to the users saying that an investigation is needed for this adverse event. If I say no and I complete this, in that case, the message will be sent to the national level saying that the first decision-making level has approved the adverse event which was reported at Crow Hill Center. Please take the next steps to approve the data at the national level. So the national level person would again come to this tracker capture search for this case. And now once they click on this plus icon, they will see this national level here. I click on this national level, same date and save. And now what I need to enter is what was the date when the report or the information was received at the national level? So say it was received today. And then what is the AFI Worldwide Unique ID for this case? And again, zoom in a little. So we have these options in the dropdown which can be selected based on the adverse event. And if I want to enter some comments, I can add those here and then complete. Once the national level completes the event, a message is again sent to all the different users in the hierarchy saying that the adverse event has been reviewed at the national level. So all of these messages will be made available here on the dashboard. So we have these here. Maybe I'll stop here if there are any major questions or else I'll continue with the dashboards. So Mike, do you think there are any? Yeah, so there have been quite a few questions coming in. We're trying to answer them in the chat and in the community of practice. If they are in the chat, we actually try to take them over to the community of practice. So that they will exist after this call and people can have their own questions answered. Just so you know, if you can enter them in the community of practice, please do so. There's a lot that maybe we'll take after the demo. One thing that has come up in a number of questions is around data validation and logic in the forms, preventing, for example, the registration of the adverse event from being before the birth date. So in DHS2, there is a tool called Program Rules where you can easily configure any number of these data validations. The package as it exists right now has kind of a minimal set of those validations which can be added to or modified based on your country requirements. The decision to keep them somewhat minimal for this generic package is two-fold. One is because the program rules themselves can end up being mini and somewhat complex. And so we wanna prevent adding too many program rules where they will become something difficult for the country to understand or modify later. So we do try to keep them to the bare minimum in these generic packages. That said, we also think that a lot of the data entry that occurs in this particular package will be coming as secondary data entry where that level that was first collecting the data may still be doing that on paper. And then they're giving that paper to a higher level or to a data clerk to do the data entry. And in that instance, we don't wanna prevent the data entry from entering whatever information is on the paper. That's not necessarily the point where they even have the authority to change data or to do something different. There should be later data validation checks where during the investigation there is follow-up. But this is still under review. So we actually had a meeting just before this where we were talking about adding more data validation requirements. We're very happy to hear more from users and from all of you about which data validations are the most important to include depending on your use case. But again, these also are modifiable by you. So when you get the package in the country, we would recommend that you do a requirements process where you look at what your needs are and then determine what changes may need to be made to the configuration to match your use case and your user flow. So we can talk about that a bit more after the rest of the demo as well if there's more questions on the topic, but I want to bunch those together now because it's come up quite a bit. Just to confirm, if you could write in the community of practice, I can't answer or I can't put any more questions in under my username, so it needs to be under your personal user. Yeah, so your questions will be better met in the community of practice. That's where we're trying to keep the record of this. And I can post the link one more time here in the chat. But then otherwise, I think we continue with the demo and then we open up for questions again after. Okay, thanks Mike and Cam. So coming back to the DHI stupidity page, now I am on the dashboard. So we have configured two dashboards here for AFI, as you see, one is AEFI, adverse event following immunization, which focuses on the adverse events, reported type of events, type of vaccines leading to events, type of signs and symptoms, age and gender-wise categorization. While the AFI surveillance focuses more on the time taken for reporting of an event from one level to another level from initiation of signs to reporting at the health facility. So I'll come to these details. On the AFI, as you see, we have, I'll just scroll through quickly on the different visualizations and we will be giving you demo access to the demo page. So you could also see those yourself later. As you see, these are the graphs which show that monthly incidents of AFI cases, then age-wise categorization of the AFI cases, then based on the signs and symptoms, how many encephalopathies, how many abscesses, how many sepsis, then region-wise cases, then we have vaccine-wise, gender-wise, based on the events, how many severe events, how many required investigations, how many did not require, based on the outcomes. And then in relation to the vaccine and the type of signs and symptoms from that vaccine. So as you see on the dashboards, we have options to add more filters. Right now, I'm able to see the data for this year and for the national level. If I'm to see data for a particular year. So we have some more data for 2016. So if I say yearly 2016 here and I select 2016, then I can see the data for the 2016. And as you see the dashboards show much more data now for this year. Now moving on to the AFI surveillance dashboards, here we have the graphs which are comparing the data for time interval. So the first one says time interval between the date of reporting at the first level at the health facility and the date of notification. So when was the event notified and when was it reported? The second one is time interval between when the date, between the onset of symptoms and when was it notified at the health facility. Then we have on the time interval between date of notification and date when it was reported at the national level and the time interval between the vaccination date to when the event was notified. So we have these four graphs available here for the surveillance of the reportant. I think that's all for the demonstration and maybe I'll hand over to you Mike now for the questions. I'll just stop sharing my screen. Great, thank you, good to go. So I'm sharing my screen again. Many people have asked about where they can have access to play around with the system. Again, we'll share these slides so you can have this link directly but we have a demo site where you can log in and play around with the system. It's also possible if your country is really moving forward to help set up a sandbox where you can do some more kind of playing around and changing things in the system. Of course, once it gets to the point where you're actually ready to start configuring your system then you would want to do that in your own development environment. So that was another question about how to get access to the app. This will be published through both our website and the WHO site where you can download the full JSON files, the configuration files to set up within your DHS-2 instance. Again, many of the countries that you're representing here probably already have a DHS-2 instance and even a team to work with. So you can work with them to establish a development environment where you can work with the package and make any changes that you'd like to make. We also have here a number of links. Oh, well, let me come back to this. There's a number of materials and resources available for you whether through our DHS-2 standardized documentation through a dedicated website that we have on our site page for immunization which will give you more information about the various packages and for the WHO packages in general and you would be able to download the information associated with the package. The packages also have not just the metadata and configuration, they also have a toolkit associated with them with user guide and documentation about the package and what is included. So that information is there to help guide you as you make decisions about what to change. And you'll have a number of these links again directly in the slides which we'll share after the webinar. I did wanna come back to this one just to say that we work closely with GAVI with the Global Fund in terms of supporting countries with their request for funding. We know that the GAVI TCA funding request are ongoing now. So if your country is interested in moving forward quickly with this, this is something to get to the teams responsible in country for making those requests and making sure that it can happen in the next round of funding. There will also be an implementation user training for this package which would help you understand a bit more how to manage the data associated with it. This will be coordinated as a training with WHO. ModHob maybe you can even say a word more about this towards the end of the webinar, but this is planned to be coming in the next couple of months of 2021. So with that, perhaps I will go over and start to share answers to some of the questions. We've tried to answer a number of them. I may repeat some of them verbally here for those that haven't been following the chat or the community of practice. So we did, again, there were a lot of questions about the data validation that is included. So I went ahead and shared Rebecca had pulled from the system itself which validations we had already preconfigured into the program that's found both in the chat and in the community of practice, but it does include some of the very basic ones that were brought up about dates of birth, dates of death and making sure that those are aligned and providing a warning and an error if they are not. Yeah, so again, those very basic kind of validations are included. Much more can be added depending on what your use case is, who is entering the data and what level of validation they need to see. Just to give you a sense of that, this program, again, we included very minimal data validation. The immunization registry, which I referred to earlier has much more. We have 300 or more kind of validations that occur through that one because it is meant to guide clinical decision-making. It allows the user to get information from the system for a specific patient about what should be their next vaccine based on their age, risk factors, et cetera. And so again, the DHS2 can support much, much more data validation if that's something that really is of interest for your country and can be configured again, just through the application for maintenance in DHS2. You don't need to be a software developer to create those program rules. It's something that can be done by somebody that is a DHS2 configuration lead. Scrolling back through again, there was something, a question about the Android connectivity and how that works. So Android is, we have basically the two modes of data entry and access to DHS2. One is directly through the browser where they need just the website, the URL, and they sign in as a user and they're able to enter data. If they're using the browser, the data are being sent directly live to the database. And so you need a strong network connection or at least not one that comes in and out frequently. So that's for those users that have a decent internet connection and have a laptop or a computer that they're doing data entry with. For users that do not have that, then the recommendation is to use Android. Android, the application lives on the device. The data are stored locally and only sent to the database when there is a connection available. And so no data are lost, they're stored locally and they can be synced at any point in time. Of course, we don't recommend being too aggressive about providing Android devices out to places where they simply don't have network coverage. At some level in many countries, there really should not try to be doing a network solution until network is more available. But as far as those locations where they actually do have network connection that they can get to once a week or something when they go in for their district interviews or supervision, those kinds of things, then the Android device is fine to be using where you know that every several days you would be able to do a connection to the database. Scrolling back through, there is a question about the Ethiopian calendar. There are several calendars of course that are not the Gregorian calendar which are unused in countries where they're using DHS2. This is a long standing challenge with DHS2 and as it turns out for many operating systems. We have made branches of DHS2 where they fork off from the typical DHS2 and are able to support any number of calendars. Ethiopia does use DHS2 and has an operating solution right now for this but it does live on a branch of DHS2 meaning that when there are updates that are provided annually to DHS2, then there's more of a manual process to make sure that your branch works with that update and your country, the team that is running the DHS2 instance needs to verify that there are compatibility is 100% before they do a switch over to that. I wish I could say that there was a more integrated solution right now for the various calendars but we are working on it. We're trying to decide what is possible to do there and it's something that we'll have more definitive answers to in the coming year. It was a part of our roadmap planning process for 2021 that we just conducted about six weeks ago. So we'll have more to say on that but yes, right now it can be done through a branch. Looking through about more questions that are available. There were questions about Vigibase and when and how it connects. So again, we built a tool for exchanging data with Vigibase that is a process for the country to determine whether they want to do that or not. It's not automatic. So using DHS2, the data are stored in your national instance. This is not hosted by WHO. It's not hosted by the University of Oslo. It's hosted by the country. You can make the determination in country where that is hosted. It could be at a server that the Ministry of Health has that satisfies the requirements for this kind of a hosting need. It can also be set up as a cloud instance that is run through the cloud but owned by the government. But in either scenario, the data are first in that national instance. They do not travel or are shared with anyone else until the country has decided to do so. If they want to report to the global repository which of course is recommended by WHO, then they can establish an automated routine for exchanging that data based on your own internal timelines. So for example, if there's a process for data review, data quality checking before submitting to Vigibase that can be automated in a scripted way to be able to exchange the data with Vigibase after it has gone through whatever processes are in country. So none of your data would be shared without the authorization of the responsible parties in the country. Sorry, just scrolling through now to see if there are any other questions that I should share the answers to. So a question that's just come through on what's the experience of integrating DHS-2 in complex information systems? So integration and interoperability are complex topics. This is something that if you have one or more systems to interoperate within country, for example, you have hospital information systems that you want to be able to exchange this data with or if you have a separate data warehouse from the HMIS that data need to go to, then that is work to be done by competent technical experts to do the exchange. DHS-2 is set up to be as open as possible with data exchange. So in order to support that, we have a very well documented and functioning API, which is the one way to do kind of a manual mapping of the data to be exchanged. We also support a number of the standards that are associated with data exchange. For example, with Vigibase, there's the E2B standard for reporting safety and critical information. That's the standard that we're using to report to Vigibase. We also work with HL7 fire, particularly happy fire for data exchange. But again, those standards are really only useful if the system that you're trying to integrate with also uses that standard. So there's no, I think, kind of plug and play interoperability with any system. It's an assessment that must be done about which systems you want to integrate with, what standards they use, who is the responsible person that can do the exchange, can you automate the process? So I bring this up in detail because it is something that should feature in any funding requests that you have for Yavi or Global Fund or any of the others that may support these DHS-2 activities. There's, it really is a piece of work to be done. So somebody's going to have to do that work and it should come as a funded part of any integration requests that you have. I think there's a good question from Sharjeet. The Gavi TCA funding might not be available until June, 2021. Can funding support be made available through TA support for COVID facility? And then adding on to his question, please elaborate more on the next steps that offer an operational processes for getting it working in the country. So on the first question, yes. In fact, the funding for this does not have to be tied to any particular funding approach. So the TCA is one way of getting this going. Your country may already have COVID funding that has been allocated to it to support the immunization process through any number of mechanisms. Any funding that you have available, of course can go towards adopting this system or this module. And we think that it's a very wise decision for countries to move as quickly as possible in setting up their information system to support the COVID vaccines. There's going to be a massive push in 2021 for being able to roll out the vaccines. And there's going to be a lot of competition as well in terms of which countries received the vaccines first and how those vaccines are managed. It will definitely be of support to the country if they can say that they have an information system ready to go. It's something that many, many donors I think are willing to fund. Okay, also from Shuji. The operationalizing. Oh yeah, I was going to say the older version of DHS2 need to be upgraded to incorporate a fight tracker tool. So the first one about how to actually do this in your country. So again, from, it depends a bit on the state of your country with regards to DHS2. So there are some 90 countries worldwide that are using DHS2. Some 70 of those have it as a national HMIS, meaning that those countries already have a DHS2 support approach, whether it's the IT group in the ministry or some supporting partner, they already have an environment for DHS2 where the new modules could be installed. So that's the first step is to identify whether your country already has DHS2 and who is responsible for it and work with them on adopting the package and getting it integrated into that environment. But no country I think would just take this and plug it in and start using it. The first step should be always to hold a stakeholder meeting with the people responsible for the adverse events following immunizations, review the contents of the program, make sure that it covers all of the data that you want and produces the types of analytics that you want, make some of the decisions around, for example, those notifications, who they should go to, if anyone, what, whether you're sending them as SMS or as emails or as in-system notifications. So there's a process associated with adopting the package which is to ensure that it covers your work processes and data needs and that those changes can be made to the system and then training associated with it would be carried out. So again, it's not simply a matter of plugging this in anywhere. There should be responsible groups that are associated with adopting this technology and a plan made for implementation that includes requirement gathering and training and the integration with your existing systems. The question about if the older versions of DHS2 can adopt this module. So we have built the WHO packages as far back in the DHS2 versions as we can in order to support the requirements because we know that many countries do not operate on the latest releases. So this AFI module was created in version 233 which is three releases back in DHS2 terms. We do two releases a year. So we built this in the version of DHS2 that came out a couple of years ago so that we could ensure that as many countries as possible would be able to adopt it. But if your country is operating on an older version of DHS2 than that, then an upgrade would be needed to ensure compatibility. You can of course try installing the package with an older version and see what challenges you run into but we haven't provided the testing on that to ensure that it is compatible. The intention actually is to help countries also make the decision to do some upgrades. The this part of DHS2, the tracker part which is where the module is located has undergone significant performance improvements in the last four or five releases. So if your country is operating on a version much older than that, you really should be upgrading your tracker instance anyway. You'll get a lot better performance and security out of your DHS2 instance if you do an upgrade. But again, this is version 233. So that's a question that can be answered by your DHS2 team in country about what version they currently have and then a decision made about whether they need to do an upgrade. There was a question about access privileges to the different levels. So that should also be a decision that is made during the requirements gathering process as to who should have access to which parts of this system. So it's the authorization module in DHS2 is very granular. So you can, what you do is you set up a user roles and user groups depending on their need for access. Are they somebody that only needs the dashboard and the analytics? Are they somebody that can review clinical records but not write to them? Are they somebody that can write to clinical records but only to a part of the clinical record? All of these are possible and they should be determined by the roles that you already have in country for your AFI process. So we have preconfigured into the package a number of these generic roles associated with data entry, associated with data review, associated with analytics review. But those again, should be reviewed based on what the roles are that you have in your process and adaptations can be made, different user authorities can be decided on. One comment on that is that again, a lot of this comes maybe at the district or higher level in terms of doing the actual investigation. And so you would want to make sure that that level of your system has the authority to write to the organization units below them. Meaning that if you're at the district level and you support nine health facilities, you want to both be able to find the appropriate records. So your search scope needs to cover those nine facilities. And you also want to be able to write to health records underneath those facilities. So that you can make sure the data are registered to the appropriate organization unit for the analytics purposes. I think there's a question about when the digital data package download for AFI will be available. And I just wanted to say that we're currently finalizing our process and our review and publication process, but we're hoping to have that for the end of the year. So the end of the next couple of weeks it should be available and we'll make a COP post about that to notify everyone when that's ready. Okay. And I think that we've actually gone past our scheduled time a bit here trying to answer these questions. We're happy again to follow up answering questions in that community of practice link. It will stay long after this webinar. Feel free to enter more questions there and we will continue to respond. We will also be sending out this set of slides after the webinar so that everybody has access to them and can follow the links that are in there. I wanted to see if ModHub is still with us if we haven't lost him. ModHub is there any final word that you'd like to say about the process for the implementation, user training or for considerations they have for adoption? Yeah, thanks Mike. Thanks a lot. First of all, I want to thank the University of Oslo team as well as the his team, as well as the most importantly, the participants who have really contributed. And I see when you look at the level of questions which have come in, the kind of interactions which we have had, I think it's really very, very good. And also the amount of enthusiasm which this has generated, I'm really very impressed to see this level of enthusiasm which this has generated. Now it's not enough to just have enthusiasm, but we need to kind of practically convert this enthusiasm to actions. And therefore what really matters is to go back and do the homework, which I think is very important. When we start, we also learned the hard way when we started implementing this data management solutions for countries like Bangladesh, for Uganda, Malawi, and so many people I can see on the call also, they are already there. And I'm sure that they are also keen on seeing how they can upgrade their systems. We have also learned that a lot of background information is necessary, not only our information, also the background work is important, especially with relationship to mapping the different centers, who has to be given what kind of privileges, how to give them access, all those things need to be kind of flammed out. And on top of it, there is also the trainings which are also concerned. So thank you very much. I mean, it's really great that we were having such a good interaction. I'm sure that more and more questions will be coming up and that's why we are having colleagues in WHO, University of Oslo and Gaby to support you to answer them. Over from my side. Bye-bye. Great, thank you so much. And thank you as well to you, Madhav, for working with us on this over the last year. Plus, it's been invaluable to have such expertise and you've been able to arrange input from so many different groups. So we are really excited to make this available. The package will be published in the coming week or so and we'll be sending out an announcement with that so that you know where and how to access that. So thank you again everyone for your participation and we look forward to hearing more from you. Thank you.