 Hi everyone, lovely to see some known faces, some unknown faces but hopefully there will be some more people joining as well because there were quite a lot of people signed up. Couple of practical notes, people will be listening also from Zoom so I understand that people will want to move from one room to another but if you could kindly pay attention when you stand up because the chairs are very loud and yeah for people especially who are listening from home there's just like people slamming things and they don't understand what it is so if you from home are hearing things slam down everything is okay here, it's just the chairs. So this is just a little overview of immunization and DHIS, of course we're not here to teach you what immunization is, how immunization works, why is it important or what not. We are more here to focus on how the digital solutions with DHIS have been integrated and strengthened the systems and how they had an impact in the countries that they have been supporting and providing information for. I mean everyone knows here hopefully that DHIS too it's well established as a healthy measure on management system both for routine immunization and of course with the arrival of COVID was quite a wake up call for a lot of people but nowadays we have more than 45 countries that are using DHIS as pretty much a backbone of their immunization programs and national programs for routine for emergencies for campaigns and such so and we have more than 30 that uptake to up took it like very quickly for COVID so you see already from the map here that you might also have seen before like we have spread very quickly and very rapidly throughout the globe and most importantly some of you might be acquainted with these lines some others not Karen is laughing because she knows it very well do not get scared this is just a quick overview of everything that we have what we say immunization toolkit immunization toolkit spans a little bit from surveillance down to routine immunizations some other applications that are fully integrated with DHIS but of course we have much more than that we have triangulation we have stock because of course there's no vaccination without vaccines that are carried around here and there aren't they and and we have data use work stream so trying to understand better how people are using the denominator and the population data in order to have a better overview of their actions and the coverage just not to have like very skewed final indicators so you can actually make better decisions in the end and of course these also trickles down in GIS and populations so we are spending a lot of time and energy is in trying to improve our our maps and the way that we can visualize the information in order to to have better micro planning for to decide our our decisions and and of course what we have in the in the toolkit it's it's just like a it's it's a wide range of information that gives us the possibility to monitor our our our efforts to inform us in order to have evidence decisions evidence-based decisions so we can go from uptake just to coverage but most importantly we are based in our packages out of standards out of global standards that can be shared and out taken quickly from countries and can be adapted afterwards in country to better mirror the context of the country that is out taking this information and of course all the things that we saw before in this flow chart and then we have on top all these cross cross cutting tools that we have seen some of them this morning because we had this presentation this morning so for example mobile data capture improved analytics and dashboard interoperability there will be some sessions on interoperability it's a hot topic very sexy topic we're aware of that and of course as we said like this morning as well mentioned our master facility lists and and target population in order to have a better overview of our targets or what we're doing where we are doing these these things and why we are doing it there and I don't want to take too much time actually so today's presenters are a wide range of of presenters coming from very different areas actually it's very interesting to see like how widespread we are nowadays but most importantly how we are going to attach different areas of the of the immunization toolkit so we're going to have Dr. Dhani Ritsato later from Paho and we are starting a little bit like our vision is a little bit as a journey so you have like Paho that is more like on the conceptual side of things that we are still planning we're still going ahead seeing how we can implement this information especially for for vaccine safety in the in the pan-american region to move then to Mali I mean we have Dr. Suleiman Samake from Heespa West Central Africa who's currently translating these informations from French to English for us and we are having a general overview of how data have changed before and after the implementation of Covid in country and then we're going to move to Mr. Rajat Billy from the Malawi Ministry of Health and we are going to see how we can leverage from the beginning of the implementation from Covid also to improve the uptake of information have this information in for example here electronic vaccination certifications but how we can leverage on previous investment to then improve also our informations also for routine data and finally from Dr. Parmot Amakrun from Heeshrulanka he's of course famous in the DHIS world last year we also had a presentation from him because in the end Sri Lanka is one of the countries that had one of the most widespread national implementation especially for for Covid so it was only worth it to continue this story to see from last year we were like at the peak of the pandemic when we had the all virtual DAC last year so to see how it progressed through time and see how it followed up and now we're going to see for example the data quality assessment that they have done after one year of data collection and hopefully like you will have like a better idea of how countries are implementing these these toolkit because in the end it's you don't have to be at 100% with everything you see that there are like a wine range you can be starting and have just like an aggregate before and after having a quite quick analysis of how your things are improving or not improving but just like so you can have a better overview that is centralizing a system up to a very well established implementation like the one in Sri Lanka so I hope it's going to be quite interesting for you to see different levels but most importantly different countries different type of implementation and different kinds of ideas that these implementations are carrying forward so I am leaving the stage to Dr Ritsata and Anya and the Baha presentation. Thank you Victoria I hold my seat. Thank you University of Oslo and all the people involved in this meeting and organizing all the aspects of the presentation but also in the support of our project. I'm part of the Pan American Health Organization and I will be presenting as Victoria said about the digital transformation of the vaccine safety surveillance in the Americas and the role of the HS2. This will be the agenda for the next 15 to 20 minutes. Who we are, what's about the adverse event following immunization surveillance and which was our proposal for the region and for the member states and also as I said which would be the role of the HS2 as a system at the regional level and at the national level. Many of you would know that the Pan American Health Organization it's an international organization for the Inter-American system. It's 120 years old actually we are celebrating our anniversary this month but we are also the regional unit of the World Health Organization as other regional offices that are present here and in that case in that sense we are working towards the people health maintenance and improving of their well-being in the Americas. Within this huge challenge immunization has a really really big part that has been highlighted during the COVID-19 pandemic. The recently appointed director of the immunization unit highlighted this role of immunization not only in healthcare but also in general well-being. As we have been saying we we didn't gather for the last two years for these DHS2 meetings so it's partly thanks to immunization and vaccination that we are little by little coming closer to the the let's say the new normal that wouldn't be the normal but at least as as close as possible and you must know that besides immunization as a huge tool to tackle the pandemic there was a huge effort and a really quick speed to roll out and develop those vaccines but on the other hand we have to take a really close look what on on secondary effects or possible secondary effects that are named as adverse events following immunization in English and in Spanish we in the Americas region we try to call them events supposedly attributable to vaccination or immunization and this supposedly highlights that it could be because of the vaccine or the immunization but it could be not related to immunization and that's something interesting to take to be aware that many people is using these figures to to be an anti-vaxxer or to highlight or to battle the vaccine campaigns so this is somehow we have to be very clear with the information that we are giving in the context that we are giving it and we have to be very clear also with the population because this is a matter of trust if we have the trust of the people they will be coming and getting their shots and that's why the Pan-American Health Organization as many other organizations and ministers of health across the world were working in the vaccine safety surveillance especially during the last two years so our goal was to build a regional AFI surveillance system on top of what's previously done usually the AFI information was sent on an aggregate basis and usually was something that was updated maybe monthly or three times or four times a year and sometimes yearly so the information that we had about adverse events was not on real time and of course during the pandemic we needed as many information as much information as possible but also on a really timely basis to take better decisions and to understand if those vaccines that were basically for good weren't doing some harm but on the other hand we also wanted to involve the people that was carrying out and rolling out these vaccine programs within the Pan-American Health Organization within other regional and international organization but especially within each country because each country are the real the real representatives of what we are doing as a regional office and as I said the core value here was trust if we can build trust regarding vaccines then we can say that we are doing our job regarding adverse events following immunization surveillance so more or less two years ago we started this project within the organization and we tried to understand which was the baseline and we performed the survey and almost every member state happened to fill this survey and to have some interviews in other cases and we understood what's the reality in this aspect in our region and because of what I've heard it's more or less the reality of many other places almost 60 percent of the of the countries didn't have an electronic system on the data gathering part as at least they were pen and paper based reports and they were transcribing this information and they they are doing that still on maybe spreadsheets, Excel sheets or other basic information systems to aggregate this data there are other countries more or less 20 that they do have information systems in place but for different reasons sometimes are political fragmentations sometimes are different governance between the the pharmacovigilance areas the ministries of health and immunization areas and so on that they have like different parts of the adverse event following immunization information so that it's a different challenge besides the one I mentioned at the beginning and there was a third group of countries they they did have and they do have a centralized web and let's say a robust information system in place and any other actor in their country that was related to adverse events following immunization surveillance was going to send information or to input the information in this system nevertheless even these systems usually don't follow international standards in terms of semantic standards for example for vaccines for adverse events so there are challenge also in these well-developed countries taking this into account we try to design a plan that could be achievable for the different levels that we find in the different countries trying to follow a two-faced approach so in in the first phase we try to gather information as soon as possible as quick as possible asking every country every member state to submit a copy of their database in an anonymized way so they they shouldn't share personal data regarding identity or or contact information and so on for the for the public health surveillance sake so they were sharing this information in a really basic way as a shared folders with encrypted keywords and password a user and and password to send this information but at the same time and this is how we have been gathering information for for the last six to ten months and almost 20 countries are sending information periodically to us indeed in this way at the same time we are moving forward to the second phase and it will be challenging more in in one countries and and maybe less in other countries to build a proper national AFI surveillance system on their own so it's not a matter of building just the regional layer but also strengthening each country system as well of course we were leveraging many of the DHS two capabilities as the electronic data capture ideally at the point of care or as close as possible the individual tracking capability trying not to manage aggregate data as a basis but to aggregate that afterwards the regarding or or taking into account which are the indicators that we are trying to build to focus also on data quality we all know how challenging is to to have a good quality in data and this is also an important matter in terms of international standards as I mentioned previously and the goal of course is to have good data to build evidence local based evidence because it's not the same to see the figures from from other high income countries or other regions and and to take this evidence and apply it right like that in our region that could be done that is done actually by now but try to develop our own figures our own indicators with local information to make better decisions to decide what to do with vaccines if we should prioritize some some groups or we should avoid some vaccines in some other groups for example at the same time we're trying not to increase the burden of the countries regarding our demands because as a regional office it's it could be easy to say okay you should send me this information but then the people in each country and you do know that they don't they have to gather the information process the information on their own for their own but also to send information to this international agency so we are trying also to make it as easy as possible to exchange this information in an automatic fashion in an in an automatic way but also within each country and this is also the reality of many of of the countries that are present here there are many other information systems so we have to foster interoperability to make the the vaccine safety systems to work along other systems that could be in place like for example immunization records or other types of health management information systems well as I said we work with DHS2 we we try to promote the DHS2 especially at the country level but we understood that as many countries didn't have the proper capacity building to start right away with DHS2 we might have to deploy a regional instance for the Sentinel surveillance so there are just a group of hospitals across the the region that are performing active surveillance and for those hospitals we created DHS2 instance with the DHS2 tracker model implemented at that level and these hospitals even though they are in a country that don't have an information system an electronic information system in place they can input data into this system for their own use but also for the reporting task to the Pan American Health Organization for the regional surveillance system especially for Sentinel or active surveillance we did some adjustment adjustment to this AFI package because we understood that it was a good starting point but we would like to to enhance those capabilities so for example we included some more variables that were included in the regional in the in the regional manual for AFI for AFI surveillance we also tried to and we are working on including international standards for coding for example for vaccine coding we are trying to adapt hoodrack that it's the standard for pharmacovigilance for for drug names pharmacovigilance and also Medra that it's an international standard for adverse events uh codification at the same time we are trying to be open and understand that some other countries might use other standards for example ICD-10, ICD-11, SNOMED and so on so we are trying not to be like really restrictive in this case but trying to understand that an international standard it's a good starting point regardless of which of them they are trying to use or they decide to use because at the end of the day it's a sovereign decision which standard to use within each country and finally we did some minor adjustment regarding the stages of the AFI package or the workflow that was occurring that was going on within the the same process uh as I said we had different installations uh we are working with the the HIST and University of Oslo in the in the regional one but this is going to be like the the mirror where the different countries are going to look for their experience to install their own system as a national information system for AFI surveillance of course we had some challenges um the first one and was also mentioned previously is the the governance challenge and one of the aspects is the data governance and and how the different data and information is flowing from one part to the other as we are an international organization there and with quite a history we do have previous agreements in place but sometimes those agreements are not that detailed in terms of which data sub should be submitted and which data should not be submitted and when we are trying to deal with automatic submissions or with electronic exchange this starts to be really really um we need to be really detailed on this kind of of issues so we are building um memorandums of agreements and and data sharing agreements with country so each country can be can be can rely on on those agreements to send information without fearing that this information is going to be misused especially in this uh in this area that we know could be really really tricky and really challenging because of the so-called anti-vaxxer groups and so on regarding adverse events of course we had the issue of capacity building within each country we are trying to promote the HS2 as a tool and to to train people from the from the region you have seen that in many maps of the HS2 implementations America in general is quite white so we don't have a lot of implementations but we are promoting it from different uh projects and and different aspects and we are uh we we know that this will change across the the time and regarding starting standard terminologies and this would be uh maybe quite technical but we had some challenges within the HS2 because those standards that that I mentioned Houdrag, Medra, ICD-11 and SNOMED I are quite large uh datasets so uh we we were used to having maybe ICD codes and there are uh 12 000 uh codes or terms in ICD-10 then when when we are moving to ICD-11 or SNOMED or Houdrag or Medra we are talking about hundreds of thousands so sometimes it's not that easy to create a value set or a dataset to import right away as an internal catalep because the whole system struggles to deal with it and also the user experience is not so good because when you are looking for uh any other concept sometimes you are not getting what you are trying or what you are searching so um maybe and when we are trying to promote also this discussion it's a time to build the the capacity within the HS2 to connect through an API with us a terminology server that can manage that complexity as a different system but something that we also are trying not to build so customized thing for our project because then it's really hard to to maintain it and it would be hard also for the countries to maintain such such a tool so we are thinking more on having something that could be improved from the core functionality of the HS2 like having these API capabilities and terminology service as an outsource coming and and helping me with the terminology uh data gathering and selection and of course as I said the interoperability challenge that is connecting not only with the Panamerican Health Organization but also with other system within the country as a proposal as I said we started with the Sentinel surveillance system and it's more or less like a software as a service for countries it's not something that will be used uh really in in a wide way but all just in in 20 hospitals across the region so it's a small small scale that's a project but at the same time it's a project that will help countries to see how it works in the real life let's say using real data from their hospitals and so on and that's why we started doing this uh initially in in test versions in in demo instances so they they get used to uh inputting data uh dummy data initially in this system and then we can move forward and start uh using it with real data and when they decide that it's a good option to have a national deployment on their own we can leverage these experiences these these lessons learned and best practices from the regional experience to build or to help building their own national implementation uh with local ownership this wouldn't be something that the Pan American Health Organization would deploy but the country itself should deploy as a national system but as I said uh many other countries do have their system so we shouldn't or at least we decided not to go uh for a custom information exchange for the for the second phase we I said that the first phase with was really simple really customized share your data as it is and we are struggling trying to aggregate this data now but for the second phase uh using standards and trying to to uh keep a structure we decided to build a fire implementation guide to um to share with all the member states fire is the acronym of fast healthcare interoperability resources there there will be another meeting uh regarding fire and the HS2 so I suggest to to attend that and the the decision or the the the reason uh why we decided to go with this was because in that way we could not only receive data from uh from from systems like the HS2 but also from other systems survey and one two three come care custom systems or home ground systems from the from the countries uh so all of them would benefit from uh creating this standard type message when sending this information to the Pan American Health Organization besides that they will use this standard to other use cases because they already are doing that for example Brazil, Chile, Argentina, of course the United States, Canada they are using fire for other projects in public health and beyond public health so there is capacity building regarding this standard to exchange information so why not using that in public health why not promoting that from the Pan American Health Organization we also know that there are other standards and there is a well-known standard for adverse events reporting not just for vaccines but for any other medications for for pharmacovigilance and it's e2b xml and this is a quite long-standing standard but it's just for pharmacovigilance so some at the beginning actually it wasn't fit for vaccines for example so we had to to adapt it somehow uh this is just a figure to show uh which is the the the plan for deploying this in in different countries we are working with these countries to uh to keep going with the project for the Sentinel point of view for the national point of view and also uh using their own systems and fire and uh I would like to thank all the people the ones that are here and the ones that are back home uh within uh this project thank you very much getting up slowly otherwise I cannot escort you all there you go um our next speaker is actually online so we're gonna test how these mix and match presentations work so I'm gonna start sharing and is he a co-host Suleiman can you hear us oh hello you can hear me yes okay good afternoon I will share my my first screen there you are do you see it on slide uh on slide mode not yet yes now yes perfect yes floor is yours okay thank you okay thank you I am Suleiman Samaki and I'm DHS2 implementer in east west and central Africa in Togo um it is a pleasure for me today to present you uh the management of routine immunization data with DHS2 in Mali with collaboration with his or Mali and his minister minister for the presentation I will speak about the introduction and some objectives the methodology the result about organization of routine API data collections and data analysis process some difficulties and contrary and some challenges and perspective we have an introduction uh I can say that the API was initiated by the Malian health authorities in 1985 in the line with the WTW approach to the fight against vaccine preventable diseases Mali has also followed the evolution of the expansion of the API to order vaccines such as yellow fever epatitis B and hemophilician fluency B in the 1980s and 1990s data were managed mainly through paper uh copy in the health facilities and also in the health administration structures at the end of the 1990s and the beginning of 2020 we saw the beginning of the computerized data management with excel files does uh different API data collection models developed at the national level like GSM software and the international level DVD and T were used for data collection and analysis these databases were used in the form of application installed locally on a computer with the expansion of the internet and the new paradigm of server based the database offering more possibilities for storing information through the web Mali has no not remained on the sideline of this evolution that in December 2013 after the evaluation of the routine health information system in this case the local health information system commissioned by the national health department and on this with the support of major evaluation founding from u.s. i one of the recommendation was to migrate from the exam software to a web-based platform like DHS to that will integrate all the data from uh a different health program uh the objective of this presentation is uh in general objective uh to outline a is to outline the organization of routine API data management in DHS to provide suggestions for strengthening the process specific objective uh to explain the organization of API data collection in the routine explain the process of routine API data analysis identifies some challenges provide perspectives on improving routine API data management through DHS to do this to do this work we've done a we do some a documentary review based on immunization package implementation mission report and the implementation report of the immunization analysis application and W2 data quality tools a result of analysis of API data from the national DHS to database web-based article publication the national health uh and social information system plan from 2020 to 2024 evaluation of mission report on the pilot phase of the electronic registry and about our own knowledge for the organization of health information system from the operational to the central level we have some results to present here before the choice to before the implementation of the DHS to the health information system activity report were provided partially in the document order the quarterly activity report which was filed out or using primary tools such as the immunization registry the tally sheets portion temperature record sheet the monthly API report there was a first level for the community health center where immunization session were organized and also the second level for the reference health center each district combined the report and of its facility and send them to the regional level which in turn compiled the this report for all its districts for the national level at the district level another compilation was made by the API officer in DVDMT the district to send their DVD to the regional and so on uh uh for the national level they uh known after DHS to implementation since the start of the DHS to implementation in Mali in uh 2016 the monthly API report has been set up in the platform this electronic activity report are set up through the aggregate component of DHS to there was a first level vaccination form for health areas and a second level vaccination form for the referral centers in january 2019 with the revision of the local health information system tools there is still the first level vaccination form but no second level form information on the logistical management of vaccine is set up in the form assigned to the district office and not to the district has a care unit this logistical management is also part of their first level forms the update of the form made uh it made it possible to take into account certain information from DVDMT namely uh information on vaccination sessions and promotional activities the immunization forms were also set up in DHS to to date DHS to is the primary tool for managing API data in Mali this was established in 2020 following the integration of the immunization analysis application into the country's national DHS to production database another result about a data analysis process this is come is the first level of data analysis particularly the technical director of counter which must ensure that the data collected by immunization agents through the primary media used for to feed the mass monthly activity report are consistent the district is the second level of the analysis for all CSCOM and the region is the third level of analysis it should be noted that some monthly district meetings are supported by the HIS regional office the HIS central office is the last level of the data analysis at the district level the APR focal points associated with the monthly meetings for their expertise for the analysis of at the central level the API is integrated into the GHIS technical administration team which makes it possible to deal effectively with the specificities related to the immunization component the analysis are currently done on the basis of the data entered in the DHS to and the effectiveness of the application of these analysis procedures can be verified through control measures such as supervision data quality reviews and order methods regarding to the prediction of statistical yearbook on an annual basis since 2018 the data regarding immunization are taken from DHS to like all order programs integrated in the platform and the last is the analysis tools used about that a package of dashboard proposed by WHO through the University of Oslo was installed by with the support of Gay V TCA country TCA that this dashboard was installed in the national production base the HIS in 2019 and adapted to the country's needs in terms of indicators analysis this application of the adaptation of the dashboard has allowed the standardization of the analysis carried out at the different levels the same dashboard is used by the regional and central level and there is a separate district dashboard for district analysis process a pilot phase of using this dashboard was conducted in the district of Chicago and Panama in 2019 and then shared with the rest of the districts a standardization also made it possible to avoid the energy creation of the dashboard by each user in the database another analysis tool is a immunization analysis app which is an application created exclusively for the analysis of immunization data in DHS tool with the ability of automatically generate tables and graphs it has the ability to generate the same types of performance analysis have in DVDMT and it offers a set of configurable indicators under the heading rhyme indicators which is exported and provided to the WHO in the afro zone and the last application here is the WHO data quality tools about that the data quality tool in DHS tool is a contribution to a practical approach for improving the quality of the HIS data and using the quality tool for potential errors in improved data quality in many cases this serves easier to modify the data or to improve the data collection system the implementation of the mode of modern statistical methods and technology such as the quality of the tools are important factors in achieving good quality in data or statistics this application has been integrated into the DHS tool database and afterwards at the national regional and district level have been trained on this here this is a dashboard favorite captures about vaccination data completeness tracks by region and month in 2021 the completeness rat is satisfactory for all maps of the year 2021 at the national level only the menaka region has a rat under 80 for 11 months this could be explained by the iteration of the security situation in this area this is another dashboard favorite capture about timeliness of immunization data by region and month in 2021 the prompted rat is under 80 percent for all month overall for the country and we have some difficulties like geographic coverage of the internet connection network the internet connection network has some issues in in some areas and the COVID-19 pandemic has infused the implementation of immunization activities at certain levels not to mention the security in the insecurity in much of the country we have also a resistance to change with the persistence of the systematic use of examples by certain API actors we have some challenge known like maintain a good level of data completeness throughout the territory and bring the prompted rat to a satisfactory level and the sustainability of the periodic analysis of data and especially the monthly meeting meeting at the level of the health districts constituted for monitoring the collection and especially the analysis of the data for decision making above the locality at both the local and national levels uh we have about perspectives we have uh two two way here the first is about electronic vaccination registry for that in 2017 there was a combined pilot phase of setting up an electronic registry for prenatal consultation for lower among pregnant women and also for vaccination the experiment was conducted by the ministry of health and public region with the support of nature evaluation uh project and the experience was carried out in the system for for Sonoma and Kangaba and was conclusive according to the process evaluation report it was therefore recommended that to be that it be extended to the older districts of the country this electronic immunization registry has been developed in the tracker component of the GISTU which is configured on a different prediction and stuff done they aggregate on which the unization muscle activity report forms exist to begin with there will be personal data entry in these electronic registers and also entry the muscle activity report forms there will be a double uh workload but when the agents have mastered the use of the system it would be decided to only fill the electronic register in order to proceed with the automatic filling of the aggregate forms based on the program indicators resulting from the collection of the same information from the tracker the second perspective is about data transfer and for that is Western Central Africa has developed an application called data transfer to send data from the individual tracker to the existing forms in the aggregated database this could be an alternative to double the data enter entry in both the tracker and the aggregated instance uh in uh conclusion now uh the integration of the API into Mali's DHSU has strengthened the collaboration between immunization and uh health information systems stakeholders in general for better health and data management of course there were difficulties particularly the persistence use of extant tools for a given period but the adaptation of the actors in the system made it possible to switch completely to the GISTU platform in terms of routine immunization data management. Thank you. Sorry. Suleiman can you stop sharing please? Okay. Thank you. All right let me share again. I'm a screen one now. No, I'm a screen two. Voila. Rajab, the floor is yours. All right. Good afternoon. Greetings everyone. So um my name is Birerajab. I had a section within the ministry of health in Malawi Digital Health Division and I'd like to thank the organizers and everyone in the room and those connected online for the opportunity for me to make the presentation on our use case and it's Malawi COVID-19 electronic certificate. Now this is the outline of my presentation. We'll look at the background and then we'll look at the vaccine registry and then we'll look at the COVID-19 e-vaccine certificate, the challenges, and the way forward. So Malawi is a beautiful country situated in the southern eastern of Africa with a population of 19 million people and we registered our first case of COVID-19 on the 2nd of April of 2020 and since then we have cumulatively confirmed about 36,130 which is the most recent that I took it a few days ago. As a nation we adopted the integrated disease surveillance in 2002. All right so since then there have been a number of initiatives on IDSA including the introduction of the one health surveillance concept. So it's a concept that considers the interconnection between people, animals, and the surrounding which is the environment. So the picture here depicts what the one health surveillance is all about. It's a multi-sector approach where we look at the human which is IDSA in essence and then we look at the animal looking at the various diseases within animals, rabbits, etc and we also look at the ecosystem which is our environment and how they are interlinked. So when we registered the first case of surveillance efforts were already underway for the one health surveillance approach and this also necessitated the need for us to first track the human component of the surveillance and this the one health surveillance concept made the country mostly divert from the original concept which was the interconnected approach now to focus on how we can enhance syndromic surveillance through better identification of cases. We looked at how can we prevent further outbreaks of COVID-19 and also most importantly we used the one health approach to help us respond to the COVID-19 pandemic. We received our first dose of doses of vaccine, about 360,000 doses of AstraZeneca on the 5th of March through the COVAX facility and since then we managed to leverage what we had achieved through the surveillance program where we had a number of programs defined for surveillance from the port of entry because most cases were imported and then to case-based surveillance when we had potential cases and also to confirmed case management as well as contact tracing. So we customized the COVID-19 vaccine delivery to kids that was issued by the University of Oslo. We made it very simple. So we had a set of attributes and then we have two stages which is the vaccination stage and the IFE. For the attributes are mostly common. We are looking at the first name, last name, mostly client demographics and then on vaccination. We had a few sections there. We are looking at the medical history as well as the vaccination information which included the vaccine given, batch numbers, expiry dates, etc. For the IFE we looked at the signs and symptoms, medical history, IFE severity and investigations. So the electronic vaccine registry was the main tool that we leveraged for reporting or aspect of vaccination. So what we did was to, as I pointed earlier, to customize the DHS-2 trucker and we had to leverage a few generic features of the trucker using the QR code scanning. We had cases where we had a thousand people a day at a vaccination site and this prompted for us to come up with ways how we can easily manage those numbers to a point where we started leveraging what we have from the National Registration Bureau. We could scan the QR code that's provided by the National Registration Bureau for us to extract some of the demographics so that the process can be a bit quicker to register someone into the system. So the picture on the screen here shows an example of it. Apart from extracting of the QR code information, we also came up with a few program rules. I think from the early sessions of the conference there was that presentation where we could scan batch numbers of vows and then the information could be extracted. We had a similar approach but we came into a problem where the vaccine vows had different ways of how the QR codes were encoded. So to iterate on the implementation, we came up with a way that would allow someone to search a batch number and then we put autocomplete using program rules the various sections within the forms. So as you can see on the screen here, for example, if I select a batch number, the expiry date would populate automatically. And then we also came up with ways how we can autocomplete future dates and it shows on the screen one way of leveraging the generic features of the capture. So and then the need came for us to come up with the COVID-19 electronic vaccine certificate. You know, for the self-movement of people and the population at large, the EPI program thought it was a necessary approach to have a credential that would help us monitor the outbreak, the outro of the vaccines and then we developed the electronic certificate with an embedded QR code that contains a public key and it securely authenticates and protects the identity of the holder through a private key. And through DHAS II Tracker, we created a user group that would allow verification workflows and approval. So as you can see on the screen, on the far right, we have the screenshot of the capture and there are two attributes below the screen which is the verified and send SMS. We had challenges setting up the SMPP service in the DHAS II Tracker. So we came up with a way through a third party gateway that we could send, we could schedule messages and send. For example, if my record is ready for certification, I just had to go to the tracker and verify by copying the verified attribute on the screen and then for the send SMS, I could schedule SMS and what you see on the middle is the type of SMS that I would receive. For example, it would say my name and then where I can download the certificate and also provide the EPI number which is a unique system, generated number for me to be able to do so. So in essence, the certification workflow, this is how it works. It doesn't usually work the same way. Sometimes you don't get the SMS if you don't have your phone number in the registry but what will happen is someone would authorize and schedule an SMS from DHAS II and then we have a CVC script that within every minute it looks in the registry which is the DHAS II tracker program for any scheduled SMS. When it gets a notification, it will push the SMS through a third party gateway to the client. So when you receive the SMS, we have a dashboard on covid19.herty.gov.mw. That's a government domain and then you would provide your EPI number and then the system through the CVC script would check the record in the DHAS II and it will pull the record through the tracked entity EPI and then it will register the record in the script database, the CVC and then the same will be projected on the screen which is the dashboard. So in essence, this is what happens and that's the sample of the certificate when it's generated. So when you scan the QR code, it's a dynamic QR code which takes you to another, which directs you to a website, a web-based application where the system would authenticate the identity of the holder. We had a few challenges throughout the implementation in the number of iterations just to make sure the usability of the application. It's where enhanced. From the stats up to now, of course, we have inadequate mobile devices and inadequate personnel to capture the people who are getting vaccinated. The demand was just quite enormous for a single person to register around 800 people a day. As a result, we have huge backlogs of data that we are capturing retrospectively. And then as we were deploying the CVC application, I think this is a comment per se, we were developing the application using Bezron 2.357 and then when we, sorry, we were developing using a test instance on 2.358. By the time we were planning to go in production, the instance that we had in production was running 2.357. And then we realized the tracked entry endpoints were broken by then. And then by the time we are upgraded to with 1.1, which is the 2.35 again on production, we are back online. So this shows exactly how brilliant and vibrant the community is and how issues are resolved in time. Connectivity has been a challenge to a point where we have a few arrangements with the MNOs where we are able to do a reverse billing. So you would use the connectivity, the mobile networks for free. And then we are built on the central, with the central mechanism where we are paying through the ministry. Now, there's another issue of incentives. So there are different campaigns on vaccination. So for example, UNICEF, through support from UNICEF, we have the Finish the Vial approach, where the HSAs, they are allowed to finish a VAL in their incentives associated with that approach. So the data person is left out through that arrangement. So for you to get someone who is data oriented to capture the vaccination data, it's quite difficult knowing that the colleague is getting incentive while he's not. So that's another challenge. As a way forward, we are planning on developing fire compliant resources to connect our local instance of the CVC to other initiatives, regional and global. And then we are also thinking of connecting the database to the National Registration Bureau system, because we are already extracting the national ID from the cards into our database, which is the HIS tool. We are thinking of how we can extract other biometric information to enhance security. And then we are thinking of expanding the use case, the lessons to a registry where we can accommodate all other vaccines. And this goes without mentioning some partners and funders who have been here, who have been supporting us throughout the implementation with clearing the backlogs to the actual technical support. And this marks the end of my presentation. Thank you. Good morning. Good afternoon. Good evening, everyone. So I'm Pamod from Sri Lanka. I was initially supposed to present about data quality in vaccination, but because our Ministry of Health colleagues from Sri Lanka could not join, I'll be also presenting on the COVID immunization tracker, and I will combine it with the data quality assessment. Right. So a little bit of background about what happened in Sri Lanka during the pandemic. We'll be talking more about it tomorrow in the plenary session. So in a nutshell, from the start of the pandemic, we were able to produce kind of a DHS2-based ecosystem for surveillance, where we were able to design about 10 modules on top of DHS2 platform in like a four months period. So this involved a lot of capacity building and a stakeholder engagement, and we were able to take this forward following the initial six months. So that was the time when we were more focusing about whether we can actually routinize whatever we initiated during the initial times of the pandemic. So this was when the discussions around whether incorporating immunization data in DHS2 was possible. Now I'm talking about latter part of 2020, and we were able to quickly customize DHS2 again for our immunization data capture just before we received the first set of COVID vaccinations. So this was again in late December, early January 2021. So just to highlight all the modules that we have in the COVID DHS2 COVID surveillance package, and immunization is just one of them. I just want to highlight this because it has so many other components. So what do we have in DHS2 COVID vaccination or COVID immunization tracker, as we call it in Sri Lanka. So again, like just like the surveillance, we were the first country in the DHS2 implementation world to thank you to use DHS2 for COVID data capture. So the components that we have first thing is the tracker, the immunization tracker, it's a registry, and then the stock monitoring component, which is aggregate component, and then the vaccination certificate and a citizen portal for the appointments. So the aggregate data, again, it's a simple form to capture the stock data. And of course, the major component is about the tracker where we have the immunization tracker. It's, as you know, like it's a simple customization of the DHS2 tracker. But we have done some slight modifications to the tracker capture application for the integration of the certification component, which I will explain in a while. And in addition, we also capture AFI information. And we also have a pregnant COVID-19 vaccination cohort, which is captured separately outside of this tracker. I mean, it's complimentary, but it's managed by our MCH program. And also analysis and visualization of COVID immunization data. So we used the existing available analytic tools in DHS2. And also we use dashboards, we kind of export data as Excel. And we are also using SQL waves in certain certain situations where existing analytic tools have some limitations. Right. So about vaccine certificate, so there was a lot of discussions within our his community, as well as a few others stakeholders globally. So our initial version of COVID certificate, we produced, I mean, by the time we implemented our immunization tracker back in 2021 January, but that did not have this cryptographically verifiable component. So what we actually did was we kind of modified the DHS2 tracker capture, and we also have some backend component going. That's how we initially implemented the first version of the vaccine certificate. And also we had this interim guidelines, which has been getting updated time to time. This was the guiding document that we initially referred. And then this was our initial version of the certificate. Right. And then to complicate things a bit more, but to make it more, how would I say, technologically standard, we use Daivok, which is again a digital public good and used in India to producing COVID-19 certificate. So what we did was to create an integration. So this is kind of a technical workflow of how data flows between all these components. And finally, we have DHS2 and Daivok integrated together to produce the COVID vaccination certificate. Right. I'm mindful of times and I want to keep some time for Q&A, so I'm going a little bit fast. I hope you don't mind. So back to the COVID immunization campaign and where it is standing now. So the COVID immunization campaign was a country-wide immunization campaign, which we had to organize rapidly and deploy it rapidly. So this word rapid is very crucial because it was not like any other vaccination campaigns we have conducted before. We had to customize when it comes to, I mean, I'm leaving aside all the complexities around, I mean, distributing vaccine and conducting the vaccination campaign. I'm only referring to the data component. So we had to customize the solution, which is in this case, DHS2, and do some trainings in midst of pandemic, all online, and then deploy the system. And we also had multi-sector collaboration again, just like what we initially experienced with the COVID surveillance component. So we had the Ministry of Health and other ministries, including the National ICT Agency of Sri Lanka, who was kind of communicating and organizing all other stakeholders. And we also had WHO country office, who kind of played a major role in supporting with the technicalities of configuring the tracker and the certificate and things like that. And also the military, because in Sri Lanka, all the vaccination centers, these are kind of temporary establishments. Data entry support is provided by the Air Force and the Navy, right? So we have Air Force and Navy personnel in all vaccination centers, not to do anything else, but to enter data, up to date. Right. So right now, probably we are implementing, I mean, this is the largest DHS2 based tracker or else one of the largest. So we have 20 million people registered, mind you, our population is just 21 million. There are some duplicates here. So, and we have 40 million plus vaccination events and 160 million plus attributes. Like these attributes are like properties of all these, I mean, people who are registered in the system. And then we also issue vaccination certificates for all the travelers. So that is why this is one of the most crucial systems in Sri Lanka right now, because I mean, the moment if you have some technical issues on this system, I mean, we all get notifications and calls and like, so it's very complex. And there's a lot of attention to this system. Right. So what is it all about data quality of this vaccination tracker? The thing is like, this is a contrived implementation, which is not really planned in our, I mean, traditional way. So usually even, I mean, the vaccination campaigns are well planned across a couple of months. And when it comes to information systems, we kind of implemented gradually, like probably in one, one, two districts, provinces, not kind of like contrived implementations. So it was a rapid implementation. So there was no proper testing. We were just relying on and we were, we were having, we were actually trusting the data into people and the health staff at field level, because they were really supportive of our implementation before. And we had multi-sector engagement. So I told you, like, it was not just health staff who are already familiar with the DHS2 system. So we have our field health staff like midwives and public health inspectors who are usually, I mean, who have been using DHS2 for so many years. But here, we are talking about security personnel. I mean, okay, here you go. Like you have a system, please enter data. We did that was like, we provided them with the one page user document and we had a YouTube video. So you were supposed to kind of study that and probably have again, have some peer learning and then enter data. And also we wanted to do some research around this large amount of data captured in the DHS2 tracker. And again, like we have routine in many activities, which are there in our public health implementations. But we were kind of doing some fast-tracked implementation here. So it was kind of bypassing some of these like checks and stuff that were already in place. So this is why we had to focus on the data quality. So to do that, we did an assessment which had two components. One is the quantitative part and the other one is the qualitative. So in the quantitative, we were mostly focusing about the completeness and timeliness. Of course, the accuracy and validity part is happening as of now. So this is not a complete study as yet. So to do that, what we did was we used the existing analytic tools of DHS2 to assess some of these criteria. And sometimes we used triangulation. So we have the data captured in DHS2. So we always have to compare it with another source of data. So what we used is that we have this aggregate data that is coming to the epidemiology unit about how many vaccinations were conducted from each vaccination center. So we kind of used that to triangulate. And then we also used some custom scripts to pull data from the DHS2 database to do, especially this timeliness assessment was done like that. And we also did some qualitative assessments based on interviews and observations throughout the country. Right. So this is what we have for completeness. So what you are seeing here is like completeness as a percentage for total vaccination doses, first doses, second doses and third doses. Mind you, this is tracker data. So this is individual vaccination events and not aggregate data which we are entering at the end of the day. So individual patient records, I mean each vaccination event was recorded. And we have around 87% completeness for this individual data which in our opinion and our experience from previous tracker implementations, it's very impressive to get 87% completeness, I mean, within like a couple of months. And we also note that for the first dose, it's like almost 90% and when it comes to the third dose, it's less. So that means like to achieve a good level of completeness, it might be taking some time because we already done with the first dose. So probably that's why we have around 90%. Right. It gets more interesting when it comes to timeliness. So what you are seeing here is like we are assessing the timeliness, like when was the data entered. So we are comparing the vaccination date, the date, the actual vaccination happened and then it was entered into the HRS2 system. So it's quite interesting to note that we have closer 30% of the data which is entered on the same day. That means it's kind of like real time data entry. So we have around like 30% one third of data which is entered real time. And we have like, I mean, this same day combined with two days or less, it comes to almost 40%. So like whatever the data that is entered within two days means like, so why people are not capturing data the same day, we have just looked into the issue. So the thing is like in some cases, you don't have good infrastructure, internet coverage and things like that. So they prefer to have to maintain their Excel so paper records and go back to the facility and enter data. But data coming, this individual data coming in two days is impressive and really good enough for us. So up to 40% of the data we are getting in like two days time. And within two weeks or less, like so almost two thirds of the data, looking at this chart, we see like we'll be able to get it within like two weeks. So two thirds of the vaccination data, which again, we felt was good enough for a campaign which was rapidly deployed. And then there are some instances where it takes months for us to get data. So these are areas where you have some governance issues and some issues related to infrastructure and things like that. And you may also notice that there's here the negatives. So this again is some data quality issues. So negatives means there's issue with the dates, right? So this happened, we are trying to figure out what really happened. But this is very less, much less than 1% of the data that is there. So this we figure out it's something to do with the initial imports and things like that we did in the initial few months of the implementation. Right. Okay, so the qualitative data, the positives, there are a lot of positives. So for example, configuration of tracker program, like things like program rules really helped us to achieve the data quality. So we made it so structured that people don't really make mistakes. And availability of dashboards to supervisors was really helpful so that they can always figure out if something is wrong, right? They can always reach out to the people who are entering data and try to figure out what went wrong. There are areas to improve. For example, we had like, I mean, the thing is like we had so many different types of users entering data. So that was very complicating. It would have been ideal if we only had health staff entering data, but that's life. And then we also have lack of supervision, because we were kind of fast tracking some many mechanisms which are already in place. And then we also noticed some gaps in capture interface, which we are trying to improve. And then like some localized governance issues when it comes to providing like some infrastructure and guidance on entering data. So these were some areas that we're lacking. So finally, in conclusion, we believe like we were able to achieve high levels of completeness and timeliness, accuracy data pending, probably in couple of months we'll be having that data. And there's of course scope for improvement in our current configuration, the way we implemented, and we have already put some, I mean, taken some measures to improve that. And monitoring and evaluation around data quality should be established for case based data. It is already there for aggregate, but there are some gaps when it comes to a case based data implementation. And then of course, we need future research on routineization. I mean, yes, COVID was like a major focus. There was a lot of attention. But like, are we able to establish the same when it comes to routine immunization data? It's a question which has to be answered in future. So thank you very much. I must especially thank all the stakeholders who supported and the Ministry of Health, especially. Thank you. Thank you very much, Pamode. And of course, Roger, I just wanted to save as much time as possible in between you two tokens. And I hope you also understood why I was so very excited when I saw Sri Lanka submitting the abstract. I mean, a follow up after one year doesn't happen all the time. And I think it was quite worth it to have a highlight there. We literally have five minutes left. And if anyone has a question, is your time? Or you can of course, okay, there you go. Of course, questions on the community of practice as well. And for those who are here, I mean, you can just like literally tackle people in the corridor and force them to answer questions as well. I'll get there in one second. Hello. Okay. Thanks. My name is Patricia. I'm working with the WHO Health for Pandemic and Epidemic Intelligence. And I'm also working with the Digital Health and Innovation Department back at HQ in Geneva. All interesting presentations. So thank you. I had a specific question from Malawi. I was curious about the development of the COVID-19 e-certificate. We developed a DDCC guideline. So it's the digital documentation for COVID-19 certificate. Wondering whether this guideline was used, leveraging the development of that certificate. And also in the presentation, there was a mention of next steps being the development of fire resources for COVID-19 certificate. And I believe that that's something we have developed as well. Trust not to duplicate work just in case. If you need that, I'm happy to put you in touch with the team. Thanks for the wonderful patience. Yeah. So for the implementation of the CVC COVID-19 vaccine certificate, we followed the virtual guidelines. And we had to submit the same to the United Kingdom for them to start accepting the certificates as people were traveling to the UK. So we did follow the guidelines. As for the fire-compliant APIs, we are thinking of how we can connect the instance of the CVC to the rest of the world. So for example, DHI still gives us the generic APIs, which are not fire-compliant per se. I know there's the adapter. We didn't leverage that. So yeah, we could also look at the virtual, what you have developed and see how we can leverage from those synergies. Thank you. In that case, I declare it over and you guys are all free. But thank you so much for coming here and thank you so much for our presenters. And remember that now we have the group pictures. So please don't run away and only after that, if you're made for the picture, you can have the cocktail. Otherwise, you can't.