 from everyone to the third session of today. And this one is really just such a fascinating session because it's one of those topics we would have never thought to have in the conference, but we received just incredible abstracts from the community around these innovative use cases of DHIs too for event-based surveillance. So I'm sure many people know that's more than 20, 30 countries are using DHIs too for IDSR and indicator-based surveillance, but there's been some really incredible innovations throughout the community. And we'd like to share some of those stories with you today. So our first presenter is Emma Ibazibwe of HIST Uganda. And in Uganda, DHIs too, it's been used as the National Electronic Case-Based Disease Surveillance System. And so I'm very pleased to welcome Emma to share how integration with the mobile SMS platform has really been able to enrich this integrated disease surveillance system with community-based reporting of events and analysis. So welcome, Emma. The floor is yours if you'd like to share your screen. Thank you, Rebecca. I'm requesting you share your screen with my slides. Okay, I can pull that up. Just give me a moment. Is this showing up? Yes, it is. Thank you very much. Okay, floor is yours. Go ahead. Could you do presentation mode? Thank you very much. Good morning, good afternoon, and good evening to everyone. My name is Emma Ibazibwe, and I work as a project officer on the EIGSRIHR project for HIST Uganda. And I'm here to talk about the event-based disease surveillance system and the rumor monitoring that we are currently working with for Uganda. Next. Rebecca, next. I think there might be a lag. Do you see the correct screen? Yeah, there is a lag. So we, this was developed based on the EIGSRIHR, the previous one. So this was developed based on the EIGSRIHR guidelines from WHO, and they were trying to encourage the use of electronic systems. I'm sorry about that, Emma. Is it showing up correctly now? Yes, it is. Thank you. So our implementation is based on the EIGSRIHR technical guidelines, which recommend the use of immediate reporting of notifiable diseases electronically. And this can be through email, SMS, or phone calls. And Uganda has adopted the use of the SMS platform, which is working hand-in-hand with the DHS2 Mobile Configuration app. The integration of the two platforms allows the rapid relay of information about events from the community. And for people describing potential risk within the community, so these are targeted people. It is anonymous in nature, but we are targeting people that have been trained, and we expect them to report from the community about any events that might be, that might require district response or national response from them. Then to facilitate the implementation of the to further facilitate immediate reporting. Now this is for the indicator-based reporting. There is an aggregate data report that is submitted on a weekly basis from the health facilities using the same platform. And this data is then integrated into the national HMIS system that is on the DHS2 platform. Next. So the key configuration is, there is a gateway in EIDSR that has been configured to receive traffic from a keyword that is alert. So like I mentioned, we've set up an unregistered PASA that allows unregistered users to send anonymous events into the system using a Minister of Health Uganda code to inform about what is happening in their communities. Then this has been configured to further provide feedback to the people who are sending alerts, as we'll see in the coming slides, the kind of feedback they receive. The same platform is also, I can do it between the community and EIDSR as it tries to, as it tries to, it is a conduit between EIDSR and the community as it provides feedback from the community. So our current workflow is, we have a VHT, a healthcare worker or a community leader who has been trained to send an SMS with the keyword alert to 6767. And this SMS is received by the National Public Health Emergency Team at a Minister of Health who then verify the alert SMSes and prepare a response team to be able to provide the necessary response to any of their community. In case it turns out to be a risk or an actual event, they'll prepare a rapid response team either national or district. These are based on how intense the situation is. So if it can be managed by the district, then the district response team will be sent. Otherwise, the national response team is going to be sent to be able to handle this next. Rebecca, do we still have a log? I suppose we do. I don't think there's much I can do. I can share it this way if that's easier. How about we try it this way? You know, just say verbally what slide you're on because the rest of us can see it. All right. So this is the slide we are going on. So using any of the registered SIM cards, again, registered SIM cards, an individual, a community person will send a description of alerts from their community stating what they are seeing or describing what they are seeing and telling us from the location that is the village, the parish, the sub-county and the district. So once this is sent into the system, the team at national level, that is the Public Health Emergency Center, they will verify by trying to either make a phone call to the team by making a phone call to the person who sent because as we know, DHI is to register as the number or it picks the number of the person who sent this SMS. So they will try to follow up or try to verify this alert with a phone call before they send it out to the, they send out a team to do further investigations. So this notification, once it is verified, a notification is now sent. This is either email or SMS. It is sent to either a program that is in charge of managing this disease or condition or it is sent out to the rapid response team at the district level or the national level to be able to do the investigations. And right there in the middle, we have feedback that is sent back to the senders, letting them know that their message has been received and there is a team that is currently working on that particular issue of verification. Next. So we have been working on this since 2013. And in 2019, we tried to revise the situation. We met up with us, we're supposed to be on slide six. I hope there is no lag. There we sent, we had a meeting or we went back on the drawing board to try and see how to engage people at the district level. And they shared some of the gaps with the current workflow because of, because the SMSs go straight to the ministry without engaging, without first engaging the district teams, they felt they were left out. And these are some of the gaps that were identified. There was untimeliness to the response. This is due to a lot of SMSs or alerts that come in. And there is no HR to particularly verify all of this. And then the screening of alerts or events was cumbersome. And this is due to very few people to handle this and also how the SMSs come in. We don't really have coded messages and all that. The messages are a bit unstructured when they are sent out. So you have to like read through and confirm before you can actually verify. Then at certain points, there is delayed feedback to the users and this demotivates them. Especially recently we had a change in the gateway in the third parties service provider. So there is a sort of delay to get feedback to the teams on ground and they get demotivated. They feel like their messages are not reaching where they're supposed to go. And there is a poor documentation of the rumors at health facilities and districts. They do not have log books or they do not have structured ways of recording any of the rumors or alerts that come in. Inability to analyze rumors or events at the end of the day, one of the biggest things has been how many SMSs have you received? How many can be reported by disease and so on and so forth? And then there is no special mechanism to respond to rumors that come through, for example, social media. We have a lot of social media present and we expect some of the rumors that come in would be through social media. So there is no special mechanism of doing that. Now, we've tried to come up with a solution to tackle most of these gaps that have been identified by the community. Next, and the biggest challenge was how do we involve the district teams in the process of verification? How do they find out about conditions, about how do they handle conditions within their community before either the minister or any other big person knows about it? Because someone was like, I don't like the minister or NMP telling me what problem is happening in my community. I need to be able to know about this first hand and not third hand from a political official. So the revised workflow still involves the healthcare workers, the VHTs and community leaders sending SMSs to 6767, clearly indicating the location and then using the DHIS to SMS management app. This is something that we've come up with as a team at Hisp Uganda. Using this particular app, the Public Health Emergency Center is supposed to route or forward these alerts to the respective districts based on the location that was previously stated. And then it is now the district teams that are supposed to verify these alerts and while filling in an ELOG. So the ELOG is there to be able to indicate the verification process and whether you are able to verify to determine a true alert or a false alert. And then the district team will also go ahead and dispatch their investigation team to conduct further investigations. Next. Rebecca, this particular slide, we need to be in a presentation mode. Thank you. Okay. So we've developed an app that is installed in the EIDSR and we are calling this the SMS management app. And this management app provides analytics based on the SMS is received. We're able to determine SMS is received. We're able to determine those that have been forwarded for verification and the status of verification and so on. It is further able to produce an ELOG which has been integrated just a minute, which has been further integrated into a single event program in the EIDSR DHS2 platform. And then the ELOG actions can be monitored both at national and district level. So if there is a delay in the process of verification, the national team is able to follow up with the district surveillance for co-person on why certain activities are taking longer than they should for verification. So right there, you're looking at the dashboard. We have total SMS messages received. Then you have the total alerts. And the next screen is showing us what the public health emergency center screen looks like. They have the received alerts or messages. Next. And based on the districts within the SMS, they'll be able to forward this to the particular district. We saw it was Arua. So they will be able to forward this to Arua. And then the district person will come and look at the forwarded alerts. So their screen will be around forwarded alerts. Next. And they can go ahead and update events accordingly based on the verification information that they're getting from the community. And yeah, so this is the screen that they use for verifying. And as status changes, the system is able to record the status updates regarding the system. They can save if they are not able to get all the information in one go. However, if they do, then they can just update it. Next. So at the end of the day, you have a complete form. So there is an e-log that is recorded in the background and we're able to generate data using that. And then we can also try to determine the status of what is happening in the community and be able to follow up on this. So the revised workflow is involving the district teams at the time of alert verification. It is also providing timely response to the alerts in the district. There is no delay between going to national level and then another person has to share this information. And then we are able to monitor progress of the alert verification by both the national team and the district team. Then we are able to log alerts and we're also able to analyze our alerts at various levels and using various desegregations within the system. So these presentations can be done either within the app or outside of the app and using the DHI's two legacy tools to present. Next. Two minute warning. And yes, five minute implementation slides. Implementation challenges. Yeah, so our implementation challenges were also the gaps that were identified. And I'm sure HR is still going to be a very big issue but we're trying now, we try to solve the issue at national level and we're trying to engage the district teams in the process of verification. Then reviewing of messages is challenging because of missing information. Like I mentioned earlier, the SMS or the alert is unstructured in nature. So some people tend to miss out on the revealing information about the alerts that they send. There is delayed feedback or relay of messages due to gateway stability. I just mentioned we had a new third party service provider and there is a bit of instability in the service provision there or the connection between DHI's two and telecom companies. Then there was lack of proper documentation but we hope now with the ability to verify and record your findings. This is also solved within the application. Limited data use. So our next goal is to try and increase the use of this data at various levels, especially since we are able now to analyze how the events come in by period, by categories and so on and so forth. Then our media and social media is still going to be a problem but within the app we've tried to see how to add alerts that are not necessarily coming in as SMS but they're coming in through other channels. So we are also trying to collect that information and we will see how to incorporate that. Now one of the biggest challenges of implementation is actually language barrier because the people we meet, although they might be able to speak English, they don't necessarily know how to write English. It's still a bit of a problem. And for Uganda we have a lot of languages. So we cannot say send it in your local language. The person at national level might not be able to understand your local language. So the language barrier is still going to be a very big problem that we actually don't know how to solve. Thank you very much. Thank you, Rebecca. Thank you so much, Emma, for this use case from Uganda. And I apologize for the technical difficulties on my side, I really do. But this was an amazing presentation. We also have it available for folks to download from the SCED site. So I'm very pleased to welcome our next presenter who is Natalie Tibbles from the Johns Hopkins University, the Breakthrough Action Project. And so Natalie, this was such an interesting abstract when I read it and what I find particularly inspiring is how different partners such as JHU are starting to use this metadata packaging approach to disseminate their designs. So Natalie, the floor is yours to discuss how you've used DHIS2 to combat misinformation around COVID-19. I will stop sharing my screen now. Thanks. Okay, are you able to hear me and see my slides? Rebecca, I'll rely on you for time. Thank you. Okay. Thanks. So I'm Natalie Tibbles, I'm a monitoring officer with the Johns Hopkins Center for Communication Programs. We go by CCP, so you'll hear that a lot today. My colleague who co-designed the system, I'm presenting today in Cote d'Ivoire is Abdul Doso and he works in the Cote d'Ivoire CCP office and he'll be presenting for the Francophone teams in the next hour. So CCP is an academic center within the Johns Hopkins Bloomberg School of Public Health and we primarily work on social and behavior change programming as well as knowledge management, advocacy and applied research. That's my focus. We work in over 30 countries and the Breakthrough Action Project is a partnership led by CCP with a variety of partners. It's a five-year global project funded by USAID to lead their social behavior change programming around the world. So we work on a variety of health areas including family planning, reproductive health, malaria, maternal and child health, nutrition, neglected tropical diseases and personally I worked most in HIV where I first met DHIS2 projects on Ending Child Early Enforced Marriage and on the Global Health Security Agenda Programs or GHSA. So GHSA is a group of 70 countries, international organizations, NGOs, private sector companies whose goal is to address gaps in countries' abilities to prevent, detect and respond to infectious disease threats. And as part of GHSA Breakthrough Action contributes to SBC programming, SBC is social behavior change programming, designed to increase capacity for risk communication. So as we've seen in repeated Ebola outbreaks, COVID pandemic and many, many other health emergencies, the ability to quickly bring an outbreak under control requires behavior change to wear masks, to report cases, to seek care. And these behaviors are preceded by determinants such as knowledge, attitudes, norms, trust, self-efficacy. So risk communication is the real-time exchange of information then between officials and people facing a threat and the goal of risk communicators is to enable them to take informed action to protect their health. So integrating SBC approaches into risk communication interventions then the idea is to support the shift of behavior and social norms. So for example, we facilitate community engagement to foster trust, which allows for better uptake of recommended behaviors, early reporting, et cetera. We work to enhance coordination between different partners to allow for harmonized approaches across channels. We invite communities and local responders and leaders into the process of identifying rumors and information and sharing the concerns of the population with people who can kind of address them through risk communication. And that's what I'm talking about today. So this may seem really basic, but we use the word rumor, and we often don't define it. So what is a rumor and why focus on documenting rumors? Rumor, a rumor is an active communication that contains unverified or false information and rumors emerge when there's not enough accurate information or when there's simply too much information. So many of you know the WHO is called the COVID pandemic and infodemic for this reason that there's just too much conflicting information and that makes it hard for people to discern who to trust and what is right and that's what action to take. So people share rumors for a variety of reasons and thinking through this really helps me kind of have empathy. Sometimes people share in good faith, we call that misinformation, they wanna help. People have a desire to explain things and to make sense of what's going on in their lives and rumors can be a way to cope with uncertainty or suffering. Of course, some people do spread disinformation which from a technical perspective is the propagation of rumors intentionally to disrupt or to make a profit or for economic or political gain. Some rumors are harmless but others can undermine the public health response in a variety of ways. They can harm individuals actual just physical health such as the rumor that was circulating early on that drinking bleach would cure COVID, can create barriers for accessing services such as the idea that Ebola treatment units were actually infecting people or they can create obstacles to prevention behaviors such as the rumor that masks were contaminated with COVID, they can stigmatize, groups perceived to be exposed or at risk and fundamentally they can undermine trust in the public health authorities in the response. So there are also times that rumors are true and I always need to mention this, from a response perspective, we can get kind of high and mighty about the whole thing but there are rumors, for example, there have been rumors in different emergencies that humanitarian aid workers were abusing people they were supposed to be serving and there have been times that that's true and this so-called rumor was a signal of something that urgently needed to be investigated and stopped. So our goal then of a rumor management system is to systematize somehow rumors that rapidly emerge in communities so that our awareness of the rumors can inform social behavior change interventions as well as on occasion really follow up the situation that's occurring. Some systems that we're implementing through CCP capture rumors about COVID-19, others are about Zonai diseases, not COVID-19 or other health areas like HIV or family planning. So to design a rumor management system, we were looking for a digital solution, of course, with a set of features, we wanted a cloud-based, able to collect, store and visualize rumors in an integrated way. We wanted the system to function under kind of business as usual conditions but also be able to rapidly activate and scale in an emergency. And we wanted the system to be able to incorporate rumors from various sources, both people directly reporting in as well as importing from secondary sources and social media. And so the task really is, how do you organize and process this unstructured data in so-called real-time? Quantitative data entry is fairly easy. You said the data collection forms, not easy. I shouldn't say many of you are doing really crazy, amazing things, but you can define the indicators in advance and they can populate as you go along once you've set up the whole system with unstructured data, as Emma pointed out, made a very good point. It takes time to process and review that. And so how can you do that in a systematic and rigorous way? I wanna draw your attention to the principles for digital development, which is a really helpful tool for assessing your technology solution. So in our case, the system, the rumor tracking system is not just the technology, it's the network of contributors to recognize and report rumors, partners who share data to mine for misinformation, data managers, stakeholders who review and actually respond to and address the rumors, but the technology is critical and you want a technology that is sustainable, affordable, secure and fits in well with how countries are already managing health-related information. So as we're hearing today, this type of system is not, CCB is completely new idea, we're fitting into an ecosystem of health surveillance, health communication, rumor management, and we wanna make sure we're serving the interests and needs of each context and not duplicating. And so a lot of these principles help us kind of assess that and develop a technology that's gonna work. So we looked at a variety of platforms, Google Suite, Kobo, a few other platforms, but we settled on DHIS2 as the database because of any of these principles. So I wanna go into a brief overview of how the system works, and then I'll go into a little depth on each stage. We collect rumors either by having trained contributors enter them, enter what they hear into the system through the DHIS2 app, just off the shelf collection app, or we import from secondary sources using the data Z app. Data managers then code, analyze and synthesize rumors. And here again, I'm using rumor to refer to misinformation, to beliefs, not necessarily events. We share the rumor information very frequently, sometimes twice a week with government, other partners through briefs, presentations at emergency coordination meetings, and of course, through access to the dashboards. And then we offer technical support for risk communication and also directly implement mass media, social media campaigns and community engagement informed by this data. There are a variety of sources for more data surveys, social media, qualitative research studies, the real-time contributors that I mentioned, and other secondary sources. And we've used all these sources and incorporated them in various settings, but we have primarily relied on the community-based contributors and hotlines and SMS lines. And many other partners would cross and others have been doing this for a long time for emergencies like Ebola. So all the rumor data goes into an event program, typically at the district level or subnational one or two level. We have a section to enter the raw rumors, just unstructured data, can answer a few descriptive questions, and then see the actual, the people entering the rumor data, the rumor apply topical codes, which I'll talk more about in a minute. I wanted to just pause and talk a little bit about again, kind of how we're conceptualizing the system. So we train contributors to remove names and addresses of individuals when they're submitting rumors. During the coding process, we watch out for inferential identifiers where even if a direct identifier isn't included, someone might be identified through inference. For example, the chief of a certain village and the only one chief, someone could figure out who that's talking about. And this is a key point because this is a social behavior change tool to inform risk communication, not a surveillance system as we normally think about it. So someone somewhat different from other types of rumor systems where the purpose is to investigate events. So we do accept reports of an outbreak or a problem in a certain place and then refer it to the authorities. But for us, that's not the primary purpose of the system. We're not attempting to track people down who are spreading rumors or the first person who started the rumor and punish or persuade. We're not tracking people, we're tracking ideas. And so for the key informants, we prompt them with, what are people saying about COVID-19? And we ask them to submit, I've heard people say, or people are saying, dot, dot, dot, because it helps remind them and those interpreting our data that we're not trying to track down individuals. So this does involve some limitations on the amount of information provided about the original source of the rumor. And so it's not a survey, it's not population representative, it's the snapshot of community concerns, of misinformation, and it allows us to rapidly identify potentially problematic misinformation to nip in the bud before it becomes a problem, ideally, and that allows us to inform risk communication and community engagement by representing some of the concerns at a given time in the population. It's an SBC tool for an SBC response. In terms of processing the rumors then, we have dedicated data managers who code the data as it comes in. We code first topically, then we group similar rumors into belief statements that summarize the specific theme. Then we classify the belief statements according to their urgency and harmfulness. Our CCP staff around the world have years of research and programmatic experience to analyze social and communication data. And so this is where that expertise comes in. For example, early on in the pandemic with the system in Cote d'Ivoire, we received a cluster of rumors before vaccines were authorized or available. So here are some of the rumors. When you go to the hospital, it's a COVID vaccine they're giving you. People are saying not to trust vaccines or get your kids vaccinated. If someone says you need a vaccine, it's because they need to make us guinea pigs. So those rumors all would receive the topical code vaccine, I mean, the care seeking code. But we also synthesize it into a belief statement, something like routine vaccines or actually COVID vaccines they're testing on us. If we relied just in the topical code, we would be stuck trying to develop messages on vaccine. You can't really do much with people are talking about vaccines. You need something more specific and actionable that people are believing that might undermine, for example, routine vaccinations. So the idea, if people believe that the polio vaccine is actually secretly a COVID vaccine that they're testing on your baby, that's something we can assess and classify. Is this belief harmful? Yes, because if people believe it, they won't wanna get the routine vaccines that protect their children. So we then recommend, this is a rumor that warrants a response and we're gonna address it through risk communication and community engagement. We have deductive codes that we developed in advance and then we add codes as themes emerge and those would be the inductive codes. So a challenge with adapting DHIS-2 for rumor tracking is just accepting the idea that I'm creating a case management program not for a person, but for an idea. I've run into this many times before doing SBC work. DHIS-2 was designed as far as I can tell for standard health service information, yet I know I'm not alone because tracked entity type exists. So we find ourselves tracking community groups, local structures such as schools. I wanna give a shout out to my friend Simon at Aswatini who's innovating on this front with DHIS-2. We track roles, we track communication materials and now we're in a position of literally tracking ideas. For those of you unfamiliar with this picture or the media context in the US, it's from a popular show that basically makes a joke about someone who's trying to figure out something complicated and looks totally crazy. And this is how I felt trying to figure out what does it look like to case manage an idea, a belief, a rumor? What are the attributes? What are the stages? What are the indicators? How do we really use and operationalize this thing? So we basically keep the raw data in the event program and then add the synthesized statements to a tracker program. We keep track of nuances, versions of the rumor, according to that kind of checklist of harmfulness or threat which we use to recommend if and how a rumor should be addressed and then we document the response. We have an internal dashboard that tracks the number of submissions and if they've been coded and analyzed and if the coding has been verified, which health areas, the rumor addresses, things like that. We take a look at the sources and which sources are providing the sort of richness and nuance that we want. So you can see here, we looked at hotline submissions and submissions from the community contributors and the themes are similar. If you sort of break it down, but there is more detail and richness and submissions from the community contributors and looking at sources that way and kind of doing some analysis helps us figure out where to invest in terms of our rumor sources. And really, more is not always better. You can't manage a volume of millions of rumors with a couple of data managers a month. It might be in the hundreds or thousands that you're coding weekly or monthly. And so you really wanna manage your investments well. We share summaries on thematic dashboards that allow stakeholders to get a sense of what topics are common among submissions to the system. And we like to share illustrative rumors, some of the raw data, so stakeholders hear the voices of community members, not just the summary. Here you can see we've organized the raw data by belief statements. The first is it's useless or dangerous to seek care. The second is COVID-19 doesn't kill Africans. So you have the belief statement that summarizes the theme, but then you also have the real kind of voice of the communities to look at. User feedback has been positive. Community contributors described feeling inspired and equipped for their role as community listeners and reporters. They felt the system was easy to use and relevant and they wanted to continue to expand it. And the partners that we're working with and risk communication were able to incorporate insights into a variety of responses. So in summary, really the bottom line is that DHIs too can be repurposed not to track individual people, but to track ideas. And in order to say the system is functioning, this is what we really want. We wanna be able to put unstructured data in and pull insights and decisions out. So through this process, we were able to set up a system in several CCP countries through the Breakthrough Action Country project, excuse me. And we prepared guidance documents and a DHIs to metadata package to share with our own teams but also made freely available. It's not a new app, it's just the metadata for the event forum program rules indicators, user groups and updating it with the tracker program as well. So I'm not a developer, I'm just an eminy person learning DHIs too as I go. So shout out to all the non-developers here. But I hope this is something that we can kind of move forward where when we come up with something we share it with one another, try it out and work together to save ourselves time and improve the work we're doing through DHIs too. A very big thanks to my colleague Abdul Doso who has co-designed the system and to Marla Shavits who's on the call our director of digital strategies at CCP who encouraged me to share this approach more widely. And really all my colleagues in different countries who are adapting this approach and sharing what they learn. So thank you for your time and I'm looking forward to continue growing the DHIs two skills this week with you all. Thank you Natalie, that was a great presentation. You're welcome to go ahead and share your screen at this time. I just found this so fascinating. The differences between the event-based surveillance and the rumor monitoring, but between Emma's and Natalie's presentation still getting to this point of how can we use DHIs too to try to structure what is unstructured data? So with this, I'm very happy to introduce from PSI Vietnam, Huan Nguyen and Phuong La who are going to share their use case around strengthening event-based surveillance through chatbots and integration with provincial EOCs. So the floor is yours, Huan Phuong. Hello, I'm Phuong from PSI Vietnam. Can you see my screen, Rebecca? We can, if you'd like to try it in presentation view, but I can see the screen, yes. Okay, yes, okay. Then this one second, okay. How about now? Perfect, yes, thank you. Okay, so thank you Rebecca for your introductions and thank you everyone for your interest in our EBS sessions today. I'm Phuong from PSI Vietnam and today we at PSI Vietnam would like to share with you our experience in integrating a social media chatbot with the ADIS-2 for event-based surveillance by private sector. So let me just give you a little bit of context why we built this system. In Vietnam, EBS was first introduced in the early 2010s by the Vietnam Ministry of Health and supported by the United States Center of Easy Control and Prevention and the World Health Organization. However, it just started to expand into all provinces in 2019. Since the start of the COVID-19 pandemic, popular COVID-19 event-based surveillance signals in Vietnam mainly came from official quarantine units at point of entry, media monitoring, such as social listening software and individual health declaration either through a website or a paper base or a mobile phone app. And we also had a close contact exposure history app. However, these signals mainly based on reports collected and submitted by the public sector. Private health outlets hardly participate in the existing surveillance system, although they are actually one of the first points of contact for people seeking care when having mild symptoms such as fever or cough. These private outlets, especially pharmacy, don't really have a reporting habit or belong to an extensive network to report possible cases. And there are a limit of information on how EBS signals are consolidated, analyzed, or shared among stakeholders such as local health authorities at different levels. In this context, PSI Vietnam developed a digital solution. It was all started with simple idea. It was started with a malaria case reporting chatbot integrated with the ESIS-2 that we had already used. And in August 2020, using the lesson learns from this malaria chatbot, we developed a similar chatbot for reporting possible event cases by private outlets in the context of COVID-19. And on the right side, you can see an interface of this chatbot in Vietnam. We are using a local messenger app called Zalo, which is the most popular platform and provided in the Vietnamese language. It is, this chatbot uses automated conversations to prompt questions for health authority, health outlets to answer. Therefore, they can enter necessary data for event-based surveillance into the ESIS-2. The data that are collected through this chatbot is stored and analyzed on the ESIS-2, then presented on various dashboards. As you can see on the left side, this is an example of a map showing cluster of possible cases. The health outlets are trained on site by PSI staff on how to report possible cases. With this chatbot, we would like, we want to encourage private health outlets to submit possible event reports, build network for reporting, therefore collect and analyze data for tracking trends and support local health authorities in event-based surveillance for COVID-19. From there, we expect that the local health authorities would conduct appropriate intervention. After nine months of implementations, I think the results are quite promising. Around 3,000 private health outlets are involved with more than 300 clinics and 2,500 pharmacies in five project provinces. Our current coverage includes areas with high population densities and industrial zone. Around 74,000 possible cases and events with adults, one common symptoms of COVID-19, either it was cough, fever, sore throat, or difficult breathing, were reported by 61% private outlets. 65% of outlets report that they submit that the events reported within 24 hours. And the most common symptoms, most common reported symptoms were cough, fever, and sore throat. Among these events submitted through this chatbot, 4.6% of them can be seen as possible COVID-19 cases, meaning that they have three COVID-like symptoms, fever, cough, and difficult breathing at the same time. These symptoms frequencies are visualized on the SIS-2, to display on a heat map and also other graphs and individual event reports to see if, to display whether there are any cluster in a period of time. This is expected to help local health authorities to monitor possible health issues and to investigate whether something is really going on in a particular area. Now I'd like to invite my colleague, Hua, to share about our collaborations with the health authorities and other lesson loans in the process. Hua. Yeah, thanks, Hua. Can you hear me? Yeah, we can hear you. Okay, thanks, Hua. Next slide, please. Next slide, please. And as mentioned above, as mentioned above, EPS currently expands in the whole country since 2019. And following by decision of Ministry of Health in 2018, EPS procedure has five steps, including signal capture, signal triad, verification, assessment, and intervention. And EPS currently a BSI and for private sector, we just stop at step one and two. We mean that just all is a data collection and only local health authorities take responsibility for signal verification to verify the signal to make sure that they become an event or not. BSI Vietnam collaborates closely with the Central DC Control CDC, a provincial level, and District Health Center for the remaining step of EPS. District Health Center and Provincial CDC they receive a training from BSI and then they had an account to accept into the DNS to dashboard the COVID surveillance of their area. District Health Center takes part of signal verification and event assessment and CDC approves the level they oversee and manage data of their province and propose response to events according to the District Health Center feedback. And as you may see at this slide, on the right side, some important data are demonstrated to dashboard like the cluster and the alert distribution with high number of customers who have a fever. And on your left hand, this is the number of the participants and the local health authorities they can see analysis by day, month, by gender and the chance of the event. In addition to evaluate the sustainability of chatbot, next slide please. BSI Vietnam conducts a phone survey to evaluate user readiness and acceptability for reporting the signal. As you can see here, 65% of our lab participating in surveys submit the report via chatbot within 24 hours. 81% user were willing to keep using chatbot in reporting submitted people of COVID-19 and among those who use both chatbot and individual health declaration, the one that my colleague mentioned at second slide, more than a half of respondents referred to user chatbot rather than medical health declaration because it is easier, faster and more useful. And they are the reason why approximately 71%, our lab rate high score for chatbot registration and utilization. After nine months of implementation, we got some valuable lesson learned. As my co-worker mentioned at the first of our contact that the private sector did not have the habit to submit the report. So to maintain the EPS for private sector network, we need to motivate, encourage the private sector, make them be a part of the EPS and this can be done with the support from the local authority. They can share the information with the private sector by inviting them to the training or the email, record the activity and their contribution in EPS by giving them the certificate, for example. Again, BSI Vietnam and private sector network will take only the first two steps of EPS. Direct under the CDC, Department of Health and District Health Accenture. So a collaboration among the BSI in particularly private sector with local health authorities is very important and necessary. On technical side, we still have some chatbot with the chatbot even with the DRDS too. The chatbot system would change without any notification and it makes the interaction in reporting case. We need a good internet connection to make the flow of questions as smoothly and faster. And sometimes not all providers have time to wait for the next question and understand the issue so they can forget, they may forget to complete the flow or do not want to use anymore because they think it's confused, it takes time. And for aggregate data, it's limited compared to case-based event data because for the chatbot, we can submit the report case by case or we can submit the report by month. So sometimes it's hard to combine the data. After all the lessons learned, yes, at Vietnam, we realize that the local health authorities play an important role and they are the key person in the BPS. So BSI Vietnam, we are currently building the capacity, supporting and developing the network, the private sector network to support our local bundle by giving them the training and the workshop to look at how authority increasingly involves in supporting private sector to support the surveillance data which is advantage of chatbot. We consider to apply this idea for other infectious disease and expand in other areas. With the use of the chatbot, we can have the real time and good data, capture a system and then work. It's gonna support a lot to detect and recall an easily held event in future. And the other thing is very important is the chatbot. As my co-worker mentioned, currently we're using the chatbot as a low platform but we can apply this to other platform like Vibeau or Messenger. And to maintain the chatbot, we, I mean that the local health authority needs to have a long-term contract or closer collaboration with the owner of the chatbot of the platform to take advantage of the chatbot. And BSI we did do a similar work in Laos and Myanmar for the reporting of malaria case and fever case and for the TV by using the chatbot Messenger, Vibeau and WhatsApp platform. There are many things that we want to present but that all we can share for now. If you want to ask for more detail or have any questions from some, please feel free to email us or give us your question at the chat conversation. Thanks for your listening and attention. Thank you Hua and Fuang and Natalie and Emma. These were just absolutely fantastic presentations. So I encourage everyone to please, please do visit the community of practice posts. They're in the channel right now. I already see that there's a lot of activity and there are some questions happening and unfortunately we don't have time for live Q&A. So I'll just close with a couple of reflections around what an amazing use case this is, a set of use cases for the community that really brings together data points, non-traditional data points for DHIs too as an HMIS from private providers, from community listeners, from community leaders. I've seen all of these use cases are really focusing on understanding these complex data flows but they're also quite driven by a real-time analysis of this unstructured data and trying to bring meaning to it and that meaning is consistently tied with actions which I just find incredibly impressive as we look to how data use is actually driving our decisions. So the last thing is just also this ability to sustain the motivation of these sort of non-traditional reporters and try to do this through channels that are accessible. So SMS and WhatsApp and really looking at DHIs too as a platform for integration of data but also with other systems and communication tools and social media apps that are out there. So I congratulate our presenters today. Please do follow up on the community of practice. And I think with this, we can formally close the session and I don't know if Max has any announcements but thank you very much.