 I'll be short. I'll just introduce our three speakers for this session. We have 45 minutes, meaning about 10 minutes for each and some time for discussion. We'll start with Celestin Hatageka from the Harvard School of Public Health, giving a presentation on the impact of the outbreak of Ebola and the free-care policy from Eastern DRC. Then we have Karen Grepien, Associate Professor at the University of Hong Kong, giving a presentation on the early impact of COVID-19 outbreak, also from DRC, or based in DRC. And then the last presenter, Angela Aaba from Affinets, looking at routine immunization data on DHS-2 in the context of COVID-19 from Nigeria. So the topic of this session is, you know, looking at data and service data in DHS to look at the impact of Ebola and COVID. So basically using the data on other services to evaluate what the current and previous pandemics do to the health service delivery. So without talking more, I'll give the word to Celestin. I'll stop sharing my screen and Celestin can share his presentation. Thank you. Thank you, John. Let me go ahead and share my screen. Hello, everyone. It's a great pleasure to be here and be presenting our work on the impact of the COVID outbreak, sorry, of the Ebola outbreaks and the free health care policy on the health service utilization in the Eastern DRC. Before I elaborate further on our work, I would like to provide a brief background to help establish the context of our work. Ebola was discovered in 1976 in the northern part of the DRC in the Mongora province. And since then, there have been several outbreaks including the current outbreak, the ongoing outbreak in the equator, and the two other outbreaks that took place in the 2018 one in the equator, and another one in the Eastern DRC. The outbreak in the Eastern DRC affected the free provinces. Those are Ituri, North Kivo, and South Kivo. Overall, South Kivo had a small outbreak which affected one health zone, whereas the outbreaks in Ituri and North Kivo were very large, and they were actually considered as the fourth largest outbreak that the country has experienced since the discovery of Ebola. So that's why we focus on this to understand the impact of the outbreaks and the policies implemented to mitigate its effect. So we know from the evidence in Western Africa during the 2014 outbreaks that Ebola outbreaks resulted in a significant drop in the service utilization. To mitigate this effect in Congo, the DRC government and the Arangüedizi partners introduced a free health care policy in some regions affected by Ebola. However, the regions did not benefit. So this provided us an opportunity to study the impact of the Ebola, as well as the free health care policy and service utilization at the same time. We leveraged data from DHIS to enable health information, health management information system in the DRC. Specifically, we use the data from health centers as we focus on looking on the impact on primary care services that are provided primarily by health centers. These services included total clinical visits, we looked at common infectious diseases including diarrhea and pneumonia. We also examined the impact of the outbreak and free health care policy using maternal health services including antenatal care and the delivery care. We looked at the impact of vaccinations. Given the HMIS data, the time series data available monthly, so we were able to design a strong study using interrupted time series with a control group. The intervention group on one hand has the region that had Ebola and free health care policy and then the regions that were not affected by Ebola and had no free health care policy were used as our control group. We also on the other hand looked at the effect of Ebola and the region with Ebola without health care policy or still using the region without Ebola and free health care policies as our control group. Although we use a control interrupted time series, a single interrupted time series can also be used when the control is not available as you will see in the following presentation by Professor Karen Grepin. When you're using these methods, the control interrupted time series, you're comparing changes over time before and after the intervention in the exposes versus the control group. We use specifically segmented regression approach to estimate the change following the outbreak and the free health care policy. What did we find? As you can see on these monthly time series of clinical visits, we found that immediately following the introduction of free health care policy in the health zone affected by Ebola, the clinical visits are more than doubled. This gain, as you can see, remained throughout the outbreaks relative to our control group, which is the orange time series here. However, when you look at the region with the Ebola without free health care policy, there was no immediate increase in the service use from the outbreak. However, service utilization increased over time at a rate of about 3% per month. This could be explained by increasing the investment in those areas to help fight the outbreak. We examine an effect on a common infectious disease, specifically the diarrhea diagnosis, as you can see from this chart, the introduction of free health care policy did increase in the diarrhea diagnosis, the rate more than doubled, and the gain, more or less, decreased over time at a rate of about 6% per month, relative to what you would have expected without the Ebola and free health care policy. But if you look on the region without free health care policy with Ebola, there was no significant changes. So looking at the range rates of pneumonia diagnosis, you can see, similar to what we observed in the diarrhea diagnosis, that the free health care policy deserted a significant increase in the service utilization reflected in the pneumonia diagnosis here, and the gain really remained during the outbreaks. However, in the region without free health care policy, but with Ebola, there was no changes observed relative to the control group. So when we examined the effect on health service utilization, specifically looking on antenatal care first visit, you can see there was a modest increase in the service utilization as expressed as an antenatal care visit. So about 17% immediately flowing the introduction of free health care policy in the Ebola region. Ebola health zones are relative to the control group, but there was no significant changes in the regions with Ebola, but without free health care policy. When we looked at facility delivery rates over time, we observed that when the free health care policy was introduced, as you can see here in August 2018 in the Ebola affected region, in the month following the introduction, the facility delivery rate increased by about half immediately, and then it continued to increase at a rate of about four per set per month when the free health care policy was still there. However, when the policy was discontinued in early 2019, the facility delivery rate immediately dropped by about 40%, and then it continued to drop at a rate of about 16% per month subsequently compared to what you would have expected without the drop in the service, the continuation of the free health care policy. This really showed how the free health care policy was effectively increasing the use of facility delivery. When we examined the vaccination rate, we observed there was no significant changes across regions with free health care policy or those without free health care policy compared to our control group. In short, our analysis really showed that free health care policy can be an effective tool to help mitigate the impact of infectious objects such as Ebola, and then we showed that it was associated with increasing services in Ebola affected regions, and its effect was largely sustained during the period the policy was still there. We were able to do this analysis thanks to a variability of DHIS to enable the routine health facility information data. So this data can really be helpful to examine the impact of outbreaks such as this Ebola, but also to really study the impact of policies implemented to mitigate the effects of these outbreaks in resource-limited settings. So I would like to acknowledge IDRC, which funded this research through a grant to Professor Karen Grape, and I would also like to acknowledge Bruce Quayer that uses his platform to provide us access to data using this analysis. Lastly, special thanks goes to our amazing study team here. Thank you. Thank you for joining me today on my presentation. Thank you very much, Celestine. Then I can ask Karen to share her screen. You have to share first, I think, Celestine. And while Karen is setting up again, I'll remind you that you can post your questions on the COP link provided in the sketch page for this session. And then we have some time at the end of presentations to respond to them. So Karen, please, you can start. Hi, everyone. I hope you can hear me. I turned on my video. I'm not sure if that means you can see me or you can just see my slides. But thanks so much for coming to the session tonight. My job is actually quite easy because this work actually builds quite a bit on the work that Celestine just presented. So as a result, the hard work is done and I'm going to mostly focus in on some of the results. So we're looking at a very similar kind of question and a similar kind of context, although now we've switched from Ebola to COVID. As Celestine has mentioned, there have now been three separate outbreaks of Ebola since 2018 in the DRC. And now, of course, they're dealing with COVID as well. So in early 2018, when the first outbreak, so this would be the first equity outbreak came about, we set up a research project to basically understand what we might call the sort of secondary health effects or quote unquote, the collateral damage of COVID. And the reason for this is that in the West African outbreak of Ebola, in the sort of post-mortem that was done after the outbreak had reached its peak, it was very clear that there had been very substantial declines in the use of health services in the very heavily affected countries that had Ebola. And this was across the board. It was everything from maternal and child health to immunizations. And the belief is that perhaps as many people died or had significantly more morbidity because of this lack of use of health services. And there's lots of reasons why that might have happened. This could be happening both on the demand side and that people, for example, are nervous or scared about presenting at health facilities when there's cases of Ebola. Or it could be on the supply side and that the health system itself was greatly undermined. There was a lot of issues around human resources. And so for a variety of reasons, we should expect there to be pretty significant declines in utilization. So when COVID happened, the fears emerged as well that this would happen again. And because of the work that we had been doing since 2018 in the DRC, it was quite easy to then sort of pivot what we were doing to say, great, now what's the impact of COVID on the use of health services in the DRC? Modeling studies that have come out since earlier this year have really predicted quite dire... Quite dire sort of consequences of what's been happening. I'm not sure, does everyone hear the interpreter? I'm hearing it. I'm hearing it too. Just a second, we need to adjust the settings. My apologies. That's cool though. I don't think I've ever heard anything I've ever said in Russian before, so that's quite neat. Anyways, I'll continue. So the pandemic in the DRC has obviously been quite unique. So the first cases of COVID appeared in the country sometime in early March. The initial detective cases were amongst travelers who were coming back predominantly from France and parts of Europe. They quickly mobilized to try to mitigate the impact of the pandemic on the country. And so like many countries around the world, public health measures aimed at trying to contain the virus. So for example, the closer of schools and bars and at the end of mass gatherings, they also quite severely shut down the amount of flights going in and out of the country as well as within the country. Big country and flights are an important way of getting around the country. And then because a lot of the early cases were really heavily concentrated in Kinshasa, the capital city, there was a lot of additional focus in terms of trying to contain the outbreak in that city. So for example, we see, actually travel in and out of Kinshasa itself being restricted around the end of March. And then the one neighborhood, which had one of the most of the cases, at least at the time was sort of appeared to be having a lot of the cases, was basically put into very strict lockdown. And this neighborhood is called Gombe. They are effectively in lockdown through all of April through the end of June. So this is a map of Kinshasa. It is one of the largest cities in the world. It's one of the most densely populated cities in the world. There's 12 million people living in this city. You can see here on the map, Gombe is one neighborhood. It is actually one of the more affluent and centrally located neighborhoods in the city. And it was where the outbreak was heavily concentrated. In the DRC, the health system is organized into what are called health zones. Each health zone would have a certain number of health facilities, including usually some sort of a hospital and a series of health centers. There's 35 that we use in our analysis in the Kinshasa area. And because Kinshasa basically was affected differently from the rest of Kinshasa, we've thought to also try to see if there was differential impacts of the outbreak in Gombe itself. So in terms of data for our research project, we use the same data that Celestone just described. So this would be the monthly DHIS-2 data from the national HMIS system. We use the data from January through June of 2020 for this analysis. We had a number of choices in terms of which kinds of indicators we wanted to use. Obviously, there's a lot available in the DHIS-2. We try to focus on indicators for which represent a vast majority of services available and also had high reporting rates. But we also try to get a comprehensive picture of what was happening within the health sector. So some that perhaps didn't have the best reporting rates, but we wanted to be able to target, for example, the non-communicable diseases as well. So a similar kind of approach to what we did in the Eastern DRC. We use interpretive time series. Here we didn't have a control group because everyone is, every area was effectively affected by COVID. And so we're really comparing districts to themselves in terms of being their own basic control group. This approach is great in that it also allows us to control for things like seasonality. We know there's a lot of seasonality in things like malaria. But also, for example, and this is a common problem I think in many countries that are just starting to implement DHIS-2. There's increases in reporting rates and increases in data quality happening in these areas over time. And so if we think that's happening in a reasonably constant way, we can also try to control for some of that. Another challenge we had in Kinshasa is there was actually quite a localized outbreak of pneumonia in December and February of this year, basically. And so we wanted to account for that as well. So what do we find? So first off, if we look at total clinic visits and look at this sort of before and after COVID, again, it may not seem obvious because you have to account for all the seasonality and account for these different changes that are happening in that spike that you see there related to pneumonia. But once we do that, we do actually detect a decline in total use of health services, which is perhaps not terribly surprising. It was on the order of about 20%. What's also great about the DHIS-2 data is it allows us to dig deeper. And so that's effectively what we did. And so if we disaggregate the areas in Kinshasa, if we look at Gombe, relative to the other districts, here we see there was a very dramatic decline in use of health services, specifically in the Gombe district. In fact, much of the decline that happened overall in Kinshasa is really what happened in Gombe. And so it's important to be able to disentangle those things. And if we look by facility type, we see that most of the changes actually happened in hospitals. And this would be perhaps consistent with the story of people being, you know, scared to go to hospitals because that's where they know a lot of COVID patients are being treated. We can also look at different types of health conditions and we did that. So for example, we looked at diarrhea. We also looked at malaria. Again, we see these important declines and most of them being targeted focused mainly in the Gombe area. Again, the area that was most heavily affected by the lockdown measures. When we looked at the NCDs, we had data on diabetes and hypertension. Here we saw very dramatic declines in the use of health services in, again, in the Gombe area, not so much in the rest of the area. So just to summarize, we measured a basically a 20% drop in all clinic visits immediately following the onset of the outbreak or the pandemic. And really this is really heavily focused in on the Gombe area. So an 80% decline in total visits there, and only 15% of everyone else. Rates of malaria, again, they vary by disease type and by area, but on the order of 90 to 80% for some of these in the Gombe region. Drops of non-communicable diseases, which is quite interesting. Actually, something that doesn't typically get measured in a lot of, for example, household surveys. We saw really dramatic drops and that's a really interesting finding. Perhaps people are pushing off those types of health services because they're seen as sort of less urgent. But overall, I think that the main finding from this work is that it's DHIs to data on these platforms and provide basically data to generate these types of results in almost real time. Obviously, there is delays in reporting and delays in us getting around to doing the analysis, but these data are roughly available and the work can be done. So again, just to thank the research team that has been contributing to this project over the last couple of months, as well as the Ministry of Public Health, who has been very supportive UNICEF and Blue Square as well. And again, this research is funded by IDRC in Canada from an actual old Ebola grant that we were able to pivot towards COVID. I put my email address up there if anyone has any questions about doing this kind of research in your own context, your own settings. We'd love to talk to you. So please feel free to reach out. That makes sense. And I'll stop there. Thank you very much, Karen. That's very interesting. So we have the next speaker, Angela Abba. And then we will have time for questions at the end. Okay, we see your slides now, Angela. Thank you. Good afternoon, everyone. I'm Angela Abba. I work with Affidavit Nigeria. And I'm making this presentation on behalf of other co-authors. So this presentation basically is looking at the humanization data reporting on HRS2 in the context of COVID-19 in Lagos State, Nigeria. All right, so this will be the outline of the presentation. We look at the introduction. We look at methods of what we are finding, the results, what are the key messages of management and diagnosis. By way of introduction, we'd just like to reiterate that HRS2 is the only government that recognize electronic reporting platform for health data in Nigeria. So it means that the humanization data is also reported on the HRS2 system. All states within Nigeria actually report their health data including the humanization on the HRS2 system, inclusive of the legal states. So in February 27, 2020, the first COVID case was actually recorded in Nigeria and in Lagos State. And since then, Lagos State has remained the epicenter of the disease in Nigeria, it's bearing about 3% of the body of the disease in Nigeria, even as a substitute. So the emergence of COVID-19 in Lagos State and Nigeria generally posed challenges to humanization services, beginning from care givers not aware that routine services were provided at the health facility and also people scared of going to the health facility for fear of contracting COVID-19. So therefore this depth review was basically just going to see what reporting was like within the context of COVID-19 and especially for legal states, it was the epicenter of the disease. So basically, we looked at four indicators that are monitored in country through humanization in the country. Before the cases are one, we looked at reporting. Basically we have four data sets in Nigeria that we use for reporting humanization data. We looked at the four of them, what was reporting like. We looked at sessions conducted for immunization. We looked at the next session and there's an active session. So we looked at both types of sessions. We also looked at coverage for some selected antigens and then we looked at supported supervision. So for this particular depth review, we did six months with prospective data analysis. We looked at period between January to June 2019, versus the same period January to June in 2020 and we did trend analysis. The map you're looking at is showing location of legal states in Nigeria. So these were some of the findings that we have following the depth review that we conducted. Number one, like I said, we looked at data sets reporting and we have four data sets in Nigeria. So this is showing the first data set, which is the EDGMI S2013 data set. We did observe that across all the periods, we had higher reports in 2019 compared to the same periods in 2019, 2020. However, we now see a different trend between May and June 2020. Reports from the EDGMI S2013 and we could see that between May and June 2020, despite COVID-19, reporting was much better compared to 2019. So this graph, this chart is showing us the second data set, which is the vaccine utilization data set. And this data set particularly reports issues around vaccines collected and vaccine usage. Again, we compared January to June 2019 against January to June 2020. And we could see from the trend that unlike the previous data set that we had looked at, reporting was much higher in 2019 compared to the same period in 2020. And we did notice that the period of March had a huge decline in 2020 compared to the same period in March 2019. And of course this could be due to lockdown of all the states during the COVID-19 period. Again, we have two other data sets. So we have a micro plan where the sessions that are planned and sessions are conducted, actually reported. And then we have the EDGMI S2013, which was an agenda document introduced after the routine immunization module was introduced to the CHIS set. So for the micro plan and NHMI S2013, we observed similar trends. Of course, reporting was much higher in 2019 compared to the same period in 2020. And just like the south of the vaccine utilization, we also noticed that the differences were highest in March 2020. And like I said, that was because the lockdown was actually, the directive for lockdown in the states was given in March 2020. Moving over to the next indicator, the next indicator that we looked at was coverage for selected antigens. And the antigens that we selected were basically ECG, then the three, ITD, measles, and yellow fever. So across all the antigens, we did notice that coverage was higher in 2019 compared to the same period in 2020. And looking at each of these antigens, you will see that the difference was more in April across most of the antigens. Of course, March, April were more of a lockdown period to expect that these people were not surprised about them, some of these findings that we saw. So across all the road periods for each of these antigens, the coverage was much more higher in 2019 compared to the same period in 2020. Number three indicator that we looked at was the sessions that we conducted. And like I said, for routine immunization, the two types of sessions physically, we have a fixed section and then we have the active sessions. So we looked at a number of sections that we conducted in 2019 and then in 2020. And of course, again, we observed that there were more sessions conducted in 2019 compared to 2020 for all the fixed and active sessions. So we could see looking at the active sessions by the right-hand side, we could see that there was really a decline in sessions conducted, especially for the most of people. Because at some point, the conduct of active sessions were discouraged in March, which was a curtain of the spread of COVID-19. We noticed that there was a 5% and 20% difference for fixed sessions and active sessions that were conducted within at least two years between 2019. However, we did notice that in June, sessions had picked up and we will see that we are having more sessions in 2019 now compared to June 2020 for both fixed and active sessions. And then finally, the last indicator that we looked at was the conduct of supervision. This is too usually. I encourage you country from the higher level to a lower level. And then we looked at this is that we conducted between 2019 and 2020. And of course, we did observe that there were more visits in 2019 compared to this. Overall, it was a 10% difference between on the set of visits that were conducted. So, in conclusion, we're both on the DHISP platform was observed in the lower end of this meeting compared to this year in 2019 in legal states. Of course, this could have been due to the COVID-19 pandemic contributing that there were restrictions on women, contributing that the states also would get sent out of the disease. However, we do acknowledge that other factors including funding and infrastructure that could have also contributed to some of the outcomes that we saw on the platform. So we want to acknowledge the current organizations in the U.S. and this is for June and after that, thank you very much. Thank you, Angela. And thank you also to Karen and Celestine. We have 10 minutes, more or less, for questions and I see there are at least one question on the community practice. I'll share the screen with you all. So we have, can you see this? I hope so. We have one question here from Kelly Hedberg. I guess that goes to all presenters now. The tools and models you're using. How can they be implemented in something like DHS2 or how can we as a community also make sure that these results and methods find a way back to managers and other users of the system? This is Karen. I can try to take that one. So in the DRC, the technical secretariat that is responding to GoFake has, you know, they have these dashboards that they're using to track and monitor. One of the things that we've been trying to do is to the extent possible try to move some of the analysis that we've been able to do onto that platform so that it might be able to feed in in real time to some of these dashboards. What we're doing is not particularly complicated, right? So it's relatively algorithmic and it could actually be integrated to some extent into some of these platforms. At the same time, you know, every outbreak is a little different as outbreaks evolve, you know, there's needs to change. You know, with Celestin's first presentations, we actually ended up spending a lot of time studying a policy change that happened. And as in the second case, we're looking really at how the outbreak is spreading across the country. So it would require some manual kind of looking through and finding out, you know, what's what's interesting to look at. For the most part, there's there's nothing that we've done that cannot be integrated into some sort of a platform that feeds directly off of the DHIs to platform. Okay. Would you like to add something Celestin or Angela? It's not, I have also a question. And it was mentioned at least by Karen about data quality being an issue. And can you say something about how you dealt with data quality or whether, you know, what were the data quality issues here without going into too much detail on how you solve this or something. But how do you deal with the data quality issues for this kind of research? I can jump in to try to answer this one. So the first one, I guess, Professor Karen has said most of the things that we've been doing in Congo, and then how the Ministry of Health and HMIS are also being involved in that. So in terms of assessing data quality, so we've tried various approaches to try to select indicators that are of a good quality as a current discussion or presentation. So one of the approaches we use first is to select to select indicators that are being reported regularly over time and it has a higher reporting rate for those specific indicators and we then consider them. And then we also look at the facilities. So there are some of the facilities that do not report consistently, because one of the things we wanted to do is not including what I would call body facilities, facilities that do not report on particular indicators. So we do consider indicators that have a very good reporting and then we also in for those indicators, facilities that do not report consistently, we drop them. So they were missing data is very common in this time city data, which, you know, range, which is has a, you know, have a period of two to three years, and there is an outbreak coming the reporting rate may drop or there are a lot of missing data. So we use the various approaches to also a roll some inclusion of indicators with the miss with the missing data, but using as acceptable approach to sort of account that in our analysis to have estimated that are not biased. I think that's about what we did on a indicators. Oh, I forgot one point. It's on the outliers. They also, we also consider the outliers. So our eyes were common in many of the indicators we use. So one of our approach we realized was to consider indicators. We are out where they are when the outliers to to to consider those outliers as a missing and then you treat them as a missing value. So the data quality has been improving. So however, in some context, you realize that there are a lot of missing outliers and missing. So if they were more consecutive missing data or outliers and then those indicators were excluded from our analysis. Back to you. Yeah. Thank you. Anyone else wants to add to that. All right, thank you very much. So I'm just to say that the response to the question. So, number one is that based on the analysis that we have gone so far, of course, we could see that there's so many measures that have been put in by the government to ensure that. And we could see that there are really senses have picked up, especially between May and June of 2020 and we're going to see much more better performance in this period compared to what we had in in in 2019. There are a lot of measures that have been put in place, communication, risk of rendition is ongoing and people have realized that services have been provided and services have picked up. And of course, during the analysis, yes, we're going to do this analysis here. We just looked at the first, first, the first six months of the year. We're going to do this eventually across the four months in the year just to see what happened in 2019 and 2020 and it really will be impacted by the nation organization services in the country and especially in the university. Thank you very much. Okay, so that was a good reply to call us the question here. And we still have time for for the last question by Anne Griggs. It is specifically about DRC and availability of data. So it's also a bit about data quality, perhaps. Yeah, maybe, maybe I can take this one. So, as I mentioned in the talk, we initially started this project in Ecuador province, which is, you know, one of the more rural and hard to reach parts of the country. And so we had a lot of concerns that the data wouldn't necessarily be of great quality and widely available. It turns out the data are reasonably good. And in fact, when we moved into the Eastern DRC, we also found somewhere levels of reasonable completeness and availability. In the DRC, you know, there has been a result space, format space financing program. And so a lot of reimbursement and stuff that is being run through that and therefore the DHS2 system plays a role in that process. And so for the most part, the data quality were pretty good. In fact, the data are probably worse in Kinshasa than they are elsewhere in the other areas that we've considered. So definitely is variation across the country that we've been able to find data of sufficient quality to do this analysis in all of these different contexts. Thank you, Karen. Okay, I think we are approaching the end of this session to give you some time to have a short bio break before the last January. So thank you to all participants and a special thanks to our three speakers, Celestin, Karen and Angela. And please you can follow up also with a bit more questions and discussion on the community practice page. Thank you very much.