 Welcome to the Policy Panel on Data Gaps. I'm Kunal Sainth, the Director of UNUIDA. Very early in the time when the pandemic, policymakers had a very difficult choice to make. Should we protect lives or should we protect livelihoods? Predicting lives meant bringing on, bringing a lockdown and trying to spread the virus. Predicting livelihoods meant trying to avoid lockdown as much as far as possible and trying to protect and make sure that the poor people, especially an involved sector could go and out and work. Now, this particular policy choice has been the defining policy choice even now, because you've seen various of the virus coming along, second waves and third waves of the virus even this year. In a way, what was surprising is that the data behind this choice and the evidence based behind this choice was very limited really, on lives because we didn't really have vital registration systems to report deaths properly. There was significant undercounting of COVID-19 motilities either direct because of the virus or indirectly. On livelihoods, what we really saw last year and this year is household surveys, labor force surveys, consumption surveys of any kind, pretty much stopped or were postponed. In that sense, the evidence based making this very difficult policy choice that developing country governments had to face was very much, much more limited and this is the reason why we're having this policy panel today. Because we want to discuss this exactly this kind of issues. What are the data gaps that are there around counting lines lost? What are the data gaps that are there and understanding the impact of the pandemic on livelihoods? Having at the same time, one should also remember in this very dark clouds around data that we've seen in the last two years, there are some silver linings as well. What are they? We've seen fairly innovative methods being used fairly quickly, I would say, on trying to collect data. For example, phone surveys. We've seen that they may not be representative as normal household surveys or labor force surveys, but they have come along and they come along very quickly. We've seen many papers in this conference and in the conference that wider co-hosted the jobs in the conference last week, the papers were essentially presenting findings based on phone surveys. That's an innovation that probably wouldn't have been thought about two years back before the pandemic hit. Now, I should also mention that I see this panel as a mark of respect for the late Beno Undulu because Beno was a former chair at the wider board, was very concerned about this issue. We kept on talking about this until it was untimely passing away early this year. I see this panel in memory of Beno because this is something that many of us who have worked on their big countries were very concerned about. Also, this is something that wider has always been historically interested in and invested in on data. This policy panel brings together our interest in data and this really big challenge we're facing right now on data gaps around lives lost and livelihoods lost. I want to introduce this panel. We have a really remarkable set of panelists here, leading experts drawn from national disaster agencies, international agents in academia. We'll discuss the challenges on data collection in the pandemic and then the implications for the future, including possibly some innovative methods that we've learned in this last couple of years. Let me introduce the panelists in order they will present and I'll ask them to speak. First, we're really pleased I have to say that we have on the leading demographers working on COVID-19 mortality in the Global South. Tom Moultrie is a professor of demography and director in the Center for Actual Research at the University of Cape Town. Among Tom's long-standing research interests are the political sociology of official statistics and improving refining the suite of indirect methods for demographic estimation. Tom, thanks so much for being this panel. Then we have a change in the panelists from the Kenyan National Bureau of Statistics. We have Paul Kenboy-Samue and Paul, welcome. Let me introduce Paul. Paul has over two acres of experience in the Kenyan National Bureau of Statistics. He's the head of research at the Bureau and Paul has worked in survey designs, data collection, processing analysis with strong bias towards poverty and inequality. Really pleased to have Paul in this panel. Moving on to my colleague, Patricia Justino. Patricia is a development economist who works at the interface between developing economics and political science. She's a leading expert on political violence and development. And the co-founder and co-director, the house is in conflict network. She's currently a senior research fellow in UNEWIDER and professional fellow at the Institute of Development Studies, UK, on leave. When we have Lauren Harrison, Lauren, welcome. Lauren is a policy analyst and team lead in the data systems team in the publishing statistics for development in the 21st century, Paris 21 in short. We all know Paris 21. So that's easier to remember on the abbreviation. Finally, but not least, Samuel Anim. Samuel is government statistician at the government statistical Ghanas Institute of Service. Apologies for that. He's also professor of economics whose work, the Department of Economics is Cape Coast Ghana. I should also mention that Samuel is PhD in the University of Manchester where I have an affiliation with. So we have five excellent panelists. And what we're going to do is I'm going to ask each panelist one question first. I would then ask them to respond to that question for five minutes. And I hope we can stick to time because we do want some time for some time for Q&A. And so I'll go by order of presentation, ask each panelist a question and then we'll have approximately 30 minutes for Q&A. And I would really encourage the audience to send in your questions to us in the panel so we can have a really interactive Q&A session to make this panel very lively. So we're looking forward to that. First question, that's to talk. So start my question to you and this is essential as I mentioned your own work on COVID-19 mortality which has been absolutely innovative and part-breaking. There's that and you know we know a lot about COVID-19 mortality in the developed countries. By now, we have very good work than done in the UK, in Europe, the US and so on. But as we know very little we have very little knowledge about COVID-19 mortality in developing countries. So the question I have for you is what do we know about the true mortality due to COVID-19 in developing countries? And link to that, is there a profile of COVID-19 mortality is different in developing countries than in developed countries? You can respond to that and around five minutes please. Thanks. Thank you very much Canal. Thanks very much to the invite. Greetings to all the attendees joining us around the world. And thank you for a great question. You just said that we should try and keep our answers brief. I could certainly talk for an hour, the full hour of the session to try and attempt to comprehensive answer. But I'm not sure you or my fellow panelists or indeed the audience would appreciate that. So I'm going to try to do my best to answer as succinctly as possible. The answer to your first part of your question, what do we know about the true mortality due to COVID in developing countries is straightforward. Surprisingly little, bordering on almost nothing. The more interesting issue for me, I think relates to why this might be the case. And we need to really step through the reasons why we know so little about COVID mortality in developing countries. Firstly, the testing for COVID is wholly inadequate in almost every developing country. This is in part a measure of the functioning of the healthcare system. Does the system have the resources to fund tests, run tests through a lab, collate the data? We know that there's a very strong correlation between, say, the UNHDI and the log number of tests per million population. And so without testing on the appropriate testing routines, one barely gets to the first base. South Africa, for example, my home country has the third highest number of tests per million in continental Africa. But globally, that third position in Africa ranks 123 in terms of the number of tests done per person around the world. Secondly, the identification and recording of COVID deaths requires a highly functional civil registration and vital statistics system. This requires that deaths are notified not only timeously, but also completely, or at least mostly completely. We know with data from the UN Staff's division that the quality of the civil registration of vital statistics systems in most developing countries is exceedingly poor. And that in most of the developing world, in particular in sub-Saharan Africa, there really is very limited functionality to that CRVS system. In the context of the sustainable development goals and this overarching mantra of no one left behind, this really has to be an area of an intensive and extensive work. And I know that Paris 21 and other organizations are taking this forward. Thirdly, the CRVS, that civil registration vital statistics system, has to be combined with capacity within the health system to identify COVID deaths. In other words, we need a cause of death identification system. And again, without the CRVS, there's no going forward, which explains why Swaziland, the country with the fourth most reported deaths per million in continental Africa, comes in at 68th in the global rankings in terms of identified COVID deaths per million. South Africa, my country, has been fortunate in having a reasonably comprehensive testing strategy and a functioning vital registration system, one of the few in continental Sub-Saharan Africa. And that has almost uniquely allowed us to track the evolution of excess mortality. That's mortality above that, which we had expected pre-COVID in almost real time. With colleagues at the South African Medical Research Council, we're able to release data on a Wednesday evening based on deaths which were notified through our civil registration system through to the previous Saturday. So it's a four-day delay between when the deaths, the last point when deaths are recorded until when we release an estimate of those excess deaths. And since the beginning of the epidemic, we've recorded about a quarter of a million excess natural deaths. So that's in the country where there are about 450,000 natural deaths every year. So an excess of a quarter of a million in 14 months is significant. But the number of officially COVID reported deaths is 83,000. And while not all of the excess deaths are COVID, because some will be collateral deaths arising from repeated overburdening of our healthcare system during the surges and the waves, we think that the vast majority of those excess deaths are indeed COVID or COVID related. In fact, we think it's about 85% to 95% of that quarter of a million excess deaths are attributed to COVID. But we don't actually know that and we will never know that until we get the detailed information from the death certificates and the cause of death, which might take two or three more years before we are able to do so. So at the moment, we are basically grappling in the dark. We have excess deaths, which suggests the actual COVID deaths are being underreported by about two and a half to three times. But we don't actually know what those real numbers are, except we see the evolution of excess deaths tracking what we know with the surges and waves of COVID infections. Your second question, just briefly, says, is the profile of COVID deaths different from that elsewhere? Ties with the myth which has been circulating perennially based on misinformed interpretation of data from COVID aggregators, that somehow Africa is immune to the disease, that it's unaffected. This is simply not true. Yes, the population of Africa is younger, but newer variants, for example, Delta, which have affected the young as much, if not more so than the elderly, certainly are having an impact. We see that in recent data from Kenya, from Namibia. And we have a population in sub-Saharan Africa with extensive comorbidities, particularly diabetes, hypertension, and obesity, which are all co-factors for COVID mortality. The reason why many think that there's no COVID in much of Africa is simply because we do not know where to find the deaths and how to record them. Where countries have been able to do so. We have been able to track those deaths. We've seen it with Peru, which has done a fascinating study amongst developing countries of reconstituting their official estimates relative to what they have now estimated from the death certificate server code. And that is why COVID now tops the list. Things are possible. The work can be done. The resources can be generated. However, the fundamental problem is across most of the developing world really ties not this need to get in a functioning civil registration vital system. Thanks, Kanal. Thank you, Tom. I'm gonna come back to some of the points you raised later on the discussion. That was very, very informative. Thank you. I would move to Paul now. And Paul, the question I had for you is, and specifically in your role in the Kenya National Bureau statistics, what have been the major challenges that you've faced in KNBS and more generally for other satisfy agencies in Africa last year and this year. And what have you tried to do to address this crucial data gaps that have seemed to emerge during the pandemic? So two part question. First, what are the challenges that you faced? And second, what have you tried to do about it if it was possible? Thank you. Thank you very much for the question and thank you for inviting us to participate in this panel. My answer is the first question is the issue of challenges that the Bureau of Statistics has faced. From the emergency of COVID-19 in 2020, it hit Kenya on 12th of March, 2020 and since then there was an immediate measure including the cessation of movement to certain parts of the country, working from home and introduction of cavies. So the first challenge that we faced was the disruption of regular data collection, analysis and report writing in the Bureau. The COVID-19 introduced the lockdowns and therefore we couldn't continue with the data collection. The second challenge that we faced is the timeliness of data dissemination was also disrupted since the report writing workshops had been rescheduled or temporarily suspended. Father, we have the unprecedented demand of data. For example, when the ministry for social protection was looking for the information about the poor households so that they can assist in the social protection targeting. So this information were lacking. We had also budget cuts. As you know, we have to get a lot of money to do a data collection. So this disease introduced the issue of prioritization of the budget and therefore the budget cuts by 50% affected us a lot. We had also the challenge of methodology of data collection from households. That is the data collection had to be switched from computer assisted personal interviews copy to computer assisted telephone interviews copy. There's also, we had also lack of a sampling frame to be used for computer assisted telephone interviews methodology during the switch. We had a problem that we didn't have the phone numbers for those people who we were to conduct. Now, the second part of the question is how do we address this data gap during this pandemic? The bureau has been striving to overcome the shortcomings and the vulnerabilities in data production systems that were brought to light by the pandemic. And these mitigations that were introduced were one we had to switch from collecting data from the field using the copy to cutty and the bureau for the first time utilize the phone interviews. The second one was construction of sample frame for cutting which means the bureau now had to work with other government agencies to get the phone numbers. For the individuals that were to be called or to be interviewed. This means that based on our census 2019 the bureau went for the IDs of these people. So based on the IDs we were able now to combine the IDs with the phone numbers and therefore we will get these people through the phone. Based on the experience of the bureau as incorporated lessons learned from the pandemic to include variables of phone numbers when they are preparing the sample frame. Also this introduced the issue of virtual meetings. So we've been having virtual meetings in everything that we do and therefore this one has disrupted the quality of statistics. Also working in ships has also brought in the issue of quality statistics to come down. Rationalization and prioritization of activities programs due to this the budget cuts the rationalization program was introduced where the key products were prioritized while to overcome the issue of the shortcomings in the budget. Partnership and collaborations was done and the bureau had to reschedule or to work with partnership in order to produce some of the statistical products. The other thing is that the bureau since the social protection required a lot of data to target the poor. The bureau now is working with the social protection using the small area estimation to identify the poor people where they are and therefore keep the database for the national whenever such emergency comes in. We are also working with the citizen generated data to make them official or reporting for SDGs and this we've been working closely with Paris 21 to give us the knowledge on how we can work with this. Then we've been also having capacity building with the big data and other alternatives of data sources. We've been collaborating with ONS of UK to come up with ways of analyzing the big data that is what I can say for now. Thank you. Thank you, Paul. That was very informative and I think in the Q&A That's what I can say for now. Thank you very much. Paul, thanks so much. I should just mention Paul has the video off because of interconnect connectivity problems, but Paul very informative and again, we'll come back to some of the issues you raised about phone surveys and small area estimates and so on. Hello, I've stopped. Thank you very much. Yeah, we can hear you. I've completed the two questions. Thank you very much. We can hear you. I will come back to some of the questions you raised in the Q&A. Now let's move on to Patricia Justino. Patricia being a leading expert on conflict-affected states, particularly where we know with conflict, violent conflict, data collision is always been a challenge, right? I mean, it's always been an issue and you've obviously are very aware of this in your own research. How do you think the pandemic has made things worse in this context? And what does it mean for effective policymaking in this country is for the future? Patricia, all the two. Thank you very much, Kunal, for the invitation to be in this panel. And thank you for all the participants for joining us from many parts of the world. I know many are with us today. So yes, so I've been connecting data in conflict-affected countries for the last 15 years. And it's a challenging, it's probably one of the most challenging environments in the world to do so. And I guess the short answer is the pandemic has made it much more difficult. And we actually had a session on conflict yesterday and we just had a fire chat, fire side chat discussion with Dominic Roon, who's doing a lot of work in his area too. And this comes over and over, all the challenges we face. It's complex, it's dangerous. In some areas, conflict has increased, which makes it even more complicated post-pandemic. Data is available, when available, it's protected by many gatekeepers, including governments. We see a movement towards, in some cases, towards more sort of authoritarianism. And therefore, things are even more difficult to access. We often talking about data that is very sensitive. Even when we access the data, we heavily scrutinize about everything. We have to say, all these challenges are there. They're becoming more difficult. And of course, this has a huge policy implication, because in a lot of these settings, national policies are being done based on very limited evidence. And also for international interventions, as donors focus more and more in conflict-affected countries, as we all know these statistics, the number of the poor is concentrated more and more often in conflict-affected and fragile countries. So therefore, the interest is greater, and so is the need to understand better these issues. But actually, I mean, we know these problems are there, and they're gonna become more difficult, but actually I want to pass on a more positive message here. And the thing is, we have made a great deal of progress over the last decade. And now, actually, we do have a wealth of data from conflict-affected countries that can be made better use of. And it's interesting, I mean, we've seen a wealth of data innovation, which has led to, but also be generated by important theoretical advances in conflict analysis. This is one field where data and theory have gone hand in hand in very interesting ways. And we now have available things like, we can map in much detail conflict events in almost across the entire world, thanks to the efforts of data sets like Akhlet and colleagues at Uppsala University. Of course, there are shortcomings, but it's been done. There's a wealth of information about conflict factors, surveys with ex-combatants and so forth. The consequences of conflict, we know a great deal about the consequences of conflict thanks to efforts to develop household surveys in conflict-affected areas. Some of these are not easy to access, some of these are, again, kept by gatekeepers. So there's a lot of work to be done in terms of compiling, making data accessible, being more transparent about what's out there, but we're making progress. And certainly in terms of quantitative subnational data, I think there's been quite a lot of work. And one interesting thing that I've observed over the last year and a half is that actually the pandemic seems to have forcing everyone to do more originally thinking about how to collect data in the difficult situations. We can't fly, we can't go to the field, we can't access the populations, we can't access our samples. My colleagues in this panel have just gone through all this, discussing all these difficult issues, but actually this is something that conflict researchers face on a daily basis without the pandemic. So we've been thinking very carefully about how to do these things. So I actually think that there might be an opportunity here to innovate and just give you some examples of things that are happening or have been happening in amongst those collecting data in conflict-affected countries that may offer opportunities for others. Is we were discussing and piloting already phone surveys and also methods of collecting surveys using SMS messaging beforehand. The World Bank had, for instance, a large phone survey ongoing in Nigeria in areas controlled by the Boko Horam. And I suspect this may have informed the recent efforts with phone surveys quite well. The use of satellite information has been quite widespread over the last five years. We first started by trying to understand conflict dynamics by looking at things like crop patterns, burn areas and so forth. And then move more recently to using satellite data to trace population movements, especially refugee movements which are very, very difficult to track. And obviously now especially with a fallout from the Afghanistan. From Afghanistan, I mean, this is going to become quite an important issue. Big data is being slowly coming in and being more and more used using social media-based data, information generated by mobile phone GPS locations. I just come across a really interesting project where mobile phone GPS location is being used to understand those that joined the events in the US of January the 6th and tracing exactly who were those present there and it's been used to also in the ongoing legal actions. There's a lot of talk about early warning systems. This has always been there. It's a big topic. You want to understand where the next conflict is going to happen. So there's a lot happening in terms of machine learning models forecasting conflict and it's becoming more sophisticated as the underlying data also improves. Again, also using big data and so forth. There's challenges that remain on things like understanding the long-term effects of conflict and with the pandemic happening, this is going to come up. And challenge is happening because all these innovations have been happening in the last 8, 9 years or so forth. And so we don't really have good ways of going back in time and understanding what was there. Can we kind of take lessons from previous conflicts? We had a paper yesterday looking at the effect of the Ebola pandemic on violent conflicts alongside another paper looking at the effects of COVID-19 on conflict and finding very similar results. So we want to learn from past experiences. There's a big weakness, but there are large improvements now in digitalizing archival data and also accessing intelligent military data as it gets declassified. So lots happening. So I am hoping that actually what the pandemic will offer is an opportunity to do things better and accessing. These are some of the poorest, most vulnerable population in the world also living in areas which sometimes are almost impossible to access. So I am quite hopeful that we can all think more creatively about how to access these populations and understand better these really difficult contexts. So that's me. Thank you, Kanal. Patricia, thanks so much. Actually, you made a very important and really interesting point that as we right now have a situation where many countries, especially the global spot, not only the global south are facing re-challenging data. This is a challenge that you've always faced and others who work with fragile states face for a long time. And so you'd already started thinking about innovations which perhaps others can learn from. That's a really interesting point. And I think we can come back to this later on in the Q&A. Thank you, Patricia. Let me move on to Lauren, Harrison. Lauren, you've already heard now the role of Paris 21 in the remarks that Paul made. So it's a nice way to introduce your work and Paris 21's work. So what do you see is the role of initially international issues like that as Paris 21 improving statistical systems in low-income countries, not just now for them for now, but for the future so that we can have more resilience to future crisis in terms of data or over to you. Thanks so much, Kunal. And to my fellow panelists, it's indeed such a pleasure and really an honor to be able to join you today and talk about one of our favorite things at Paris 21, how to solve the issue of data gaps. So coming to your question, I wanna talk about, when I think about resilience I'm thinking about it in a couple of ways. One is more on the operations piece and that's asking how do we enable and empower countries to become more nimble and responsive in moments of crisis? And the second is more what we might think of as the substantive output piece of how do we enable and empower countries to address key data gaps that ensure no one is left behind in crisis moments. And when we're talking about international initiatives I think that like Paris 21, there's really three ways where I think we play a role. And that's by adapting, assisting and amplifying. So first on adaptation, my colleagues have already mentioned a few points and we're probably all really tired of thinking about how to adapt, we've all had to adapt. But very early on in the pandemic, Paris 21 started thinking about COVID-19 as what we might think of as a dual shock for statistical systems in low and middle income countries because we saw a rapid increase in demand for data and information. And at the same time, a precipitous decline in operational space to collect data and produce information. There are many success stories, several from the colleagues that have already spoken on how NSOs and other agencies have been innovative and adapted under these conditions. A couple of examples from our shop. One is the one that Paul brought up earlier on how we've been working with KNBS to integrate quality criteria for non-traditional data sources like citizen-generated data in their new national quality assurance framework, which is forthcoming, which has been exciting. Another example is in the Philippines where we worked with the Philippine Statistical Research and Training Institute and the Philippine Statistics Authority to train local government units to use really rich granular information when they were capturing in their community-based monitoring systems to close some local data gaps. Both of these examples illustrate not only how international initiatives can continue to play a role in sustaining support during the pandemic, but also on adopting new data sources to close data gaps, which we know is probably part of the solution. It's often that we need practical tools and capacity development to make this really useful and practical for NSOs and other stakeholders in low and middle-income countries. So international initiatives like Paris 21, of course, we've also had to work on adaptation. Both the initiatives I just mentioned have been delivered more or less remotely with very little face-to-face interaction. But since the side of the pandemic, we've also launched two free e-learning courses on communicating gender statistics in partnership with UN Women. We've seen over 1,000 course participants and already have examples of impact from more inclusive land use planning and Trinidad and Tobago to data-driven advocacy on violence against women during the pandemic in Jamaica. And I think that the e-learning courses we found, these have been a really important example of adaptation for a couple of reasons. One is kind of the obvious one, that this is about democratizing and scaling access to training and tools for a wider group of people. But also it speaks substantively to a broader challenge with data gaps that we're observing, which is that it's really not just a production problem, it's also a communication problem. And this speaks a little bit to the bottlenecks that Patricia was talking about. How do you get the data that is there into the hands of the people who can use it and to make more informed decisions? So that's adaptation. Thinking about assistance, it's about supporting countries to meet evolving needs and challenges. I've already mentioned we've adapted modalities for engagement, but international initiatives like Paris 21 play an important role in tailoring guidance and tools to address the challenges that countries face in a changing data landscape, whether that landscape is shifting because of a crisis like COVID-19, or just the evolution of the modern data ecosystem. In the aftermath of the crisis, we released guidelines on how to develop a business continuity plan, for example. So this was meant to complement more medium term planning for like a national strategy for the development of statistics. Statistical planning, I think, is a really important way in which we can think ahead and help countries become more resilient. But we're also looking at how statistical planning can interact more structurally with issues around data gaps and inclusion. For the past three years, we developed and piloted a framework with UN Women to assess gender statistics and mainstream gender and statistical planning. And we've already seen how this process has actually strengthened the responsiveness of NSOs to one example of a key gender data gap, the gender data gap. In Senegal, for example, the NSO leveraged insights from the Gender Statistics Assessment to establish a dedicated pillar related to gender in their new NSDS. In the Maldives, the National Bureau of Statistics is working with the Ministry for Gender, Family and Social Services to establish a new interagency group to coordinate gender statistics. So these examples demonstrate how targeted assistance can help address some of the critical data gaps that we've seen during COVID-19 and ultimately set the stage to close those gender data gaps in the future and build resilience for more inclusive statistical systems and crisis. Third and finally and very briefly on amplifying. And I think that this is for international initiatives like Paris 21, we take really seriously our role in ensuring the needs and experiences of countries are amplified and reflected and communicated back to the wider international community. This is especially critical to address the resource shortfall we often see. Our partner report on support for statistics has reported for a number of years running that financing for core statistical systems like CRVS and others is really stagnant. And yet we also know that the need for data and information is only growing. For this reason, as the secretary for the Bern network, Paris 21 is developing a new clearinghouse on financing development data, which will be an innovative online platform to better match donor supply of financing with information on country needs for financing so that we have a better sense and countries have better representation and a tool that partners can easily use to understand the landscape of those needs better both in moments of crisis and in moments of relative calm. We look forward to launching a prototype for this new tool during the World Data Forum next month. And so those are my kind of three points on why I think international initiatives really can play a role in improving resilience. That's adopting, assisting and amplifying. Thanks so much for your time. Over to you Paul. Hello, Arun. Thanks so much. That's really very useful. And I'm going to come back later on the discussion around some of the issues around citizen led data collection, for example, that was very interesting. Paul mentioned that when you did too, because you might want to think about other ways of collecting data in the standard way that economists collect data. So qualitative data, for example, could be very important for the future. So I'll come back to that maybe in the Q&A. And now we are very pleased to have Samuel Anim. Samuel, there were some technology problems I know at the beginning, but I'm glad you could join us. And great to see you again. And I've already introduced you at the beginning. So let me get to the question directly then. The question I have for you, something that you've already touched on a little bit in the discussion, is the pandemic is going to be smarter and more efficient ways of collecting data? I mean, so we already see examples that Paul mentioned, Lawrence mentioned, Patricia mentioned too, that we had to find ways around this problem that we had. We couldn't really do the kind of data collection we could do before. And so we've seen some really interesting ways of collecting data in the last two years. So first of all, how do you think this is particularly in the case of Ghana and where you will sit as the government statistician, the Ghana Statistical Service? How do you see this in your own work? And also, what can we learn from this? What is what we observe for the future? And also I realize that you are both the producer of data in a sense as the government statistician, but you're also the user of data as a professor of economics in the U.C. Cape Coast. So again, I'm quite curious to see how you see both sides of this question of data quality. Thanks, Sam, go ahead. And thank you very much, Kunal. Indeed, I did have challenges, but thankfully just after your introduction, I got in to listen to all the other panelists. And thank you for all the listeners on this session. Indeed, the issues that have been mentioned by my colleagues do highlight some of the innovations that took place at the height of the pandemic. But what I want to do is to give us some background to the pandemic so that we can have whatever ingenuity that we're talking about during the pandemic in terms of the data that we collected. We all should appreciate the fact that on the heels of the pandemic, we were achieving some good results from the data revolution that started in 2014. And what I mean by good results was that we had quite a number of national statistical agencies in the global south putting their data ecosystems in place, i.e., making sure that their laws are working and also going into good collaborations and partnership that would essentially help them explore the four tenets of the global data revolution. And also as a background, we should also know that at the height of the pandemic, quite a number of countries were preparing for the 2020 round of population and housing sensors. Indeed, in the case of Kenya, they had just finished and released their preliminary results. And in the case of Ghana, we were at the height of the preparation ahead of the sensors taken. So keeping these two things in mind that ahead of the pandemic, these things were happening. It helps us to understand whatever ingenuity that took place during the pandemic. If you carefully look at the data revolution which has four tenets, three of the tenets, i.e., the data landscape, the innovation and the SDGs, four directly in line with the discussion that we are having in terms of data gaps from the production side. So I would touch on the accessibility side, which is the fourth pivot of the global data revolution. So whatever ingenuity that we're thinking about would make, should make reference to these three pillars of the data revolution, i.e., the SDGs that got all of us to continue to produce data in large quantities, the innovations that all of us now on this panel discussion, we're talking about citizens generated data, big data, administrative data. Indeed, these were the things that we were looking at when we had to deal with the over 231 indicators for the SDGs. And as I said earlier on the issue of the data landscape, got us to think about the environment, specifically the legal environment. Also it is important for us to know that the pandemic to a very large extent engendered desperation. And what that means is that quite a number of countries got into interventions without taking into consideration what the data is telling us and moving forward what we should look at from the data perspective. So clearly once we're thinking about the ingenuity, that led to quite a number of NSOs coming up with different sources of data in response to it. We should also keep in mind that we made some mistakes along the line because the interventions that we put in place preceded whatever data considerations that we should have during the pandemic. And the last thing that I want to talk about as background is the sensors had to, the sensors taking how to battle with the risk of increasing the infection rate, that is a COVID-19 infection rate, and also the sensors had to battle with. And for that matter, all other in-person data collection had to battle with the panic in terms of whether is not going to increase the infection rate. So keeping all this in mind, my perspective is that if you want to have a conversation around the ingenuity of data collection processes during the pandemic, there are quite a number of things that we need to take into consideration. And I'll quickly run through these things. The first is it is really giving us the opportunity to revisit the rudiment of quality statistics. And when I say quality statistics, all of us would make reference to the 10 fundamental principles of quality statistics. And we begin to ask ourselves whether these new sources of data collection really are reliable, really are representative, really address things that are academics over the period we've taken for granted. And Kunal, if you permit me, I would want to stretch the argument on data collection away from the field exercise itself when we are interacting with the respondents and begin to think about what the pandemic taught us from the perspective of those who are going to collect the data, how they are identified, how they are selected throughout the process. And the modes of training earlier on, one of the panelists talked about virtual engagement and also as academics, whether and also as policymakers, whether the few papers around recall ability to capture responses as they are in consistencies between responses, given that you have one respondent and different enumerators go there and how these feed in our discourses whenever we are doing some analysis. So the discussion around ingenuity should move away from the source of data, primarily saying that it is traditional sources of data, it is non-traditional sources of data, and step back to begin to think about who is collecting the data, especially when we are dealing with telephone interview, for instance, when we are dealing with online interviews, for instance, how do these eventually play out with the quality of the data that we're going to collect? And that is what I'm saying we need to reflect on the issue of the rudiments of quality statistics. The other thing that I think should be central to the conversation has to do with the partnerships and as I indicated earlier on, quite a number of things were happening ahead of the pandemic. So in the case of NANA, where we relied extensively on core detail records to look at mobility and the infection rate, this was an activity that we were already undertaking, but at the outset of COVID-19, we've now found the real need to do it in the context of the pandemic. So partnerships, collaborations are one of the things that I talked about as results that we're achieving on the heels of the data revolution that started in 2014. And premise on what I talked about earlier on in terms of the panic and the relationship between the pandemic and infection, the pandemic and response rates is the whole conversation around publicity, education and advocacy, once you are thinking about new ways of data collection because if we continue to work in this atmosphere, what we're going to see is how to deal with non-response. I don't know the experience of my colleague from Kenya, but when we were using the telephone interview away from the in-person interview for tracking the pandemic on businesses, one of the things that we had to grapple with is high non-response rate relative to what we would have been able to achieve if we were doing in-person training. So the whole conversation around publicity, education and advocacy has now become more relevant in the space of data collection within the pandemic. So a few things that I think we should reflect on as lessons moving forward is harnessing the options that are available to us for non-responses in the bid to ensure that we have quality data. And more importantly, we need to work aggressively towards identifying and addressing the biases associated with these new sources of data. This brings to the fore my third issue that hybridization of different approaches of data collection. And yet when I say hybridization, I'm not looking at only traditional and non-traditional data sources. Within traditional data sources, how do we think about hybridization and within non-traditional? How do you think about hybridization and also across traditional and non-traditional data sources? And this is the way we need to go and we need to consciously work towards reaching the point where, as I always say, data is now a byproduct, especially when countries in the global South are mimicking what is happening on the other side of the globe where digitalization is intensive. And if you go to the Scandinavian countries, specifically Denmark that we are collaborating with, but we are moving towards a situation where we tend to have sensors, pure base and not demonstrating data. This thinking should inform how we are working towards it over the inter-sensor period and you do that. Lastly, I want to emphasize the need for national statistical agencies to anticipate shock. Now we were all caught by surprise with COVID-19 and I'm asking myself, what about another shock that is going to come in a different perspective? I.e. with quote and unquote, with this rush towards digitalization and the use of non-traditional data sources if technology fails us, how are we going to prepare ourselves as NSOs to continue to provide data in areas that are needed for that reason? And thank you very much. I'll pause here and I'm hoping to further elaborate on some of the issues that I've talked about. Thank you. Thanks, Sam. That was very, very interesting and extremely useful to think about for us. I want to just, and I'm going to raise a couple of issues here and I don't want to respond to it from you all right now, because I want to also get to the questions. One thing I think we should do as we get back to a point where we might be able to do in-person interviews, in-person surveys, I want to see some pilots done of seeing the reliability of phone surveys versus in-person surveys. Because I'm not convinced yet the phone surveys are all that reliable. So I want to see pilots where you do the same survey for the same person over the phone and then in-person, try to see the difference there. I also would like to see much better thinking through retrospective questions. And perhaps again, trying to do a pilot on retrospective questions versus questions that are asked in the baseline. Because in this particular case of the pandemic, we had a lot of surveys, which are asking retrospective questions. We don't really know how accurate those questions, the data was from that. So I would like to see much more of thinking by the international community on those sorts of issues before we get to the next situation of another possibility of where we can't get in-person. I would also like to see, speaking as an economist, I would also like to see much more thoughts on qualitative methods, much more thoughts of citizen-led methods, much more thoughts of participatory methods, because those are easier to do in the situation in the pandemic. We have not, as economists especially, invested enough thought on those methods. I remember the World Bank had, it wasn't a poor project where they tried to bring in participatory methods. It went, it just went on, it was sidelined after some time. You didn't really pick it up, especially among economists. I think it's important for us to rethink these issues and try to be prepared in a much better way on these methods, because we know exactly the problem with large-scale surveys that are in-person surveys. And even though we thought of very innovative ways to handle that, they may not be foolproof. So I think those are things I think we should all think about for the way forward. Now, I have several questions in the Q&A already, and they are actually mostly to Paul and to Lauren. So maybe start with those questions, and then I'll have some questions for all of you. So Paul, I have a few questions for you, which you might have already seen in the chat, but let me read them out for the benefit of the audience. So the question are as follows. First question is from Hyde-Syn DeVries, who asked the question that, so did COVID accelerate the use and shift towards big data at KNBS, such as financial transactions and mobile phone geolocation data? So was there a pandemic kind of a shock towards big data collection in KNBS? Let me ask the second question to you Paul at the same time. So the question is then, and this is a very specific question, can one access the survey data that you've collected from the KNBS platform? Is that publicly available or not? So that's probably quite a straightforward answer that yes or no. And if I don't, if you don't mind, I have a third question for you. The third question is, what would you advise students and other researchers into switching from Cappy to Catty, which you mentioned in your own remarks, what are the cost and quality implications of this different data collection formats? Actually, that's quite an interesting question that cuts across to other NSOs too. So Paul, maybe you could start and then I will also have a question for Lauren and then questions for the other panelists. Thank you. Thank you very much for these questions. Actually, on the issue of the big data, we are switching on slowly to big data because we need also to build capacity on how to deal with the big data, but we are switching it slowly into that. So on that question, I don't have so much because we are building capacity through ONS. Now on the issue of availability of the information in the website, Pothole, we've been migrating because of the service provider. And when we migrate, we lost, we didn't migrate what we call NADA. The Canada is where we host most of our results from the surveys. So it is underway, so it will be there in due course and most of the reports will be available there. On the issue of the older reports, we have a working, you can go back to the website and look at the older reports on CPI or other, they're all in that web. The next question was about what advice we should give to students or researchers to switch from copy to copy in light of the COVID-19. Now, actually to say the truth is that when using copy, you are directly asking questions to individuals and you can see the answers and the body language. You can understand whether somebody is cheating or saying the truth. When you're using a copy, you'll never know whether the person is saying the truth or not. So in terms of copy, you can guess the answers straight and when somebody is lying, you can repeat the question. But for copy, you'll never know. So that one lowers the quality of data. But when it comes to the issue of cost, the cost in copy is very low because what you need is to have people and you are training them through the virtual and then the cost for transportation to the location where the households are is cut down. So in this case, the cost will go down but the quality will also go down based on that. I think those are the questions that were said. Thank you so much, Paul. Very, really good responses and very useful and really good to see so much of interest from the audience or the work that KNBS is doing. Now, Lauren, there's a question for you too which is also pretty specific actually. The question is that interactive data portal that you mentioned, will it be an open source platform? Open source platforms are extremely critical for resources in developing countries. Yes, thanks so much for the question. So the Clearinghouse platform is basically designed to pipe in multiple data sources on financing for statistical systems and budgeting for statistical systems. So the modeling part is still underway. The data sourcing is still underway. What we're releasing next month is a prototype of the platform. What we're also working on is a governance arrangement to make sure that we can use this data responsibly and at the same time produce the most value obviously for users. So that's a, you probably are picking up the long-winded answer to the question without actually answering the question. We are still in the process of defining the parameters for the platform, but open source certainly is a priority for us. What that will look like in practice for this platform I think is still to be determined. And I am not the person working on that specific aspect but it is certainly something that's a priority for Paris 21 and we're very aware that more open source is better and that is certainly a priority as we're working on the design. So I would say, stay tuned, join us at the World Data Forum in October and there'll be more information on the functionalities and how to use the platform going forward. We really, really appreciate the interest. Thank you, Laura. Do you want telling us when the dates of the World Data Forum is so that perhaps people can tune in? Ooh, that's a great question. I don't know the bottom of my head. I think it's the first week of October. I think I wanna say the sixth, seventh, eighth somewhere around there. All right, excellent. And that's supposed to be in Paris 21 website on the website you would have just be the dates at some point. Yes, I think we'll have some information on our events coming up but also there is a really nice platform already for the event. It's a hybrid event. So you can join in-person or online registration is open and if you're interested in data gaps there will be much, much content there to continue to explore these interesting issues. Excellent, thank you. There are still a couple of questions but I'm going to just ask a question and whoever from the Tom, Patricia and Sam can just jump in because we have literally like three minutes left for this panel. So the one question I think is a really interesting question is about mobility data. Google mobility data is now, actually we've seen several papers being used have used this data. So do you think that's the kind of data that we should be trying to collect more and how useful is that? So that any of you can answer that. And let me just put in the second question so that one of you can answer that about ethics of data collection especially when you can't do the standard constant forms and so on in-person. So is there a difference between collecting data through the over a phone versus in-person surveys in those ethics and so on? That's a very important question as we start moving more towards phone service and so on. Who wants to answer the first question on mobility? Tom and then maybe I think Patricia might also want to answer the next one. Yeah, just very briefly, I mean, time is running out. The question of the utility mobility data I think is crucial. I think it is certainly one of the ways which the world is going. I think certainly in the developing world in sub-Saharan Africa, you have to ask who has a smartphone connected to Google or Apple? And that's obviously a major concern. But beyond that I think we need to also almost press pause now. The COVID epidemic provided a spur for the most unbelievable unleashing of data collection exercises, particularly in the global south. We found mobile phone data being used to try and track the epidemic. And so we need to think about the ethics and morality and data privacy issues of unleashing all that data before we understand fully what the modalities of it are. So I'd like to emphasize the ethics of it. I'd like to also say we should avoid throwing out the baby with the bathwater. We need civil registration, vital statistics systems. If we, without those, we are never gonna get to the kind of data richness that we have at present in the developed world. So I'll leave it there and let Patricia pile in. Thank you. Thanks Tom. But to share the ethics question, do you have anything, any thoughts? Yeah. I mean, this is gonna be the biggest issue here. I mean, just to reiterate what I said, this mobility data was being used before to do things like tracking pilot events, protests and so forth in a variety of ways. And I see a lot of value now. Obviously, who should also mention spending patterns and all the rest of it that's been used by various, in various ways. I see a lot of value in this data versus two spot patterns of vulnerability. You see people moving in and out of different places. You can also, it's also being used with the patterns of displacement, for instance. You can see, suddenly a whole village it moves out somewhere. You see there is a problem coming up in that area. So it's being used in a variety of ways. And, but like Tom rightly says, I mean, the ethics issues are huge. Like to give you an example, when we first start thinking about these issues in terms of using phones in very remote conflict affected areas. One thing that you find very quickly is giving out mobile phones is not a great idea because that person becomes a target for armed groups. So there's all these really subtle issues that need to be taken into consideration. And we know also that some of these data is being used by different governments to track activists, journalists, et cetera. So as researchers, we have a lot of responsibility to act that. And we don't have time in this panel to discuss more, I can talk about for a long time. But one of the issues is what Kanaal mentioned before. The use and the qualitative understanding of some of these contexts is going to become really crucial. I suspect in the times to come. Thank you, Patricia. I have one, we're going to go slightly over time, but this is the last session for this day. So we could probably do that. I'm going to ask Samuel the one last question that came from the audience. That's a really important question in my view. And it's about opportunity that we are, opportunity of data collection with a new method of data collection. Do we see a difference in optimality in terms of the new methods versus the older methods, which were obviously much more large-scale because there is samples and so on? Or do you think we can still maintain the optimality that we need as researchers in the data collection that we are doing with new methods? Sam, over to you. Thank you very much, Kunal. Just a second, let me touch on the previous question on whether the mobility data can give us some economic indicators. And my answer is absolutely yes. At least from three perspectives, especially at a time that countries are beginning to pick GPS coordinates on all their structures in the rounds of the 2020 population housing sensors, there's huge potential to track where people are going to in terms of essential services and also where people are going to in terms of the district capitals and so on. And these are economic indicators at the geography level, which would help us understand a lot of the cluster-level effects that when we are doing analysis, we are not able to do that. And at least it's going to give us some small-area estimates on expenditure and especially for countries in the global south that do not have economic statistics at the lower geography level. Certainly this cost of mobile phone expenditure could be one of the indicators for economic and statistics. To your question on objectivity, Kunal, as I said, we are moving to a point where data is becoming a by-product. If you think about big data, it is a by-product. If you think about administrative data, it is a by-product. So certainly that objectivity comparison will be far-fetched in terms of the traditional and the non-traditional data sources. I think what we should begin to do is think about the adjustment factors, if any, so that once, as you rightly said, if data is collected using in-person.