 the honor of introducing Dr. Caroline Bucky. I guess I was asked to do this, but there's a hand privilege of writing a story about her work a couple of years ago, the MIT Technology Review. As you may know, Malaria kills about half a million people around the world every year and infects another 200 million. And Caroline is a leader in figuring how to use, among other things, the movements of ubiquitous phones to map where populations are coming and going and use that as a way, in her words, if we're going to eradicate malaria, this is how we will do it. It's really amazing stuff. And she's an associate director of the full title at Harvard Medical School. So I will turn it over to Caroline to explain her amazing work. And thanks very much for coming. OK. Thank you. Thanks for having me. So I have 20 slides. I was told 20 minutes and then discussion. But I'm really happy to stop any time. Just shout out what we can discuss along the way. Most of my work is on malaria, but I do infectious disease epidemiology in general. We use mainly theoretical models. And the focus of my work is on infectious diseases in low and middle income settings, particularly among children and vulnerable populations. So today, I'm going to talk about the use of mobile phone data in general. And this is not like mHealth. I design an app. I download the app and engage with people that way. This is about passively collected data that's routinely stored by mobile phone operators. And that raises all kinds of politics and privacy implications. And we'll talk about that at the end. I'm first going to talk through the utility of this data for public health. OK. So we've all seen figures like this of some kind. I think I'm going to talk particularly about epidemics and epidemic containment and the utility of these data during disasters and response to disasters. So when we think about as epidemiologists, what do you need to have a really huge outbreak? So we're concerned about this for reasons of pandemics and epidemics. You need a sufficient density of susceptible people. And as you know, we've had this incredibly exponential, rapid growth of populations in the last couple of 100 years. And that continues to be a big problem just over population and high density populations. And in particular, our cities are becoming incredibly highly densely populated. So this figure is from the UN, I think. And it just shows the percent of people living in urban areas in different parts of the world in 1950, 2007, and projected to 2030. So globally, 60% of people by 2030 will be in cities, certainly in places like Asia. We have the growth of these enormous mega cities. They're huge. And in general, I think it was in 2010 that we surpassed more than half of the globe's population living in urban areas. So these are highly densely populated regions. And that's sort of perfect if you're a pathogen. It's like little bonfires that you can spread to. So the other requirement for an epidemic is sufficiently frequent introduction events from outside, right? So we now live in a highly globally connected world. This map shows international travel routes in 1930. And you can see it sort of looks like this. And this is now by 2010. And of course, we're all familiar with these types of crazy airline trips that we all take all the time. So not only do we have massive population densities and very large urban centers, we also have huge global connectivity. So this is sort of the perfect storm for the spread of infectious disease and unprecedented capacity compared to the past. So you think about the Black Death in the 14th century, part of the reason that was limited especially was because we just weren't that highly connected. People didn't move that far in their lifetimes. So this is of course very problematic. And we've seen over the last decade really this new emergence of pandemic type infections often starting with a zoonotic event from an animal reservoir, right? So we've had SARS, we've had MERS, we've had H1N1. This shows the spread of H1N1 out of Mexico, very rapid global dissemination of the virus. Then Ebola in 2014, and then most recently Zika. So Zika was circulating in French Polynesia had an outbreak a while ago and then international migration moved it everywhere. And then of course we've had this enormous outbreak since 2015 in Central and South America and that continues now. So responding to containing these kinds of epidemic events are very important. And I think with this administration we're gonna pretend that this is all we do is just containment of pandemics that affect America because otherwise you can't get funding. But anyway, of course there are many other aspects of kind of spatial considerations for the control of infectious diseases that aren't epidemics per se, or they're epidemics but they've been going for so long and they affect people, some populations that we're not so interested in helping out. But of course they're equally important for control. So we have vaccine preventable infections and from a national control program perspective the questions there that are spatial are to do with where the unvaccinated populations are, where do we need to send our help workers, where do we need to focus our efforts? And then what are the risks of importation that can spark outbreaks? There are plenty of seasonal epidemic infections like cholera, malaria, dengue where we need to know when outbreaks are coming and prepare for them somehow, right? So sometimes this can be vector control, if it's a dengue outbreak, sometimes it can be about doing a preemptive vaccination campaign is kind of thing. And then of course for emerging epidemics the spatial component is really critical. So there's an outbreak somewhere, how do we stop it from spreading not just within a country but internationally? So the spatial requirements for containment, where do we put the mobile health clinics? Where is it gonna go next? Where do we need to enhance surveillance? And these, and of course trying to understand how quickly and in what direction the pathogen's gonna spread. So these are really kind of critical pieces of any healthcare program. And one of the ways that infectious disease modelers go about trying to understand these dynamics is using a kind of spatial approach to mathematical modeling of the disease outbreak. So you will have subpopulations defined by some variable spatial unit. And what you wanna know is what is the prevalence of the disease in subpopulation A and what is the connectivity between subpopulation A, B, C and so on. And so we can model outbreaks within populations and then we need to know about the mobility between regions in order to understand and contain the spread of the disease. And so we can use network models where populations are defined by some spatial or demographic region that makes sense. We connect them with mobility flows and then we're trying to understand, say where this country, the low-risk country, how often are we getting infected visitors over our border and how often are our residents going away somewhere that has a lot of the pathogen and bringing them back. And generally speaking, we're trying to differentiate between importation events and local transmission. And then ultimately the reason we do that is so that we can contain or control the disease either through surveillance, so we stop importation. During an epidemic that might be through travel restrictions which happened during the Ebola outbreak or quarantine. And then we can also do transmission reduction. So for Zika that would be spraying, vector control, these kinds of things. Okay, so what are the kind of spatial and temporal types of data that we need to parameterize these models and actually understand migration in human populations? So this is a review paper that shows the spatial from kind of small scale to large scale, spatial dimensions of mobility that impact disease. And here's the temporal from daily to long-term aspects of this, right? So at the fine scale on a daily level, we've got mixing in schools and at work and people moving around on a daily basis. Then we have all the way up here, international long-term travel. So these are like migration events, refugee populations that move for the long-term. And then we have all of this intermediate type of travel that's really the most important for things like national control programs and spatial containment of epidemics because they deal with inter-region connectivity. So within a country between districts or something like this and that drives some of these questions about resource allocation for containment and control. So we have, there are different ways you can get at these rates and flows of travel. So we have travel surveys and little small scale GPS studies where you like stick GPS on people and follow them around or put an app on their phone and follow them around, right? Those are by necessity limited in scope, usually short-term and they're certainly not scalable for routine use. Travel surveys are difficult because people forget sometimes they don't answer honestly for various reasons and they're just not particularly reliable. They're also limited in scope just because it's hard to administer surveys well. Census data is fantastic because you get all this demographic richness and detail and information but it only asks about long-term like where did you live last year kind of questions and obviously on the timescale of an outbreak that's not gonna be particularly helpful. And then there's airline and shipping data that deals with kind of international travel among very particular demographic groups, right? Most people that I'm interested in aren't able to buy a plane ticket. So we've got this gap, this big data gap that for regional travel and until recently infectious disease FPP people for their modeling would use very, very simplistic theoretical models derived from physical laws actually to try and parametrize migration. Very hard to validate, it's not clear whether it works well especially in low and middle income settings. Okay, so here comes mobile phones. Again, you've probably all seen figures like this too, right? Massive penetration of mobile phones globally even in low and middle income locations and in fact in many countries we're reaching saturation in terms of the number of used sims and handsets that there are in countries and I'm sure you guys all know about that a lot better than I do. Suffice to say that yes, it's true in poor regions not everybody has a phone, there's a lot of phone sharing, all of these kinds of things but there are mobile phones basically everywhere now and that's likely to only increase into the future. So mobile phone operators routinely store data from phones in order to for reasons of churn analytics and this kind of thing and what they store is this thing called the call detail record and what that does is it logs for every activity on your phone and it doesn't matter if it's a smart phone or a dumb phone it logs a cell tower ID every time you make a call, text, whatever, right? So what that means is that as you do things on your phone and of course you have to do some inference here because you don't use your phone all the time but as you do things on your phone you have this like trace of cell towers that you were close to while you were making those calls. So we have a trajectory for that person over time and the resolution of your estimates for where that person was will depend on the cell tower density. So in rural areas it might be like five kilometers but we're not really sure where you are. In cities it can be like a city block we're pretty sure where that person was, right? And we'll talk about that later when we talk about privacy. But in any case, so this is just sitting in the basement of operators everywhere. Sometimes they get rid of it after three months or six months or three days but it's there and it's easy to collect it's essentially free and it's updated in real time. It's also scalable so we're talking millions and millions of people. So really what that means is that suddenly, okay the resolution is not always great but we have data on this part of the spatial mobility dynamics that we need for our models. And so this is great, it's cheap. I think it should be used and it's already being used by the operators, it's just a question of how we use it for public health. And it's not just this type of data, right? So here's a picture of Kenya. We're also getting increasingly high resolution satellite imagery. So we can overlay multiple layers of metadata now where we have, so this shows the population density of Kenya and of course mobile phone towers tend to follow population density. We, you can't really see this but work by Andy Tatum at the University of Southampton and others, they use different types of satellite data to delineate where settlements actually are and that's usually much more reliable than census information, especially in areas where census gets updated really quickly. We need to know something about the pathogen of interest for this type of work, right? So this is a map showing the prevalence of Plasmodium falciparum, human malaria parasite in children and you can see highly heterogeneous coverage like a lot around here, Lake Victoria, some on the coast and then not so much in the middle, there's highlands there so the temperature is not so permissive and then here we've just overlaid all of those things including the cell towers, so the cell towers are where the black dots are, right? So you can see it follows population density and then what we can do, right? We, if we have cell phone data then we can measure connectivity between towers, longitudinally over time because there is important seasonal migration events. We can use that to delineate our settlements using some of the satellite information and then we can make some modeling estimates about how frequently we think the parasite will be traveling between these places. And so that's useful because it turns out that there are usually particular foci of transmission that are spreading the disease to other places so these maps show if you just do the travel, the travel analysis and you ask the question where do people come from and where do they go to on average, right? So you're looking at anomalous movements to and from particular places. This shows travel and you can see this is Nairobi the capital. A lot of people travel to Nairobi so we refer to that as a travel sink. A lot of people are coming from these highland regions to the capital or to work around these lake regions. Once you overlay your epi data and estimate flows of the parasite you see a different picture where most of the infections that are being imported across the country are coming from this region near Lake Victoria and they're spreading into the highland regions where we do see epidemic outbreaks and into Nairobi and Enveron. So this is important because you don't get local transmission here but you see these cases popping up and they're most likely imported. So the lesson here is that if you really do a good job of controlling transmission here you're gonna have knock-on effects, right? If you're endlessly distributing bed nets here you're gonna be mopping up imported infections and sparking transmission events all the time, right? So it's about making resource allocation more efficient and targeted for control. So that's from malaria which is more or less endemic in Kenya. In Pakistan we looked at dengue to try and understand whether we could do a better job of forecasting. So this is moving beyond just trying to look at general flows and where things come from and go to and actually see if we can do a good job of predicting how things are gonna spread. So in this study we used mobile phone data and we looked at an outbreak that happened in 2013 in the northeast of the country in Lahore. And we asked the question how much better do we do if we have this type of information embedded in our models compared to a standard modeling approach? And the answer is we do a lot better which is comforting to me. And here you can see one way that people think about dengue risk which is just suitability for vectors and it's sort of this hard to interpret, right? But if we use our modeling approach and you incorporate this kind of data you see very clearly that there are high risk regions right around Lahore and we predict that those are in fact the places where you'd get outbreaks. And in fact there are a range of things you can do along these lines. So it turns out that general mobility patterns so just how much do people travel in this particular place are highly predictive of how many children have been vaccinated in that household and whether the women have got prenatal care. So mobility is also a kind of general proxy for other things and can be useful more broadly. And then even large scale seasonal fluctuations on the district level. So this is not high resolution mapping or anything. This is like large movements of people seasonally can do a good job of explaining dynamics of childhood infections like rubella which was interesting to me because children were not necessarily able to model children movements, right? So but it's a direct measure of kind of overall population connectivity and it does a pretty good job. Certainly a lot better than previous proxies which were essentially school closures and school terms and rainfall and things like this. Okay so that's useful and those are kind of routine health kind of questions about disease control. So during outbreaks of course this becomes urgent and we're trying to understand how things are gonna spread from a new outbreak event. So in 2014 Ebola hit and there was this massive like pile on to try and get this data from UN agencies and researchers and everyone and it was a mess. And operators didn't want to share the data because there weren't protocols in place for privacy. The regulators were bulking for clear reasons. The government was getting involved. It was a mess. We didn't get any, well we did eventually get data that Cori's now working on but much after it was useful for the actual outbreak. But we produced some maps that were kind of modeled estimates based on mobile phone data that we did have for other countries in West Africa. So we put these out and said if this is helpful you can use it. I don't know how helpful it was. It would have been a lot more helpful if we could have just accessed the data and then we could have done a better job. But the only good thing I guess was that we learned we helped the GSMA which is like the mobile sort of international mobile phone regulating type organization. We helped them develop some protocols and mainly the protocols are about how you aggregate and anonymize the data in such a way that we're not worried about targeting particular individuals or particular ethnic groups or whatever. And that's something that still hasn't really been standardized and that we're still working on and it's a constant issue. So I'll just mention one other thing. There's a very obvious use for this data in natural disasters. There's an earthquake or a flood or something and you wanna know where's everybody gone? How quickly are they returning back home? Where do we put supplies? And so this in collaboration with FlowMinder Foundation which does this routinely you can do things like just look at a fraction of people who move during some of these events. Where did they go? How long do they stay there before returning home? All of these kinds of things. You can sometimes in the data you can see things like money transfers, where's money being transferred around? And it can really be helpful for targeting aid. So I guess my vision for how I think these data should be used for disease control is that you have your national control programs. So this is a risk map of malaria in Cambodia and it can look heterogeneous. It can look however it looks, right? And then you have your at risk population. So normally what happens is that the national control program allocates, you can't see the arrows, but anyway, allocates resources to populations using these risk maps to prioritize who gets what, right? And the data that and then these report back, they report back, you know, how many people we had with this, how many tablets did we use, all of this kind of thing and then that goes into the next risk map. So I think there are lots of ways that mobile phone technologies can be used in this cycle to improve health. So something that is not to do with CDR analytics that I didn't talk about, but of course participatory surveillance. So that's like, you can imagine flu near you but on your phone, right? Where people are saying like, I have these symptoms. That can be geolocated. We could have automated reporting by clinicians and that could do a lot to improve our initial risk estimates. You could, you know, what I've been talking about, you could imagine that operators in country routinely produce mobility maps that could be integrated into control programs so that they have a better idea of where to target resources. And then, you know, you can also imagine that phones could facilitate a personalized information back the other way. So operators, I mean, so national control programs could target educational messaging and particular response to places and people based on all of this data. So there are loads of places where mobile technology could be used as an input to improve and facilitate public health programs. At the moment, that absolutely does not happen. So I will finish by talking about some of the problems with using this kind of data. So privacy is obviously, protecting the privacy of individual subscribers is absolutely central to some of this, right? So at the moment, we try and develop best practices on like how to anonymize and aggregate the data, but there's no consensus. And it really depends on the country you're dealing with. So some countries you'll go in and they'll be like, have some data. And you're just like, please don't show me the SIM card number, right? Like there's no kind of regulatory framework. It's not standardized. Other countries, you can't get the data no matter how well you're gonna aggregate it. So for example, most of the, when we worked in Pakistan, all the data stay behind the firewall. The operator's data scientists did the initial anonymization. The operator's research team did the aggregation to our specification subsequently. So it was already anonymized. Then we aggregated it to a level where it was only transition matrices on weekly between fairly large spatial scales. But it's really hard. That aggregation has to happen in a research question specific way, obviously. And that's difficult. At the moment, there's a push to have, so governments understand that this is useful, right? So what happened during Ebola is in Liberia, they said, great, the government will have the data and they will just use it for good. Seems like a bad idea, right? So the operators need to maintain control of their data, but the regulators need to be on board with how that data is used. And then the lastly, I think there are misaligned incentives across the board for making this use routine. National control programs largely just don't wanna lose their jobs. They wanna help health as well, but they are conservative. Academics get rewarded for publications and that's it, not for seeing through these things and making sure that they get into populations. Operators have minimal incentives to share this and anonymize this data. It's a cost to them, apart from some PR, it's not clear why it's good for them, except in the long-term investment in the populations that they work in. And then the regulators and the governments also have their own complex stuff. So there's no standardization. There are a lot of difficult negotiations that have to happen. Data quality can be variable. So what happens is you negotiate for ages and then meanwhile the operator is like, great, you're gonna have a result next week because we just gave you the data, right? Which also doesn't happen. So this is just a flow diagram for one of the projects, right? So you say to the operator, can we use your data? We're gonna do a project on X. They say yes. You talk to the control program and you say, can we use your epidemiological data for this project? They say, well yes, if the Ministry of Health has signed off. So then you go to the Ministry of Health and they say yes, but has the operator got regulator approval? Then you go to the regulator and you get regulator approval and the regulator says yes, but has there been Ministry of Health approval? And you say yes. And then the Ministry says, but what about the Ministry of Telecommunications? Have you got a letter from them? So then you go to them. And they say, well, if the Ministry of Finance is, I mean, Ministry of Finance stands above all ministries, obviously. And they say, well, okay. And then you have to go back to national control programs and you need obviously all of the normal ethical approvals for using EpiData. So you can imagine, there's a lot of waiting for letters. There's a lot of trying to bring people to the table. So that's for one project, let alone saying now we should make this routine, right? There's a huge amount of bureaucracy. So I see this as one of the biggest problems. Privacy aside, and I think there are good ways to anonymize and aggregate this data that will protect subscriber privacy, but you've got to navigate this every time. You know, and the Ministry of Health usually says, well, we'll do a dengue project, but you weren't signed off to do cholera. So then you have to go to the cholera control program and it all starts again. So this is a very problematic. And I think it's probably the biggest hurdle to getting this stuff into routine use because the mobile phone data itself, it's not rocket science, right? It's like elaborate counting with spatial dimensions. And putting it into the model takes some capacity building, but it would be straightforward to do. So I think this is some of the problem. And the worry about privacy is legitimate, but I think there are answers to it. So I will stop. I'm happy to discuss anything. And these are some of the partners that we work with. So thank you. I guess on just building on your closing comments, what's a realistic solution? I mean, is it something that operates at the level of these large multilaterals? Like UN, WHO, that set standards, and then how do you ensure compliance to those standards? And I think the point about sort of vertical permissioning based on health condition is on point. Yeah. You have some conditions that are more stigmatizing than others. So if you wanted to approach some sort of blanket permission around a health issue, emerging health threat, et cetera, how do you avoid kind of weaponizing a policy framework that disadvantages some individuals or plays into the hands of others? So I think that sociology is always harder than the technology for things like this. So that's focusing on. I totally agree with you. And I think while we can anonymize individuals, if you clearly have a region that is largely refugee populations, the stigmatization issue is very important. And so for example, in Swaziland, for malaria, they're very close to elimination. And you have to prove, for elimination status with WHO, you have to prove no local transmission for three years, but imported cases are okay. So already, so what happens generally is that plantation workers from Mozambique come in seasonally and they work in Swaziland and they're already stigmatized to some extent. So adding to that problem is not ideal. You want accurate estimates without the possibility of particular groups being stigmatized. There is an unfortunate correlation between high-risk groups, vulnerability, and stigmatization. And those are the ones that need the most access to healthcare and all these things. It's a really hard problem. So some people like Gates have started to talk about whether it would be best to have a mobile hub, right? So you have all the operators say, okay, we're not gonna decide individual projects, we're gonna put our data somewhere else. And then they decide how to, I personally, I'm not sure that's a very useful solution because it creates a different kind of bureaucracy. It means that the data is not, you're not nimble, especially for emergent epidemics, right? Because you need this thing to be seamlessly. And it also reduces control, it reduces kind of autonomy of individual subscribers to do things through their CSR. And I also think that there is an issue with data leaving countries. It's best if everything stays kind of in country. It's individual governments. One of the biggest problems we have is that you just get permission from one person and then they lose their job and then it's another person and then you've got to establish trust again and all of that stuff. I, that's a governance issue that I don't know how you usefully solve. I do think that the GSMA and there is a, you know, like if you look at genome data, right? We have like standard ethics principles and guidelines for how we think about genomics data and how it should be used and all that kind of thing. I think it's very important that we try and establish standardized protocols for this type of information, maybe through the GSMA that is agreed upon by maybe not the WHO, but some overarching body so that when researchers or whoever goes in, there's some guidelines somewhere and they can say in their paper, like we followed this protocol. That's really important. So I would say, yes, we need to have these overarching principles and protocols that are based on like evidence-based science. How that's implemented within individual countries is always gonna be open for, like there is a country that I was in last year that's sort of politically a little bit unstable and there was a guy in the operator and his only job was to feed this data to government upon their request. That opens them up for all kinds of ethical issues, right? And that's certainly not what I'm advocating for in the control programs. I don't see how you get around like corrupt governance issues in country, but I think from an IRB or academic research perspective, yes, we can absolutely make guidelines. It's hard though. I have one additional question. So if I understand what you've done here, you're really modeling population flows by these captures on the cell towers. So have you tried to, or at least measured or quantified what's the value added of putting individual search query data into that? In other words, like are people searching about the symptoms of malaria, et cetera, which you can also kind of model. So those are two different inputs, the movement of populations relative to these reference points and the other four people asking about them, you know, reflects that individual. So we haven't. But right, so when I was talking about participatory surveillance, right? You could easily imagine, at least spatially, I don't think right now our protocols are specifically not for linking individual personal data to the CDR records. Yeah, but you can actually on a spatial level say like, where are people? Yes. And that is something that I think we will see more and more of. Linking up this, it'll be like another metal layer of data, right? Yes, I think that's, the other thing is I think you could imagine a big study on potentially where you have, you have a like, you know, whatever you look up your symptoms on a map, whatever. And it says, do you give us permission to use your CDR? Then through the operator, you could then if you got explicit permission and consent to use people's trajectories, then you could do it, you could do a great job, right? Cause then you could actually look at individual movement patterns and health outcomes. I'm sure they in China with Baidu, I'm pretty sure they can do that already because there's less regulation stuff, but I'm sure you could do it. Sean. One is, are the cheap feature phones, even if you're not using them, but they're on how they, is your location being marked at the cell tower? No, only if you use them. The other is, we understand at a high level what's going on here, but the relative impact of this versus all the other interventions, why is this such, you said at one point, this is how we would eradicate what Larry has an amazing thing to say. Why is it that it's that? Essentially, because of the difference compared to before, so there was no data about this. So people are like, well, you don't know who has a phone. True, there was nothing before. So apart from anything else, the sort of scope and resolution of data is unlike anything we've had before on the spatial scales that we actually need to start like nailing down and properly doing some of these elimination programs. To bring it back to an example of say, Kenya, which you were talking about before, everybody knows where the malaria is and you know you wanna fight it there, but what might be counterintuitive for non-doctors is if 10,000 people last month went to this other region to go work at a farm or whatever it is. And those people are gonna be reservoirs of the parasite. You didn't know that before. You didn't know it at all, but those people are now re-effecting everybody else in that new location. Right, so there's malaria all over Kenya, right? I'm not in Nairobi because it's high, but basically all over the place. So I guess it goes against dogma, right? The dogma would be, okay, we're gonna eliminate in Kenya by stamping out the low-hanging fruit, which is these low-transmission zones, low-transmission zones, so these around the country, right? So you're like, this must be an easy place because there's only a few cases, which is exactly the opposite of what we're saying. We're saying that's not right because those are largely imported and there's no way to kind of know that before. So for example, in the Kenyan Highland region, there's been this debate, very vitriolic debate about whether the resurgence of epidemics in the highlands has because of climate change or because of increased population densities and migration, but nobody had ever been able to measure the impact of migration before, only temperature and rainfall, right, whatever. So you can actually measure it and now you can say, yes, we would predict there should be tons of malaria coming into that region and causing epidemics. If you cut out, if you manage to, if that region in the lake, if you manage to get just like absolutely blitz it with interventions, we predict epidemics in the highlands would go away. So it's a sort of ecological, dynamical framework to think about control rather than just like, where is the malaria? We put things there. And in resource poor settings, that's a big deal because it saves, like there was a project in Namibia where Andy Tatum, a collaborator, was working with their control program and they were able to reduce, in one year, they were able to reduce the number of bed nets deployed and the regions deployed by like 80% because they could target the actual places where all the malaria came from, based on this type of approach. Just to follow up on that, they reduced them in 80% reduced with bed netting, like I always hear like in bed nettings, the single most- For transmission. Low cost intervention, the idea of like, well, let's get rid of 80% of them. So there was a follow-on that said, even though we did that, we saw the number, that's... Yeah, so they're working long-term in Namibia, yeah. That's amazing. Yeah, it's amazing and I think, so bed nets are for transmission reduction, right? They assume that there are mosquitoes there. And so for malaria in particular, it matters because how you deal with imported cases is totally different than how you deal with local transmission. Why is that? Oh, because transmission reduction is about mosquito control and bed nets and spraying. Treatment of imported cases is about like, access to automiscinine and things like that. But you don't need to attack the mosquitoes because it's not. So yeah, so it's very, it sort of changes the scope and the efficiency and sort of targetability of some of these health goals. And then for, I mean, for emerging epidemics, I think it's a no-brainer, right? Like there's a new flu strain in Bangkok, right? You could imagine it could happen easily, right? You need to know where everyone's going, you know? Especially like with Ebola, it's 10-day period where you might be wandering around without symptoms, right? You really need to know where everyone's going to even think about planning your containment strategy. So. Is it actually literally true what you said about funding in the US right now requires, what was that, a little facetious? A little facetious. I think that for sure, one of the few areas that is sort of safe is probably gonna be biosecurity, pandemic, you know, things that directly threaten the US. And I think global health, more broadly, is likely to suffer from root cuts. I mean, they're already foggity is on the chopping board which is very short-sighted, obviously, because, you know, where do they think these things come from? But that's a separate discussion, maybe. Other questions? Are these tools actually being used anywhere? You said some countries are better than others. Is there one place where they're doing it right? And they're doing... So I would say Namibia, as that example. This is now their sixth year of Andy helping them with their risk maps and deployment of... actual deployment of control programs and monitoring. That's about it. So in Pakistan, we thought we had a good thing going and we thought the model was good and everything and then politics took over and became slow. I hope, I mean, I hope that this will become routine. It's just incredibly complicated and political. Paul, can you say something else? I think it was really astute that the incentive structure are operating on academics. A lot of these things end up being our demonstration projects and our NIH grants that we get our people out. Yeah. It's done. How, in your view, do we navigate the relationships with host countries and host country investigators or government offices in that setting? Because there has to be a sustainable handoff for this to live its own life and maybe there aren't the structures to absorb them, given that. And we're not in a hospitable moment with our own NIH and AID and all that kind of stuff to say that's really key. Yeah. So if you're a PI of one of these projects, or somebody here in this room who may want to be, so what are the implications for running your study? So the idea would be that I never have to do it again because people there should be able to do it and they can't, it's not lack of sort of ability. What I find is that so the data scientists and the operator are incredibly technically gifted people. The junior people in the national control programs are less technical. And I personally think that, so someone like the Wellcome Trust, right? They invest in long-term relationships in particular places where they do capacity building and they build up these long-term relationships with control programs, governments and trained people. And I think that the, if we can do a good job of, if I had money, I would throw it at junior scientists in low and middle income settings to work in control programs, right? Applied epidemiologists. So we're running a workshop next month actually. We're bringing, this was a visa nightmare. We're bringing 15 junior people from control programs from 12 different countries here to learn mapping, modeling, talk about this stuff because you, and networking and stuff. Because you need, I think if you have a strong team of technically able people on the ground, you need the one guy in the operator. You need the one guy, and I mean guy or gal. You need the one guy in the control program. And as long as the higher ups have signed off, they can do it, but that capacity is not there. And in many places the kind of signing off is not there. Once it becomes routine, I think sign off will not be an issue. It'll just be, you've got to keep the operator incentivized to actually do the anonymization and send the data over. And then you've got to have the counterpart in the control program. But that kind of echelon, the actual data scientists, those are the ones that make it happen. It's not. So I think we have to invest there. And I know, I don't know, maybe the World Contrast will invest there, I hope so. Because the researchers aren't incentivized correctly. You just like publish and go, so. Yeah, and you know, that's the model. That's, you know. It's hard to get around that to you. Yeah, that's the context in which a lot of us work. Yeah, yeah. Do your best. Yeah, so I think capacity building has to be like a major sort of programmatic investment area for people like the World Contrast and other, and in country, too. Any more questions? I think here now. No, they're coming mid-May. I have now learned all about requirements for a visa from Papua New Guinea and requirements for a visa. So it's great. I think it'll be wonderful. I just hope that they're not all stopped at the border. And I'm going to have to rush and try and rescue them somehow. I think it should be okay. What countries are they from? Bangladesh, Myanmar, Cambodia, India, Papua New Guinea, Rwanda, Madagascar, where else? Not Brazil? We have one Brazilian coming, yes. So, yeah, 12 countries I can't remember. And how about Ebola, it was an outbreak? How this was applied or not applied? So, all right, I could let Corrie talk about this. So, it wasn't applied because nobody, the data was not, it was a sort of comedy of political errors. And this academic thing really got in the way because everyone wanted ownership. Everyone wanted to brand their name on it before there was even any data. And there was one point during the outbreak where I think Andy basically was sitting in front of the data and he was like, we can do this right now and send it to MSF or the Ministry of Health, we can send them a map right now. And it was like, no, we can't, right? Because the UN needs to blah, blah, and this person blah, blah, blah, blah, blah, and it's just, yeah, and it just got, and it was really frustrating because it wasn't like Andy was in it for glory, it was just, it's right here, this would be useful, why can't we just do this? But I think this kind of need for, there are people that need to be seen to be like, look, we're the tech and health agency or whatever. So, we've got to have this as our brand, right? And it's like, who cares, right? Like, who cares, like just, why don't we just get the data out? You can write about that. So, Corrie's writing a paper because we did eventually get the data after the fact. And it's, all we did was we just said, do, can you measure in real time the impact of travel restrictions during an outbreak? And the answer is yes. And it correlated with Ebola prevalence, which, not surprisingly, and what was it? Like, long distance travel was reduced by like 75%, yeah. So, as a kind of just a policy tool, like, is your intervention working? You can do this in almost trivially. And that's- I think it was that paper after that on cessation and air travel and I for the gross discontinuity and flu maybe or- Well, as interesting as people went right back to normal straight away after the lockdown. But it's, I mean, the point of this paper is just like, this is straightforward. You can do this in a way that protects privacy. You should do this, you know? It's unclear. Hopefully it will be useful for the next time. Like, lives saved had the state of, you know, like that projection. That's harder because the epidemiological data was so bad. And that was one of the big problems for that outbreak was the EPI data was just, it was the management of the data and all of that was hard. So useful for next time. And we were discussing the privacy issues. I was very glad to hear about sort of the pre-aggregation and the obfuscation. We have a whole team here working on like doing that for data sets. But what I was wondering is specifically with respect to this type of research, does the, let's assume everything is perfect for you. Like, you're allowed to get what you want, you're allowed to do what you want, it's all good. Does that data have like a timeframe in which it is useful? Because you mentioned right at the beginning like, great, would have been nice to get this two years ago. With respect to, okay, you gathered it. You're doing all the appropriate things, but you know what, you could delete it now. Because it's not germane anymore. Or will it always continue to play a role? So I think it's useful for lots of things that you can't necessarily anticipate. One of the things that's clearly very important in many countries, especially in low income settings is seasonality and migration. You need multi-annual data sets to even look at how that's changing. But also, we have data from Kenya from 2009 and it was, you would have thought that it was basically obsolete now, right? But other people have parasite data from 2009 and then you can match it up and it's still useful. So well, and operators do get rid of it after a certain time, right? They chuck it out after some variable, three months, six months, nine months. I think one solution, one way that you could do this in a way that might make people happy, right, is get rid of the raw data. Yeah, get rid of the raw data, but do enough aggregations that you're pretty sure you can answer, right? In terms of pleasing certain regulators, if you could say, look, we're gonna delete it later. Yeah, we're just gonna keep the transition matrices. Yeah, and I think that's a good way to kind of do that for sure. But also, the raw data is just a pain because it's enormous and it's not very easily, that's not very tractable. Thank you. Thank you very much. I'll find a way to collaborate with the group.