 Here, Patrick Meyer, talk about his new book, Digital Humanitarians, you can see on the screen or in person. There are copies for sale outside, as you may have seen on your way in. I assume Patrick, when we're done, you'll have time to sign copies. So yeah, if you haven't gotten it yet, and would like to get a copy, there's copies there. I encourage you to do so, and Patrick will be around to sign them. Patrick wears quite a number of hats, including the one he has on literally. But figuratively, he is the head of the Digital Humanitarians Network, as well as the director of social innovation at the Qatar Computing Research Institute. Patrick has a PhD from the Fletcher School at Tufts, and a master's from Columbia. And blogs very, very actively on iRevolution.com, Patrick? Org. Org, yes. Orgnet, it's bookmarked in my browser, so I don't remember the suffix, but it's a great way to keep up with the many subjects that Patrick discusses in the book. The book looks at, and I'll just very briefly, I'll just let Patrick tell you what's in it. But it's got a lot of good stuff, both about social media, satellite imagery, UAV imagery, and a lot of other great stuff. But I won't tell you about it twice, I'll let Patrick tell you about it himself. Patrick's going to talk, he has a PowerPoint here for a while, and then he and I will ask him some questions, and then I'll have the rest of you ask him some questions. So thanks again for coming, and without further ado, Patrick. Thanks so much. Thank you. Thanks for the kind introduction. Really great to be here. This is my first book talk in Washington, D.C., so it's fun to see some familiar faces as well. Thank you very much to New America for hosting me, really, really appreciate it. So digital humanitarians is really a collection of stories that I've come across over the past five years. And if there's one, two common threads to these stories, one is technology, but the other is really hope, hope in humanity. They really are stories that have inspired me to try harder and to do more in my own professional life. And I hope that coming away from this talk, and maybe also reading the book, you'll perhaps feel a renewed sense of agency, and that we can actually really change the world as a digital collection of humanitarians. My own personal story begins on a very specific date, on January 12, 2010. And as this slide probably shows, this was when the earthquakes struck Haiti in Port of Prince, and the reason that my story begins here is because my wife was in the middle of that disaster. And for a long time, I didn't know whether she was okay or whether any of our friends who were in Port of Prince at the time during research had actually survived the earthquake. And so I found the news, I saw the news that afternoon. I was in my dorm room in Boston, where I was a graduate student. And as the breaking news started playing on CNN, obviously I went into kind of shock, anxiety, stress, all the possible human emotions you could possibly think of. And I really felt quite paralyzed. I didn't know what to do. I tried to text, I tried to call her and my friends. I looked at them on social media, I did everything I could to see if, you know, some kind of sign of life could be found, but I found nothing. And I just couldn't bear the thought of just waiting. Just waiting for the news and seeing more of the devastation being broadcast from Port of Prince. So I decided to try and do something, anything. And I was director of crisis mapping at Ushahidi at the time and I figured, okay, I'll just map. But it's important to know that it was not a strategic decision on my end. It was purely emotional. I just felt like I needed to do something, anything. What happened next was a somewhat surreal, and still when I share it today feels surreal. I started mapping that evening with a couple friends and the way we were able, the reason we were able to map was I went on social media originally looking for my wife and had not found her, but found a lot of Haitians tweeting live from Port of Prince. And so looked for those tweets and then went on Facebook and found Facebook messages. And some of these messages and tweets included geographical information, which meant they could be mapped. He would say, oh, the pharmacy, the drug store, this location has been closed. The church at this location has been damaged and so on. And so with a couple friends online via Skype and I, we started mapping that information. But the next morning, first of all, thank goodness I got word finally an SMS that my wife and my friends were fine. They had just narrowly escaped a collapsing building in Port of Prince. So they were very lucky, but obviously the vast majority of people in Port of Prince were not nearly as lucky. We mapped the whole night, didn't go to sleep. And the next morning when the sun came back up, we were completely overwhelmed with the amount of information that was generated from social media to mainstream media, you name it. I mean, mainstream news went 24-7 and with daylight returning in Port of Prince, a lot more people went out witnessing the damage, were also sharing that witnessing on social media. And so we were overwhelmed and looking back, as I say in the book, this was really in hindsight our first battle with big data. We were just completely overwhelmed with the amount of information being generated. So I sent an email to the student listserv and said, hey, I'm trying to map Haiti. If anybody's around, this was winter break. So not many people were around and if they were, they weren't about to start doing disaster response necessarily. So, but anyway, I sent an email out and showing up that evening, a few friends and complete strangers showed up in my dorm room and we just kept mapping and mapping and mapping. And we were still overwhelmed. So a few days into the operation, we sent word to the undergraduate students at Tufts University, which is where I was doing my graduate work. And sure enough, on a Saturday morning at 7 a.m., we had a hundred or so undergraduate students showing up. This is middle of winter break. They had no reason or no need to really be there, but they really felt compelled they wanted to help. And so we ended up having a full-fledged citizen-driven disaster response agency, basically spring up out of the middle of nowhere in very snowy Boston, some 1,500 miles north of Port-au-Prince. And we were really honestly making it up as we ran along. None of us, no one, had really done anything like this before. So how do you monitor all these social media feeds and mainstream media? How do you find the GPS coordinates of them? And then how do you map it? How do you verify the information? All that had not really been done at this scale before, and it really required hundreds of volunteers and far more than you see here, because we had a few visiting students from Geneva and from London and from other universities around the world. And they emailed back home and said, hey, folks, we're setting up our own student-driven situation room, crisis management room here at Tufts. Can you do the same and help us out? And we literally had a global network of student-run situation rooms around the world that was supporting our efforts here in Boston in snowy Boston. And it was a mess. I won't lie. It was a complete mess, making sense of all that information using the digital tools we had at our fingertips. And that was primarily Google spreadsheets and Google Docs, Google Groups, emails, and literally copying and pasting information we were seeing online. So it was all manual, literally reading tweets, one after the other, finding those that were relevant, copying and pasting that into a Google spreadsheet. And then another team would be responsible for going through that Google spreadsheet, trying to find GPS coordinates, and another team would try and verify that information as best as possible. Another team would then add that information to the map. If you think more than 100 people were working on the same one spreadsheet at the same time, it was just chaos, complete, complete chaos. We're changing all the time. It's like a Christmas tree. And it was just possibly one of the worst ways to organize and manage information. But again, we didn't have the luxury of sitting back and writing the ultimate guidebook. We wrote that guidebook on how to do citizen-based disaster response online as we went along. The result was quite astounding. And again, keep in mind, we were making this up as we went along. And there was no major strategy beyond just wanting to map and be helpful. At one point in the operation, I've been told this mic does the tricks. I don't need to do that. Our map looked like this. This is actually a screenshot a couple of weeks in. And each of these numbers represent a number of individual reports within that particular area. So if you were to zoom in further, you would see 22 individual reports of damage and needs in that particular area. And this was really no ordinary map. Because this map did not look the same for more than 10 minutes. Again, the reason being that you had hundreds and hundreds and hundreds of volunteers in dozens of different countries and every time zone around the world that were continually adding and sourcing information from the World Wide Web and adding relevant information to the map. So this map was alive. It was changing. It was evolving over time. About 10 days in, we saw this tweet from Craig Fugate. Some of you may know Craig directs FEMA. So not a nobody in the disaster response space and somebody who comes with decades of experience. And frankly, when I saw this tweet, I was like, are you kidding? Are you serious? Because this map that I just showed you was not put together by humanitarian professionals with decades or even years of experience, right? It was not done by FEMA. It was not done by the United Nations. It was not done by the Red Cross. It was done by a bunch of students who had never done anything like this before in Snowy Boston collaborating with dozens of other student-run situation rooms around the world. Entirely online digital humanitarian response. And yet they were referring to our map as the most up-to-date comprehensive map available to the humanitarian community. So this was rather surreal. Now, obviously, most people are not going to be on social media. And this was, I mean, especially two, five years ago. Obviously, there are many more today. And we knew that very night after the earthquake struck that if we wanted to have some kind of impact on the ground, we would want to necessarily go with SMS. Now about 80% of Haitian households have access to a mobile phone. It was not obviously that high, but it was 60% or 70% in 2010. So through a series of accidents and luck, we managed to set up a free SMS shortcode. The number was 4636. It allowed anybody in Haiti to text that number for free to share their urgent need and their location. Because obviously, it has to be actionable, saying that the orphanage, and this is a real text message, the specific orphanage was running out of water. It's all fine and well, but where is that orphanage in Haiti? Again, we didn't know whether this was going to work. But if it was going to work, then we could potentially be facing a lot of text messages coming at us, and not in English. And certainly not in, I'm not sure what, slides. This is a free version. We didn't know whether the text messages would not be coming into English or necessarily in French. In fact, the vast majority would be coming into Haitian Creole. And we didn't speak a word of Haitian Creole and neither did most of our humanitarian colleagues anyway. So a colleague of mine at Stanford University started recruiting Creole-speaking volunteers online. He went on Facebook primarily. There were a number of Facebook groups that had been set up by the Haitian diaspora. And they were self-organizing online themselves in support of the relief efforts. And we found hundreds of these Creole-speaking volunteers. And sure enough, we started getting the messages coming in. We had other partners in Port-au-Prince who got the word out through local community radio stations that kept broadcasting throughout the earthquake. At no point did they ever go offline. And so word got out about this 4636 number. We got the word out through the Haitian diaspora radio and television as well. And text messages started coming in. And these volunteers, these Creole-speaking volunteers from all around the world, started basically translating these messages. And we received tens of thousands during the first couple of weeks. And the average turnaround time of the translation was about 10 minutes. So literally 10 minutes after the text message left somebody's cell phone in Port-au-Prince, we had that same message translated in English and geo-referenced, ready for mapping on our platform. Here's the wordle of the most frequent words used in these text messages during the first two to three weeks. And one of the reasons I share this wordle is because there are two words that kind of strike me as please and thank you. Keep in mind, this is a population that's just gone through probably the most traumatic experience of their lives. They've most certainly lost one or more loved ones. And yet they're saying please. And yet they're saying thank you as well. It's a human story just as much as it is a technology story. We found out even later that the US Marine Corps had been making use of our map. Again, remember, when we launched this map, we didn't do that in partnership with the UN or any first responders. We just launched the map. Others eventually came and found out about the map. And an email that they allowed us to make public later on, they very clearly wrote that they had been using our map every day and that thanks to this map had been able to save hundreds of lives. This is according to them, not me. This is also surreal five years later when I share that with you. Again, we were in our PJs most of the time in Snowy Boston. And again, sort of making it up as we went along. It goes to show you that a lot is possible with these new technologies and importantly when people care. But the story that I just shared with you would not have been possible were it not for another equally remarkable story that is about this map. Because when the earthquake struck, half of Port-au-Prince was missing when you went on Google Maps. The reason being, why should Port-au-Prince be a priority for Google Inc? They've got more interesting things to do and more money to make elsewhere. The fact that half of Port-au-Prince was missing made it very, very difficult for us to georeference any of the tweets and text messages that we were monitoring. And because we couldn't look it up, we wouldn't know where the roads were. People on the ground did, but not necessarily us, a few hundreds of miles away. So another team called Humanitarian OpenStreetMap acquired satellite imagery from the World Bank and partners and basically crowdsourced the tracing of this satellite imagery up to date, the most recent satellite imagery available and created in a matter of days, crowdsourced in a matter of days, a map that was far more detailed than what Google Maps had provided. And this is just an animation. I'm not sure if you're gonna see it with the lighting, but you'll remember the earthquake striking on January 12th right now. And each flash of light that you see here is an individual edit by a volunteer, some volunteer all around the world, that cared and wanted to trace the map to provide a far more detailed map. It almost looks like synapses, right? Firing off in the brain, again a very organic, alive looking map. And it was thanks to this, these efforts, these digital humanitarian efforts with satellite imagery that a few days in we were starting to be able to map even more content and gone back to other content that we had to put to the side because we couldn't geo-reference it, now we could. So it was an invaluable contribution and Humanitarian Open Street Map remains to this day one of my favorite, all-time favorite digital humanitarian groups out there. So a lot has happened since the Haiti earthquake. And while this is more of a caricature in terms of a dichotomy, there is a massive change that's been going on ever since. Humanitarian organizations used to be familiar with a world like this one marked by information scarcity. The lack of information during disasters meant that it would take a lot longer to identify who had been affected, how badly, and where. And increasingly, again, it's not an on and off switch, but the flow is very, very much one way to this other kind of information environment characterized by total information overload. And frankly, humanitarian organizations are in no way capable or prepared for this other new climate. They are not data science organizations. They've never had to be. They've had to deal with a scarcity of information, not the overflow of information. And as they've realized since the Haiti earthquake is this overflow of information can be as paralyzing to humanitarian efforts as the lack of information. So they're actually no better off in either of these two worlds than they certainly need a lot of help, which is where digital humanitarians come in. The challenge is really one akin to finding the proverbial needle in a haystack. And that needle in our context are actionable, informative, potentially life-saving pieces of information, user-generated content. It could be satellite imagery as well, but going through billions of images, how do you find that one image that shows disaster damage, for example? They're completely overwhelmed. So what the question is, how do we find those needles in these haystacks? And how do we do that searching in near real time, not six weeks after a disaster? Well, what we did in Haiti was crowdsourcing, right? We got a bunch of our best friends and a bunch of strangers and we just rushed that haystack. And it could be a lot of fun for the first few minutes, but after a couple hours, not to mention days and weeks, this becomes just very tedious and a recipe for complete burnout. Doing this nonstop day and night is not fun and it's not efficient, as I think I showed you with the Haiti example. So what we're moving towards now to make sense of these giant haystacks of information and finding those needles is we're staying with crowdsourcing. There's an advantage to using crowdsourcing, but we're using crowdsourcing and we are powering crowdsourcing with artificial intelligence. So it's crowdsourcing powered by AI. And it sounds maybe somewhat science fiction-like, perhaps that the UN is using artificial intelligence for disaster response, but it is actually reality and I'll show you some concrete examples. From just a few months ago, Typhoon Ruby was initially a high-end category five Typhoon on the direct collision course with the Philippines. And we were activated, and I'll explain this a little more in a bit, by the UN to basically make sense of the haystack that was about to be generated, user-generated content on the ground. One of the reasons I'm sharing this particular tweet with you is because when I saw it in December, during the Typhoon, I thought back to Haiti and before the Haiti earthquake and a few months after the Haiti earthquake, when crowdsourcing was simply something that humanitarians did not understand, the single fastest way to discredit yourself at any humanitarian meeting in New York or in Geneva was to mention the word crowdsourcing. Especially if you were advocating crowdsourcing. If you were criticizing crowdsourcing, then you were the world's leading expert on information management for disaster response. Because of course crowdsourcing was useless and it was folly to think about crowdsourcing disaster response or crowdsourcing crisis information. I can tell you this from personal experience, especially at the UN in New York. That was the fastest way to end the conversation was to suggest perhaps looking at crowdsourcing. So to give the UN folks credit, they've come a very, very long way and we wouldn't be able to do this what we do as digital humanitarians. We're not for the change agents within these organizations. Now unlike five years ago in Haiti, we have something called the digital humanitarian network. And again to give credit to our humanitarian counterparts in Geneva, this digital humanitarian network was co-founded by the United Nations, by the Office for the Coordination of Humanitarian Affairs. And the purpose of this network is to provide traditional humanitarian organizations with a search capacity that they lack to make sense of big data. And this is increasingly serving as the official interface for that kind of collaboration between decentralized network-based collective action online and brick-and-mortar, top-down, hierarchical, slow-moving traditional humanitarian organizations on the ground. So the digital humanitarian network was activated just days before Typhoon Ruby made a landfall and the request received was to look at all the tweets being posted and identify which tweets referred to urgent needs and to look at the pictures being posted on Twitter and identify pictures that showed infrastructure damage in order to help accelerate damage assessments that the UN was carrying out. To do this, we deployed an experimental prototype platform called MicroMappers, which is free and open source and a joint initiative again with the United Nations. And the purpose of MicroMappers is to combine crowdsourcing or human computing, it's the scientific term that my colleagues use, to combine human computing and machine computing, i.e. artificial intelligence, in order to make sense of different types of data, not only text-based information, but videos, pictures, Instagram pictures posted on Twitter, for example, as well as imagery, satellite and aerial imagery. This is exactly what digital volunteers from all around the world saw when they rallied in response to Typhoon Ruby. You'd see a tweet. We'd automatically collected the tweet and this is fairly straightforward and then we would push the tweets to MicroMappers and this is the interface and volunteers would simply tag the tweet that correspond, the category that corresponded to the tweet, to the message in the tweet. Now if you're dealing with a few thousand tweets, a few hundred thousand tweets, that's all fine and well, but if you're talking about millions of tweets being posted in a matter of days, like the 20 million plus tweets posted during Hurricane Sandy in about five days, then you're not gonna get very far with crowdsourcing alone, right? Even if you had a million best friends who were ready to look through all these tweets with you and basically search that haystack with you, is that really the best use of human time, right? If you do a little bit of math, I hope I did it right, but it would take one person six years, nonstop 24 seven to go through those 20 million tweets and categorize them. Six years of human life just reading tweets is probably not the best use of our time. So if we have an alternate opportunity or solution, then we should use that and you might preempt this, the alternate opportunity is artificial intelligence, which is where ADER comes in, AIDR, which is a platform that my team and I have been developing, again, experimental prototype, partnership with the UN, ADER stands for artificial intelligence for disaster response. And what ADER is is AI engine, it's a smart computer, if you'd like to think about it that way, that sits behind micromappers and learns, it's volunteers are tagging these tweets. ADER uses something called statistical machine learning to recognize and to learn from the humans. They say, oh, I see, so these 17 tweets that have just been categorized as urgent need appear to have this same pattern to them. And what ADER then is able to do once it's learned enough is to automatically tag any future incoming tweets. And we've stress-tested the system and we can tag about two million tweets per hour, if need be, which is way more than Sandy generated. I mean, it's overkill at this point in the big data space, but we want to make sure that we can actually win the big data battle, not lose it in the future. And obviously, most people, again, are not gonna be on Twitter per se, especially compared to text messaging. So we've got an official partnership with UNICEF, they're receiving tens of thousands of text messages a week, and that's a figure that's growing as part of their UReport project in Zambia. And they're struggling how to classify and make sense of all these text messages. So we've extended ADER to something we're calling ADER SMS, but it's the same idea, combining crowdsourcing and machine learning, to automatically classify up to two million text messages an hour, if need be. And we just got an offer call last week with the World Food Program in Rome and they're sitting on the same problem. They tens of thousands, 60,000 text messages are sitting on and they are not particularly interested in going through that manually. So we hope to basically work with them as well to make sense of that information. Imagery or videos, multimedia content is becoming more and more important. And this is not, this is me sharing what I'm hearing from the American Red Cross, from FEMA, and other emergency management professionals. They, you know, a picture speaks a thousand words. And so the request we got from the UN in the wake of Typhoon Ruby was to also go through all the pictures being posted on Twitter. Not three months after the Typhoon, but as these pictures are being posted, identify them, post them to micro mappers and invite volunteers, digital humanitarians, to categorize, to tag the level of damage that they see in each picture. And that's really as simple as it is, unlike what we did in Haiti five years ago with the mess of the spreadsheets, right? It's single-click activism in LA and it actually does work, unlike what my colleague Yvgeny Moroza seems to think. And I can give another talk about that. But what we done then is basically as simple as this. If you can click like on a Facebook picture, you can be a digital humanitarian. And what we did is we showed every picture to at least five different volunteers because we want to ensure some level of data quality, right, and only if there was consensus that one particular picture, like say this last one here, showed severe damage, would we take that picture and then basically geo-reference it, put it on a map. And that's exactly what we did with hundreds of pictures, sharing that information, put it on a map, and that gave the UN a very rapid assessment. Quick and dirty is perhaps the best way to describe it, right? It's not perfect by any means, but the UN's definition of a two-week disaster assessment, a rapid assessment, is a two-week assessment. And that's not a criticism against the UN. They have to do this on the ground, carry out the surveys, assess the damage. But they realized full well that they would prefer to have something before two weeks, an initial sort of quick and dirty look, understanding as to how badly the damage is and where, which is precisely why they are partnering with digital humanitarian. This was a resulting Twitter map for Typhoon Ruby that was shared with the UN and shared publicly. The underlying data with the personal line and fine information that was only shared in private with humanitarian partners. So that gives you an example of artificial intelligence being used for disaster response and being driven by digital humanitarians around the world. Now, where are we going with micro mappers as a so-called next generation humanitarian technology? Well, there's not only tweets, right? And there's not only multimedia content. There's a lot more information out there that we need to make sense of. That includes satellite imagery. I've talked about social media and so on. Multimedia content, text messages, aerial imagery. I'll say a bit more about that in a bit. And of course, let's not forget mainstream news, which now, thanks to the GDEL platform, we have direct access to, basically all of the world's digital news, we have access to in our fingerprint. And that's certainly a source of big data as well. And we wanna make sense of all that information using a combination of human computing and machine computing, filtering that information from these different sources and layering that on one map. Because there have been a number of really interesting peer-reviewed studies in the past year that have showed that these different layers can help enrich other layers, right? Having social media map and layering and aerial image over that is hugely beneficial as well as other types of geodata. But there's still a challenge. Yes, we're finding those needles in those growing stacks of information, but what if those needles are not to be relied on, especially with social media, right? We can't trust everything that we see on social media. So how do we make sure that what we're collecting and putting on those maps is fairly accurate, at least fairly accurate, reasonably accurate? Well, this brings me to the red balloon challenge, which some of you may be, actually, how many of you are familiar with the red balloon challenge? So this was a while back now. Back in 2009, DARPA discreetly planted 10 red weather balloons like this one and put up $40,000 cash prize to the first person or team that would find the accurate location of these 10 red weather balloons. This is a great example of looking for needles in a haystack. We're talking about, what's the surface area of the US? Three, four million square miles? In three, four million square miles, you're looking for 10 little red weather balloons. So for those of you who have not heard about this challenge, how many weeks do you think, or if you wanna be bold, days, do you think it might have taken to look and find the accurate location of these 10 red weather balloons? I'll start with three weeks. So does anybody courageous enough to say less than three weeks? But you know the answer. Okay, two weeks, all right, do I hear one week? Yes, do I hear five days? If you're a bold crowd, do I hear three days? I don't know, some of you are cheating, I think. So a whole eight hours and 40 something minutes. And this was done entirely through crowdsourcing. The team, obviously from MIT, these guys keep winning everything, which is kind of boring now, but crowdsourced the search for these 10 balloons on social media. They never left the laptops. They never had to because we're increasingly human sensors all around the world with our cell phones. And so they use a very fun and smart strategy, which they refer to as time-critical crowdsourcing, to very, very rapidly identify eyewitnesses who might be walking by these balloons and basically have them take pictures and confirm the location of these red balloons. So this is pretty an amazing feat, right? Now what if we could use this for crisis events? This is a great example, an Instagram picture. There was a train crash outside Paris a couple years ago and here this person who lives, as he says, 20 meters from this crisis event, snaps a quick picture, puts it online, makes it searchable and findable on social media, on the World Wide Web, even if you're in Australia. So we're increasingly acting as digital witnesses around the world. So what if we could, instead of looking for red weather balloons, we could look for information relevant during disasters and certainly not relatively small events like this, but major category five hurricanes. Like Hurricane Sandy, you may remember the footage on CNN at the time. It was really, obviously, huge, huge. Okay, please laugh and reassure me that this is not true footage, right? No, no, no. Not exactly real, right? I think from a day after tomorrow or some, yeah. So, but this footage was being shared on social media during Hurricane Sandy and we know from other disasters thousands of years ago to today that there are rumors that spread during disasters and it's certainly not helpful. We need to find a way to verify them. This was another picture, one of my favorite, that was shared during Hurricane Sandy, all my favorite superheroes, right, in one. Now you don't need a PhD in computer science to tell you, hey, I think this picture is probably not authentic, right? But what if you'd seen this picture first? Might that gave you a pause? I mean, it gave me pause, I wouldn't know, right? What was really interesting during Sandy is there are at least two individuals that I was following who decided they took it upon themselves to start verifying rumors and debunking rumors or confirming unconfirmed reports. One was a journalist, one was a graphics expert and they set up their websites for that and explained, they would carefully explain why this picture was actually authentic and why this picture was not authentic. Those are two individuals and as far as I know they didn't even collaborate. So what if you actually had hundreds, if not thousands? Imagine what a billion people could do for you in terms of verification if they were all connected. This is a friendship network on Facebook and while the famous Kevin Bacon six degrees of separation is true offline, online that figure is closer to four and that's for a global map, right? But if you go to New York or DC obviously in urban areas we're even more closely socially connected. So all of us are maybe one or two hops away from somebody who's about to witness a disaster in DC. Just one or two hops going through one or two of you we can find that eyewitness. So could we use that same idea from the Red Balloon Challenge instead of searching for balloons we can search for truth. Sounds grandiose but maybe we could to verify content and rumors during disasters and that takes a new form of detective work. What we've done to help this along is we've set up the Verily Digital Detective Agency where we are looking to digital humanitarians and digital volunteers to become digital detectives and the platform they're using is the Verily platform and this is an example of a recent deployment we did with the BBC. They need to know whether this picture was taken in Syria or not. Now we focus with Verily only on natural hazards or at least we don't do any kind of graphic content, right? And we figured okay this is obviously about Syria but there's nothing graphic here so digital detectives are not gonna necessarily see something they don't want to see. The way that Verily works is with a verification question which very specifically has to be asked in the form of a yes or no question. So as you can see here, was this photo taken in Syria yes or no? And then you might see if you're up close but in the back perhaps not. There are two ways to answer this question. Yes because and no because. That because is really important. We're not just digging up and down what do you feel might be the answer. We want people to actually be digital detectives. We want people to go out and try and discover whether or not this picture was taken in Syria and to explain why. So not only are we trying to crowdsource the search or the recruitment of digital detectives and crowdsource the search for evidence to answer these verification questions we're also looking to crowdsource critical thinking. We wanna try and stop this mindless retweeting during disasters that helps spread rumors that is very unhelpful to everyone and get people to think, right? Now we don't expect folks to be miraculously adept at verifying information during disasters. Part of the verily mandate is to educate and create a more skilled online public sphere and that's why you have a help button up there and why that help button basically takes you to the content in this book which is a phenomenal piece of work by some of the world's leading experts in verification from BBC to humanitarian to story full and others. It is really the how-to guide for verification. What we've done with their permission and in collaboration with them is we've extracted all the nuggets, the how-tos from all these experts around the world. How do you verify a picture? How do you verify the authenticity of a video? How do you verify a tweet? We've taken that, put that into nice user-friendly little modules and so these digital detectives can learn new skills as they're actually supporting relief efforts around the world. So that's the verily platform. How much time do I have left? Zero. Ten minutes? Five minutes? Yeah, five minutes. Five minutes. All right, this is the last bit. So I mentioned satellite imagery and the phenomenal work that humanitarian and open street map continues to do to this day. Satellite imagery like other technologies do come with a host of challenges. So my European colleagues at the European Commission told me at the time after Typhoon Haiyan in 2013 that it took 64 hours for the satellites to be tasked for the imagery to be collected and processed, analyzed and shared with humanitarians on the ground. And 64 hours is a heck of a long time in disaster response. So is there a way to perhaps shorten that in using a different technology? And one technology that may help is the use of UAVs or non-lethal drones in disaster zones. This is footage from Typhoon Haiyan. There are many advantages of drones and perhaps I'll skip on the obvious one so that I can finish the slides. But if you're on the ground and you've got a trained team, you can capture the imagery in a matter of hours and add a fraction of the cost without all of the data sharing restrictions that we've seen happen with satellite imagery. So I founded the humanitarian UAV network right after Typhoon Haiyan. They've been seconded to the UN to support the relief efforts in Manila and the Philippines. And I was struck by the number of, the large number of UAV projects that were ongoing. We had never seen anything like that. Certainly not in Haiti, there were a couple, but not a dozen. So we decided to basically launch this humanitarian UAV network, which has since been endorsed by the United Nations. And the UN also sits on the advisory board together with the Red Cross, the World Bank, the European Commission leading UAV manufacturers as well as experts. And what we do is we match UAV experts and pilots with humanitarian organizations around the world within hours, if not just a few weeks ago, within half an hour of a request we got from the World Bank for drone pilots in Malawi, we were able to put them in touch with a vetted professional pilot. So this is not a hypothetical. Humanitarians are already starting to experiment with drones for disaster response. But they can't be everywhere at the same time, right? And in fact, they're increasingly overstretched, which is precisely why we created this crowd-sourced crisis map of aerial videos and aerial photos of disaster zones. Professional humanitarians can't be everywhere at the same time, but as the crowd, as the members of the public increasingly have access to these very low-cost quadcopters and so on, they can be the eyes and ears. Just like the UN and others are making sense of what they are using and leveraging social media during disasters, they can also make sense of this user-generated content. It's a bird's-eye view, it's aerial social media, if you'd like, but it helps to extend the eyes and ears of humanitarians on the ground. Of course, there are a number of major challenges and dangers with this, which is why we're actively promoting our code of conduct, which has been drafted together with humanitarian professionals, drone manufacturers, and drone experts as well. Members of the public can be a part of the solution just as much as they can be part of the problem, but we want to make them part of the solution, which is why we have this code of conduct and why we have this map. So, I'll just skip ahead. As the CTO of FEMA recently said, aerial imagery is bound to become a big data challenge. As more and more people around the world are sharing this imagery and the videos online, it's going to become a big data challenge. So we've partnered, we've continued working with our UN colleagues to crowdsource the analysis of aerial imagery as well. And again, looking to combine that with machine learning. I'll just skip this because it's over time. But to let you know that we piloted this new crowdsourced aerial imagery platform in Namibia with a wildlife reserve, because you ideally don't want to launch a new platform that's been untested in the middle of a disaster. If you can help it, it's not always an option. Sometimes you just have to jump in. But in this case, we were able to partner with a wildlife reserve. They were sitting on tens of thousands of aerial images of their wildlife reserve because they wanted to know what kind of animals they have, how many, in order to do better protection. There's a huge rhino poaching issue in Namibia. So we crowdsourced it. Something that would have taken the rangers themselves, and they told us, they would have taken us months to look through all the images when they actually have to be not behind a computer, but out in the bush, hopefully saving the wildlife. We were able to crowdsource all the images within less than 24 hours. So compare that 24 hours by the crowd versus two months, at least, by these rangers. And the volunteers had a blast looking for giraffes and rhinos and all kinds of things. And what we were able to do is then take the resulting traces and work with an EPFL, Polytechnic University in Lausanne, Switzerland, to create machine learning classifiers. So now, we can automatically identify gazelles or not during disasters, but in wildlife reserves, right? And the good news is that they are going to be doing this flying again in May, and we can then automatically identify the gazelles. And we want to try and identify other animals, too. But this is just a proof of concept, what we want to do with micro-mappers and aerial imagery. No, this is going to be way too long. So I'll just skip to that. You'll have to read the book. Planetary Response Network, it's something I'm very, very excited about. I want to end with this slide and just say, you know, when I, this is from Haiti, just to come back and close with Haiti. It's easy just to see dots on a map when you do crisis mapping. It's abstract. You forget that perhaps there are real people, real voices, real human beings behind that, right? And it's important to remind ourselves that this is more than just a technical exercise. It really is about humanity and human lives. When I also look at this map, I don't only think about the folks who are affected during disasters, but I think increasingly as well about the digital humanitarian. Because this map here, this very map would have been completely empty. There would be not a single red dot on this map if it weren't for people caring. If people didn't care, they wouldn't be using these technologies to try and help others in need. It's because people care that these maps exist. So again, this is as much a story about humanity and what it means to be human as it is about technology. So all these cruel and upsetting ways that we are using technology today and that you see in the headlines to kill each other, spy on each other, harass each other, it can be very dehumanizing. But I look at this and I see this as an example of technology extending our humanity. We don't only have a private emotion when we see bad news on television about an earthquake, we can actually get online and have some collective action happen to support relief efforts. It's a different world. So the faces behind the digital humanitarians is something that I try and remind myself all the time and why the cover of my book is all dozens and dozens of digital humanitarians who've made such a difference over the years. So that's it, that's my story. The digital humanitarians have been called digital Jedi by the United Nations, something I very much agree with. So I hope you're asking yourself, how do I become a digital Jedi? And I think I've hopefully convinced you that anybody who can get online can be a digital Jedi. And obviously I can't cover everything in my book, so there's a lot more. One of my areas that I'm very interested in is digital humanitarians who do what they do in countries under repressive rule, under authoritarian rule. It's phenomenally interesting and they're incredibly courageous. And one of my favorite parts of the book. So if you read the book, you'll find out why spam filters are important for disaster response. You'll find out about the very first crowdsource espionage that's taken place using satellite imagery and could have huge repercussions in the future. You'll find out why Putin hates this picture. And the Kremlin looked very, very bad during the Russian fires of 2010. And you'll find out what massive multiplayer online games have to do with disaster response and why these games may actually be the future of digital humanitarians this far. Thank you very much. Thank you, Patrick, for that presentation. And in case some of you came in late and missed, we do have copies of the book for sale out front after this discussion. So we'll start out, Patrick and I will talk for a bit and then open it up to questions from the audience. I wanted perhaps to start with this example of poaching since it was just the begin at the end and then go back towards the beginning. You mentioned sort of the effectiveness at a low cost, quite efficiently, of being able to find, say, rhinoceros in a large area, which can be very useful if you're trying to protect it from poaching. It can also be a great technique if you're a poacher. And I think that same capacity of the tool to be used in many ways applies to many of the other things you've mentioned. And I think the emphasis on the book is in laying out the ways in which these technologies can do good in the world. However, there is another aspect to it. And I was wondering if you could elaborate on the downsides that you see. We do have Guineas book and it's full of downsides if you haven't, the dark side of the net or something. But yeah, I'm more of an optimist than you have Guineas. And of course, my own doctoral dissertation a few years back was looking at how repressive regimes use technology to repress and also how civil resistance movements are using technology to circumvent that repression. And it's a bit of a cat and mouse game, right? Loopholes, circumvention and so on. And I don't think there's necessarily an end to that game but anything can be used for any kind of technology can be used, a car, a knife has multiple ways. So the way we use technology, I think, to quote a colleague of mine, Grisha Asmolov is simply a reflection of our ideals and our principles as society. And of course, we see all kinds of ideals and principles in our society. But at the end of the day, for me, the bad guys have all the money and the technology that they could possibly want. Super sophisticated drones, you name it, sadly, they can get, they buy satellite imagery if they need to. I mean, in fact- Which bad guys are that? The missionaries. It sounds like a Bondville in the way you describe it. Yeah, they do. The folks who are actually involved in poaching. I've been spending quite a bit of time with experts in this space because I'm interested in wanting to help out, see whether our technologies might be able to help. But folks who've actually been on the ground and told me, this is low intensity warfare. These are folks who've been fighting in all kinds of civil wars and wars around the world who are mercenaries and they have the latest, most expensive military equipment to go and hunt down these rhinos and these elephants. So they won't need anything that we're developing. They're far, far beyond that. What we're looking to do is try and maybe, even the playing field a bit. And of course, if we do ever get to a point where we develop a machine learning classifier for rhinos, we're not gonna make that public. We didn't make public where those rhinos are located based on the imagery that we tagged. That's totally private. And it's up to the rangers themselves to decide whether they wanna make that public in one form. So data privacy, data security is also a very big point in my book that I devote quite a bit of time to because obviously it's not something you can overlook. It's really, really key. You have to be careful. And it can certainly, things can backfire. There's no doubt about that. You mentioned sort of automatic classifier for rhinos or the automatic classifiers for the degree of damage that an image shows. Again, sort of along these same lines, automatic classification can be a very powerful way of getting through large amounts of data very quickly. It can also be if you're sort of coming up against an algorithm because your picture, there's severe damage to your house but it doesn't appear in the picture and you can't get past the sort of algorithmic gatekeeper. And I'm wondering, again, sort of what your thoughts on that are. Yeah, well, automation is gonna be the only way to make sense of big data or semi-automation is either we try and make sense of big data or we don't. And if we want to, we have to move to semi-automation. And I think there's a lot of ways that big data analytics can actually support good efforts that are in line with human rights principles and so on. It's true that there are issues with big data as well. There are bias issues, exclusivity issues. If you don't have a digital footprint, you don't count or you're potentially marginalized as well. There's no doubt about that. And I think the solution to that is not necessarily a technical solution. Some of the point that I really try and drive home in the book is there's only so much that technical solutions will give you. We also need enlightened policymaking, enlightened leadership and the appropriate laws and regulations and accountability measures in place in order to minimize the harm that can happen from the use of these technologies. That's the hard bit, if you ask me. The technology been the AI relative to the policies. That's the easy bit. They've been my colleagues who have PhDs in this computer science and so on have been doing this for years already. It's not new to them. But it's going to be about the social aspect of this and the politics and so on, which is a lot more messy, which is why we need advocacy, which is why we need more transparency and accountability in this thing. So those are both sort of questions about the potential harm that might occur from the same set of technologies. Conversely, another thing I'm curious about is not how things might be used badly by bad people, however we wish, which is I think a discussion that requires probably a great deal of political nuance that we're sort of skipping over for the present talk. However, also knowing that what you're doing is in fact making a difference, that the data you're gathering has an effect. You mentioned that would sort of struck out at me in the book that you had in fact commissioned yourself an outside study of your efforts in Haiti by a group at SICE and said that that study sort of had some different results, some similar results to what you thought. I was wondering if you could sort of discuss that process of how one goes about evaluating the effectiveness of these sorts of efforts. That's a really, really good question. So the study was done by Tulane University, I think it was, just to give them credit because they did a lot of work. And the results were very mixed. They were very mixed. Now granted, this is the evaluation, the impact assessment that was carried out a year and a half after the Haiti earthquake. And so they did go to Haiti about a year after the earthquake to try and interview people. But a lot happens after finding people, trying to interview people a year after a major disaster. So there were some challenges that they faced but it was mixed nonetheless, it was mixed results. And I think it's very fair to say that more than anything else, at least with respect to the Ushahidi platform and the crisis map there, it showed a potential. Again, it was very messy as I think I showed you guys, certainly not perfect. But it showed a potential and it showed that people cared and if you marry that care with technology, potentially really important things can happen. Less, and that was the evaluation specifically on the crisis map, not on open street map. The open street map was phenomenal and the impact they had was absolutely, it was very well done. And certainly as the book sort of narrates very lucidly, those technologies have evolved a great deal in the last five years. It's been five years. There's still a challenge on impact evaluation. It's really hard because we can't necessarily be on the ground and doing the evaluation ourselves. In fact, we shouldn't be. It should be an independent professional team that does that. The only reason we were able to do the evaluation or commission the evaluation in Haiti is because we had a very wealthy private individual who was so taken by what we had done in the basement at the Fletcher School at Tufts that she basically literally wrote a check for a lot of money and said, do whatever you want with it. And we said, well, the most important thing we can possibly do with all this money is commission somebody to do an independent evaluation to find out really what worked and what didn't work. And that whole check went directly to the team at Tulane University. Getting that kind of funding to do evaluations is very, very, very hard. And I would love nothing more to have that same team or other teams do evaluations of everything that we do from start to finish. If you know where to find the money, I'll make it happen. The best we can do right now, unfortunately is anecdotal. I'll add this, when a humanitarian organization wants to activate the digital humanitarian network, they have to fill out a form. They have to apply to actually activate the network. And we don't make it very hard. We don't make it easy for them to activate the digital humanitarian. They have to make a very, very compelling case that the data they want is actually data they really, really need and can't get any other way. And not only that, but if we did deliver the data that it would actually be used operationally on the ground. If they cannot convince us of that, we say no. And we have said no before and we will continue to say no if they can't actually, so the burden of proof is on their end, right? That's how we try and make sure that if we do create the data and we do mobilize hundreds of volunteers online, that there's a very, very good chance that they will be used in the field and hopefully have an impact. That's the best we can do right now in addition to then after a disaster saying, okay, how did you use the data? Where was it used? Frankly, for a humanitarian, they don't even know what the impact of their own projects are for the most part with traditional data, with nice, clean, you know, beautiful data sets, they don't know what their impact is as well. So there's a huge issue in terms of decision making within the humanitarian space. They understand the issue, they're trying to do something about it, but it's not because we're throwing social media at it that it's going to solve a dysfunctional, structural dysfunctional problem within the humanitarian. I have many more questions for Patrick, but I think the irony of giving a talk about crowdsourcing and not leaving time for questions from the audience would be a bit rich. So please do wait for the microphone to come to you because we're broadcasting this on the web. The gentleman in the back seems to have raised his hand. Oh, he has the mic. I actually would like to throw out as a fellow digital humanitarian, Patrick's book is a pretty awesome explanation of this. And I'd also like to throw back for him to respond, all of us who have been in this, we've always heard this threat that outside people are gonna use these systems to do something and it's just not happened. And part of it is because of the open nature, you're signing into open street map, you're identifying yourself, and in social media, emergency management or digital jedi-ness, being reputable is as important as the information you're conveying. I don't think I have anything to add. If Patrick doesn't have anything to add, I do very briefly, which would just be to say, we have seen a lot of discussion about the use of social media by say radical Islamist groups, which is part of this same broad discussion about this technology. So I think it's empirically false to say that we haven't seen any use of the sort of this set of technologies to do things that are generally not desirable in the world. Anyhow, on to the next question, here in the front. That's not how I understood your comment. Just, that's fine. I think, for example, we don't have any evidence that the Libya crisis map that I write about in the book was used by Qaddafi loyalists or anything. Also, another example, the Syria tracker, which is a Syria crisis map, the longest running crisis map that I know of. When I spoke with Syrian activists who were here in Washington DC a couple years back and told them, aren't you worried that this Syria crisis map is actually providing the regime with increased situational awareness? They literally laughed at me. From across, we were having dinner. They literally laughed at me and said, are you for real? Do you think that these guys actually need your stupid little map to know what's going on within Syria? They've got a phenomenal security apparatus. They'll know anything that happens way, way before you or any of your crisis mapping colleagues hear about it. So that's how I took your question, is again, with the poaching, these guys are fully funded. They don't need stupid little classifiers, too, to kill more animals. That said, you're absolutely right. I've also blogged extensively on how terrorists use social media for coordination purposes. It's absolutely true. Why wouldn't they? ISIS is using drones. They're using DJI phantoms to gain additional situational awareness. Yes, a spoon can do a lot of harm or it can feed a baby in Ethiopia. This is the issue I had, very public debate I had with Yevgeny years ago. Enough is enough with all these anecdotes. Yes, we know that it's a mouse and cat game. Let's go beyond that to empirical data and time series analysis and quantitative data and get more nuanced analysis there. But anyway, so we can go on and on with these ping pong anecdotal matches. Sorry, little vent. Hi, Ben Schneiderman, University of Maryland Computer Science Department. Thank you for terrific talk and for terrific work. I think it's really great what's going on. The questions also highlighted two of the important issues of the dangers of going negative and also the importance of valuation. I wanna add a third one to the table. I think also you're a wonderfully positive antidote for Evgeny Morozo, which is great. The question I like to have is, I think people were very much inspired by your positive attitude, your informed approach, your take action attitude. And I think that's great. So I want you to explore the microstructure of motivation as it applies in different situations because the red balloon to me is not a model to build on. It was a $40,000 payout that was propagated top down. And you've created a remarkable $4 million for more network of people who go to work. How is it that what you did could be copied by others? What is the essence of that motivational structure? That's a really, really great question. And you're absolutely right. The UN doesn't have $40,000 to throw out a verification question. But we don't need to. And one of the reasons I shared those verification examples that these two individuals did during Hurricane Sandy is that nobody paid them to do that. Nobody told them to do that. They did that because they felt this was their expertise and they could help, right? Because it's what makes, okay, I'll take a few steps back, because I'd like to think that wanting to help others in need is part of what makes us human. That's my eternal optimistic attitude that will continue to shine, no matter what. And I've seen that. I've had such a privileged position over the past five years to see this digital goodwill just appear out of thin air. Time and time again, people want to help. And if you provide them with an ethical way to help, they will help. It just, the book is the evidence of that. So it is, that's why I'm glad you asked a question. And again, it's a human story. There'd be no dots on that map, right? We don't need $40,000. Now, the challenge is a few months after the Haiti earthquake, I got a call from my World Bank colleagues who say, right, very, very nice Patrick, well done. Now, for these other issues, the slow burn crises, climate change, poverty and so on, how do you mobilize tens of thousands of digital volunteers to care about that? And it's a very, what I do in sudden onset disasters is easy compared to what others have to do with the chronic disasters that are happening around the world. Those don't get media attention. People don't react in the same way. It's harder for them to understand what's happening and they feel more removed from it. The example that I've given, because it's a question that has rightly and importantly come up in the past is, from what humanitarian open street map did in Indonesia. What they did is they went local. Somebody in the world cares about baseline data in Indonesia because it's important for disaster assessment and disaster response and disaster risk reduction and so on, but the folks actually living in these villages, in these cities and towns in Indonesia, it matters to them that their town is very well mapped on an open street map. It won't matter to a skateboarder dude in San Francisco as much, right? They won't feel as compelled to. So it's a different incentive structure. It's a different incentive mechanism when it comes to these slow burn crises. If you can find people who are affected by that, there's a feedback loop that exists and they see the situation getting worse and then they give them an opportunity to perhaps do something about that situation. The incentives are there. They're already there. It's a matter of finding. Finding. You said that the effort just appeared. You made it appear. Okay, let me. I would say it's better what it is. You did that. Others fail. No, no, no. It's nothing to do. No, no, no. It's nothing to do that. It's to do with the emotions because my wife and the other friends that we had in Port-au-Prince were all part of the Fletcher School. We were friends. We knew each other, right? There's a community there. The social ties, the social capital was already there. When some of our friends, my wife, all of a sudden basically went missing, there was a social network that cared about these individuals that could identify. We're all Fletcher School. We're all graduate students, right? We're all studying international affairs. We all want to make the world a better place. So I immediately went to that network. It was my immediate network and the ties were already there. And then it propagated, but it didn't jump all the time, right? It would be a visiting student from Geneva who called back in Geneva and said, hey, look what we're doing in Boston. So that social capital is really, really important. And then I think over time, this whole space has definitely taken a life of its own. And there's no way I can or would ever take credit for that. It was just that initial little emotional reaction when my wife got going missing, right? I think we'll move on. We'd only have time for maybe two more questions. And before we do, just briefly, because there have been a couple of pot shots taken against Yevgeny Morozov, whom I have no personal connection with. I met him in passing. His book, Patrick's book is very good. Yevgeny's book, To Save Everything, click here, I think is also very good. You may disagree with it. I think it's worth reading and I think it's not that he's sort of simply waving his hands and saying technology is bad. He's making some nuanced arguments that people might take issue with, but I would say it's worth reading his book as well. Anyhow, we'll take a couple of questions from the lady up front here. Is it on? Yes. Hi, I'm Alah Morrison with the World Bank. We are great fans of your work. So thank you for your presentation and look forward to reading your book. A couple of questions. One is I'm curious whether you explored the use, I'm sure you have explored the use of other types of data sets and what is the future that you see for other types of data like sensors and perhaps CDR data. So we'd be curious to hear that. And also as governments are starting to look into investing in big data, let's say if a government with limited resources is interested in putting some kind of infrastructure, policy capacity building in place into big data with the focus on emergency preparedness, what would you recommend they focus on? Thank you. I don't know that I can answer that second. I'm not qualified to answer that second question, but I'd love to brainstorm with you later. In terms of sensors, yeah, this is an exciting new area as well. The advantage with sensors that they're already in a way in a structured digital format. And so it makes it a lot easier for analysis compared to unstructured social media text messages and so on. And we're definitely gonna have to, that's gonna be another layer on that map that you know what I should really add. So thank you for mentioning it. It's gonna be another source of way, another way to help verify and enrich different layers. It's inevitable that this is gonna happen. And it's already happening. I haven't myself looked into that extensively, but just hired a colleague who will be looking at that extensively from the humanitarian development perspective. But there have been some interesting examples already in the ICT for development space and using sensors for monitoring and evaluation and so on. So I think it's an exciting new area. I'm not sure I have much more to add. And I'm not dodging the other question. I just probably need to think about it a bit more and chat. So you mentioned the Red Cross and the UN. How do you find those other organizations that you didn't mention to get them interested and make them understand why this is useful and why they wanna do it? That's a good question. It comes down to individuals. It's individual relationships. I've had the good fortune of working in the humanitarian space for a good 10 years and have built just professional connections and so on. I really wanna give these individuals a lot of credit. Folks within the UN and the American Red Cross and ICRC and others who get who are, it's unbiased, who get it. And the rest of the organization does not get it. And yet they're able to work the system sometimes to circumvent the system and ask for forgiveness later in order to collaborate with us. And that allows them to start demonstrating the value. They'll say, look, we were able to do this assessment in three hours and it would have taken us three days. And it's really thanks to these individuals and these pre-existing relationships. And then as our work of digital humanitarians, like again the phenomenal work that the Humanitarian Open-Street Map has done, that gets on the news and humanitarian organizations start using open street map and then others see that and they imitate, they mimic, they take that data on, they start collaborating with these different digital humanitarian groups as well. So it starts to spread. And I really didn't think we'd be here in 2015. I still, I thought it would take another five years to get where we are at in terms of being able to collaborate officially and directly with established humanitarian organizations. And it's in large, large, large part thanks to those individual change agents within these huge, traditional dinosaur organizations. I think we have time for one final question. And if there's anyone else with a very brief question, we can take them all together, otherwise. Thank you very much. Michael Zwerin from ADESO, a humanitarian organization based in Nairobi and a fellow Fletcher School alumnus. I have a question more about the slow onset disasters. You've already said that it's harder to get that emotional response that quick, everyone sit down on your computer and help out when there's a slow crisis, a famine, a drought, for organizations working in the developing world where these sort of things are more typical than a hurricane maybe or earthquake, how does one make advantage of these incredible tools that are being generated where the needs are vast but the timeframe is a much more diffuse? Really good question. And again, it's the hard question. So I partly answered that with respect to suggesting that the incentive mechanisms are there. We just have to find out how to tap into it in the most appropriate way. Let me answer it in a different way now and say maybe it's also a design question, a design issue. That last slide I showed was massive multiplayer online game called Eve Online with millions of gamers, right? And then you look at World of Warcraft and League of Legends and you've got hundreds of millions of gaming hours spent every week. Billions of months, billions of hours of human hours are going into these massive multiplayer online games every month. Imagine if we could just tap .001% of those hours and inject what are called human intelligent tasks or inject micro tasks within these massive multiplayer online games in a way that is not intrusive and is obviously opt in as well. You can throw any data, however boring the data is and work with game designers who are phenomenal individuals and incredibly creative and artistic and insert those tasks that way. Now, there's gonna be an announcement in two weeks time about what I write in the book with respect to gaming which I'm incredibly, incredibly excited about which will hopefully, in fact, we should definitely keep, they're looking for use cases. So please be in touch if you've got very boring, non-sexy data. Oh my God, I'm a poor kid in charge of the mining goal. I will also mention in response to your question and then I'll let Patrick have the last word. But Patrick made this analogy of a spoon and a spoon can do good things and a spoon can do bad things. And a spoon can eat soup, right? A spoon will not be so good if you have a sandwich. And some of the problems you mentioned like climate change, you can get all the world of Warcraft people in the world and they can't put a cap and trade system on carbon necessarily. So there are issues in which this set of tools can be tremendously powerful. There are also issues in which politics remains politics. Interest groups remain interest groups. Maybe those interest groups might be mobilized over Twitter on Facebook, but that's not a fundamental change in what the process of politics is. And I think that's, when you frame the question as how can these tools work, you might not be asking the right question for some of these things, in my view. So, and if Patrick disagrees, I'll let him have the last word. Oh, that's fair enough, absolutely, no. Thank you all for coming. I really, really appreciate it. Books are for sale. Do buy them, I think credit cards, cash also probably. And Patrick will sign them. Thank you very much. Thanks, Patrick.