 All right, it's wonderful to be here at SoCAP 2019 and today what I want to do is really make the case for why rigorous data and evidence is absolutely integral to creating impact in the world. So presumably everyone in this room is bound together by a shared belief that millions of people lack the opportunity to live their fullest lives and that we're all here to change that. The main proposition that I want to make today is that the combination of great intentions plus incredible brilliance plus money is simply not enough to actually create the change that we want in this world and that instead we need to take a hard and rigorous look at the data and evidence in order to fulfill the joint aspirations of everyone in this room. To motivate this point I want to draw on a fairly well-known example of the Millennium Villages project. So Professor Jeffrey Sachs, who's a brilliant economist at Harvard and then at Columbia, thought that in order to raise the incomes of people across low-income countries in sub-Saharan Africa you need to have a big push of healthcare, agriculture, sanitation and education all at once and that would lift them out of a poverty trap. And so Professor Sachs raised a huge amount of money and by showing the bottom line on this graph which is that in the Millennium Villages asset ownership which is a sign of overall wealth rose fourfold over a three-year period. But what was emitted from the narrative was that all of rural Kenya, all of urban Kenya and all of Kenya as a whole was also growing at similar rates. And so when we take a second look at the data oftentimes really promising programs with a lot of money led by a brilliant team often don't pan out to create the impact that we actually want to have. Now against this narrative there's often the complaint that actually rigorously measuring impact is just super hard. It's too expensive, it's too time consuming and ultimately it deviates a social enterprise or business's core operations. And what I want to argue today is that while a lot of that is often true, we actually can rigorously measure impact in the context of impact investing in social enterprise and not only can we measure the impact but if we don't do it then we have a very high likelihood of failing at our shared mission. So I want to differentiate between two types of impact measurement. In the context of impact investing, folks often try to blend together what's good for business is good for impact. And while that's sometimes the case, I think we need to be more discerning about the fact that improving sales is not always tantamount to improving impact. But we can deploy rigorous impact measurement tools from randomized controlled trials to machine learning to answer each type of question. But the first and most important point is that we need to understand what are we making a decision for? Is it to improve our internal operations or is it to improve our overall impact? And depending on who the audience is and what the critical question is, we need to deploy impact measurement tools accordingly. So to take the first question around how do we actually use cutting edge data and evidence tools to improve business operations, I'm drawing on work that we did years ago with the social enterprise, international development enterprise IDE in Cambodia. So IDE and the Gates Foundation had designed a low-cost, high-quality latrine. And open defecation across the world is one of the leading causes of infant mortality. And so the goal was to try to market this low-cost latrine across Cambodia in order to decrease diarrheal deaths. And there were a huge number of questions that even once you've designed high-quality low-cost latrine, whether people would actually take it up. And IDE and the Gates Foundation were trying to debate between different methods to actually increase sales. So they're deciding between subsidies, doing a very sophisticated marketing and information campaign, as well as providing MFI loans to actually increase the uptake of these latrines. The problem is that there was very little evidence from Southeast Asia on which of these would be most impactful. And each strategy would have required substantial investment. And so before IDE actually made the decision of which one to scale up across Cambodia, they wanted rigorous evidence but on a fast timeline. So what we did was ran a randomized controlled trial, randomly selecting some villages to get the marketing campaign, other villages to get the microfinance loans for latrines. And what we found was that when you just have marketing, only 12 percent of households purchased a latrine. But when you add on financing, because folks face such a high liquidity constraint that even a $40 toilet was unaffordable, you see a huge jump in the sales as well as usage of latrines. And this type of rigorous evidence was generated in just four months at a very low cost. And with this confidence, IDE was able to invest in latrine financing to scale up across Cambodia. Now the important distinction here is that it's a great example of how you can use RCTs in a fast, cheap way in order to determine what is the best business path for you moving forward. But it's a very different question from whether or not people buying latrines and using them actually leads to decreases in infant mortality. And a lot of times in the impact investing sector, we tend to conflate purchase of really good products like this low cost latrine with whether or not that actually leads to improved lives. And I want to argue that it's useful to use these tools to make business decisions. And sometimes we need to do more rigorous evaluation in order to actually determine the effect that we're having on people's lives. And that you can do that rigorous measurement in a way that doesn't detract from the core business operations of social enterprises. So the next case study is D-Lite, which are solar lanterns in Uganda. And this is an example of how you can actually do a rigorous impact evaluation on the end outcomes we really care about, whether or not people's lives have improved in a way that doesn't fundamentally distract from a business enterprise. So the motivation of D-Lite was that if you can provide home solar energy systems to low income communities across rural Uganda, that would have the potential to improve educational outcomes, health outcomes, and income outcomes. And a lot of times you can just look at sales, but there's a huge question of whether or not people buying home energy systems would actually translate to improvements in health, education, and income. And so USAID and a number of other impact investors actually wanted to know whether or not these home solar systems led to improvements in people's lives beyond access to clean and modern energy. Now the big problem is that D-Lite really relied on a sophisticated marketing campaign that used social networks within communities to help spread their product. And so when you go to them with a proposition that you need to randomly select who will get your product and who won't in order to measure impact, that's simply not feasible from a business perspective. It would have required denying certain customers the product that wanted it, and it also would have fundamentally altered the way that they rolled out their marketing campaign. So the question is, do we just give up at that point and measure sales of energy systems or can we still find some rigorous evidence on what would have happened to these communities' lives if it weren't for the sales of D-Lite? So what we did was a statistical matching study where we identified 50 households that actually received and purchased D-Lite home solar systems, and then we went to nearby villages to identify comparison households that looked really similar to the profile of D-Lite customers. And by comparing comparison households, even though they weren't randomly selected, we can still get a fairly robust measure of whether or not the D-Lite home solar systems were actually improving people's lives. Now this type of methodology still has a lot of flaws. You could imagine that the folks that were buying these D-Lite systems were just the more entrepreneurial in the group, and that's actually what was leading to improvements in their household's education, income, and health. But if you can control for enough things, you can still get a really solid measure of what the impact of these D-Lite systems were compared to what would have happened in a way that doesn't totally distort the business operations. And what we found was that these D-Lite systems compared to the comparison group decreased upper respiratory disease by 9%, increased income, and also decreased household burns and fires. And based on this evidence, D-Lite received over $20 million in grants as well as concessionary loans. So what I want you to take away from this talk is that measuring impact is really hard, and there's often a narrative that it's going to be entirely disruptive to business operations. That's not the case, and you can often use it actually to improve your business operations. But when push comes to shove, sometimes there's a subset of cases in which just looking at sales or customers reached isn't enough. And in those cases, you do want to invest the money and time into properly understanding whether or not your social enterprise is improving the end outcomes that you really care about. And when you're in that situation, you can still do rigorous impact measurement in a way that doesn't break the bank and doesn't fundamentally change your core business. Thank you.