 So it's really my pleasure, well, he's not late after all. So introduce Dr. Manisha Shah. She's assistant professor in the Department of Economics here at UC Irvine. She received a PhD at UC Berkeley and has been interested in the intersection of development, economics, applied microeconomics and health policy. Recently she's investigated the influence of economic incentives on risk behavior, choices and health outcomes of poor women in developing countries. She also conducts research in the area of child health and has written papers on how child development affects maternal labor supply decisions as well as the allocation of resources within a household. She's collected data from field work in Mexico, Ecuador, Brazil, Indonesia and India. So if we are nice, maybe she'll take us with her on one of her frequent trips. Today she's going to talk to us about her research, randomized evaluation of the life scale sanitation program in Indonesia. Welcome. Thank you. Thank you, Deli. And thanks for coming today. So I should mention also that I'm happy to take questions throughout the talk if you guys have any questions that come up. So as Deli mentioned, I'm going to be talking about some work that I've been doing actually the past few years in Indonesia looking at some of the impacts of a randomized impact evaluation of a sanitation intervention. And I should also mention that this is joint work with Lisa Cameron who's a colleague at Monash University in Melbourne. Okay, so just to give you a little motivation here about sort of child health issues in the developing world. I have some statistics here. One child dies every 15 seconds from diarrheal diseases in poor countries. Both diarrhea and acute lower respiratory infections account for more than 40% of the 10 million annual deaths that occur among young children in poor countries. And while we know that there is good evidence that improved sanitation decreases diarrheal diseases, we know that but if we look at things like sanitation measures, we still see that a large proportion of the world's poor actually lack access to improved sanitation facilities. So again, I have this estimate here that 60% of the world's poor or a little less than 3 billion people actually lack access to improved sanitation. And in the country where we're working in Indonesia, in the last year about 19 million people also lacked access to proper sanitation. And so if we think about the disease burden, the greatest disease burden of these lack of sort of access to sanitation fall on kids. So if we look at infants and children under five years old, the greatest burden of disease and death occurs amongst this population of children. And so of course the idea here is that diarrhea, by reducing normal food consumption and nutrient absorption, diarrheal diseases then can cause, a significant cause of malnutrition, leading to potential impaired physical growth, impaired cognitive development growth. And then there's also reduced resistance to other types of infections and all sorts of other long-term health impacts. For example, long-term GI disorders. So those are sort of the health impacts. If we also, I'm an economist, so I also like to think about a lot of the economic impacts. Though today this is a public health talk, so I'll be talking to you mostly about the public health outcomes we've been looking at, but just sort of as a little background and motivation, there's also a lot of important economic impacts we need to think about. There's just all the costs associated with illness. So people who are sick, they need to spend money to go see doctors, medical treatment, et cetera. And then there's also all the costs associated with productivity losses. So when kids are sick, they miss school. Miss school means the potential for long-term lower wages. So when adults are sick, they miss work. So there's productivity losses in terms of missed days from work, missed wages, et cetera. And so while I said there is pretty good evidence that sanitation decreases diarrhea, a lot of the current evidence that is out there of the health impacts of sort of sanitation and hygiene interventions, they come from studies that impose very controlled conditions in fairly small populations over short time periods. And so part of this study, I guess, is to kind of look at a much larger program, look at a larger population, look longer term. So we're going to be following these households a little over two years. And then the other nice thing about the study I'm going to tell you about is we have a really nice design where we have this randomized design. And so we're actually going to be able to say something a bit better in terms of, you know, we're going to argue that we have nice causal estimates of looking at the impacts of improved sanitation on health outcomes. Okay? And for those of you, I guess, who don't know where Indonesia is, this is Indonesia, and we're going to be doing work in the province of East Java, which is just this red circle here. All right, so let me give you a little background about what the actual intervention looks like, and then I'll tell you about what we did. So the program is called Total Sanitation and Sanitation Marketing, and this is a program that's being implemented by WSP, which is the Water and Sanitation Program, which is a group that basically comes under the umbrella of the World Bank, and they have different offices throughout the developing world in different countries. And so these guys have been doing, you know, water and sanitation work for a long time. But this program, so it's called TSSM, and in Indonesia it's actually called STOPS. This program is sort of a second generation program of CLTS, if any of you guys have heard of CLTS, it was Community-led Total Sanitation. And so the idea behind the program or what WSP was doing was they, you know, their sort of long-term goal is they want to roll out this program, the sanitation program throughout the country, but they started with the population of rural East Java, which is actually, you know, the third most populous province in Indonesia. And so they were piloting this program at a large scale. There's 25 million people that live in rural areas of East Java. And the goal of this program is basically to get communities to become ODF. So ODF stands for open defecation-free. And so there's still a lot of open defecation that goes on in rural communities, whether it's in agricultural fields. In East Java it's primarily in rivers, because most of these villages are, you know, they're based on these rivers, so people do a lot of their open defecation in the rivers. And so just to show you some data here, this is at our, you know, at baseline, we have about 65% of households that report to still practicing open defecation. And these bars are just, you know, we're going to be looking at different districts in East Java, so this is just the variation across the different districts. But, you know, you'll see that the prevalence of open defecation is still quite high. The other important thing to know about this program, TSSM, is that there's no funding for capital works. And so the idea here is that it's not that WSP is coming in and giving households money to build toilets. They don't do that. I guess, you know, a little historical background for why they don't actually do that is in the 1980s the Indonesian government under Suharto actually went in and built a lot of community toilets in rural villages in Indonesia. And they found that people didn't use them. They sort of fell apart. And so this new round of sanitation programs in Indonesia, that the idea is they want these things to be demand driven. You know, they want people to demand to, you know, to have change in their communities. But they want this to come from people themselves. You know, there's very much this feeling if we just go in and build people toilets, they won't use them, they won't care. They're actually spending their own money to build toilets. Then we'll actually see more of this sort of technology adoption happening. Okay? And so the, I guess what that leads us to is the intervention is pretty, you know, when we heard about the interventions, I was sort of thinking, is this even going to work? Because it's really very much just a lot of information. So the way the intervention works is some guy comes into your community and he spends a few days there. And the first day that he's there, he basically says, let's talk about your current sanitation practices. And so he, you know, there's Mr. So-and-So, where does your household defecate? Mrs. So-and-So, where does your household defecate? And then they sort of walk around the community and they point out where these households defecate. And there's, you know, they call this sort of the walk of shame. You know, the idea is that there's a lot of you and gross and now let's think about in a year how much, you know, you guys as a community how much poo you guys generate, basically. And then, you know, that's kind of the first part of it. And then it's like, look, you guys are generating all these poo. Now let's look at your kids. They're running around barefoot. They're ingesting feces. They're getting sick. You know, don't you want to do something about this? What kind of change can you do? And then the idea is that this facilitator, you know, this session is kind of called triggering. And the idea is that he's going to trigger change. And then what he does then is sort of spend time thinking about if you guys do want to change, what can you do? You know, like how can you build toilets? How much might it cost? Where could you get the supplies? And so that's basically the intervention. But there's no, you know, we're going to give you money to build toilets. It's if you want to build a toilet, let me help you plan how you might build a toilet. But you actually need to come up with the money to do that. So two things. So first, do we have good data on the incidence of viral disease per rate of open defecation in the community? I mean, I see that. That's a great question. Let's imagine that we were perfect public health professionals that have all the information. Is that information available? So no. And I think that's a really good question because we also went into this wondering, you know, in a way these people right now are pooing in rivers. And there is a question about if you actually build a latrine, what that does mean is that the poo then sits nearer to where you live. And if you're not cleaning that area properly, there is actually a potential where you could have, you know, kids, for example, could be more exposed to feces in this area versus if you're just going out walking to the river, you poo and it kind of goes away, right? It really goes. Exactly. And I can say as a population, we have a higher incidence that doesn't directly observe the negative health consequences. So that's one of the second things. Just to kind of emphasize your prior, you really wouldn't expect, I mean, there's a lot of kind of health behavior theory that would say we don't expect much at all with this because we're not giving anybody income. Exactly. And presumably they had some information before you came in. Exactly. And so that's exactly what we're going to, exactly. So we sort of walked into this thinking, is this even going to work? And I guess I should also give you the background of what we actually wanted to do was have an additional arm where we incentivize households where not only, you know, we sort of gave them the usual treatment, which was this information walk of shame thing, but we also said, and here's 50 bucks to actually build a toilet. But WSP was very against that. And, you know, because this is sort of the program they run, we were coming in to evaluate the program they run, and they didn't want us changing it, which was, you know, fair enough, right? And I guess, which then brings me to this point, we still had to convince them to randomize, right? They don't randomly allocate their program. And when we came in saying we want to do this intervention, and I should, let me step back also and give you a little more background about why we came in. So Bill and Melinda Gates Foundation is interested in investing a lot of money in water and sanitation right now. But it's sort of, they're not exactly sure where they should be as investing the money to get their biggest bang for their buck. And so what they've done is they've funded a few evaluations of sanitation programs in a few developing countries. So this project, I'm the PI in Indonesia, and they're funding this work. They're bringing another program in India, another one in Vietnam, another one in Tanzania. And so the idea is that they're going to look at all these evaluations, learn something about what works and what doesn't work, and then they can kind of go on and, you know, invest in sanitation. And so because of that also, you know, part of this was okay, we're evaluating this program. These are all, every country is evaluating programs that currently exist and are, you know, currently in the field. And so we, you know, we met with a lot of resistance from WSP in terms of trying to sort of change what they're doing. And, you know, but the nice thing that they did agree to do for us is that they did agree that for their phase two rollout, the part of the program that we, you know, the phase of the program that we would be evaluating, they did let us randomly choose villages. And so the nice thing here is that for the evaluation that we're doing, we're actually able to compare, you know, randomly chosen treatment villages with randomly chosen control villages. And so the idea with random selection, you know, we're going to show you that because we have this random selection, the idea here is that the treatment in control villages and the absence of this intervention would otherwise be exactly the same. Okay? And our data pretty much does back that up. And so what's nice about that then is we don't have to do kind of one of these pre-post studies where then we might worry that other village level characteristics or other household level characteristics might be driving the differences in health outcomes that we observe. Okay? Let me ask you a question about this scale. It's 20 million people in this region. And I don't remember the population of Indonesia. So what proportion of the country is the life scale? This, so the Indonesian, it's about 10% of the total population. So yeah, Indonesia, the size of the population is fairly similar to the size of the US. Right, and so East Java, there's 29 rural districts in total. And as I mentioned, the WSP was rolling out the program to all the rural districts. We got in in phase two. So we're evaluating phase two of the program. There were 10 districts that they had selected for phase two. And in the end, we have eight districts that are participating in the study. And that's primarily just because of timing and our surveying and et cetera, et cetera. And so what we basically did was we went to each of the districts and we said, you know, give us a list of anywhere from 30 to 70 villages where you're planning on, you know, running this program. And then we, so we got their list. And you know, most of the district offices between 40 and 70. And then what we did was from those lists, we randomly selected 20 villages where 10 of those villages became our treatment villages and 10 of those villages became our control villages. And then we kind of sent that information back to them and said, okay, now you can go and trigger in these treatment villages, but please try everything you can, you know, that's possible not to trigger in the control villages. I'll show you some data on compliance in a minute. And then what we did was we, you know, we had our survey firm that went into each of these villages and they, they basically mapped the universe of households with kids under five. And then we randomly sampled 13 households from each village, from each of our treatment and each of our control villages, which gives us about 2,100 households in the end who will be following over a few years. Okay. And so I guess I don't know if you guys can see this. I don't think you really can. We have an RA who does nice GIS work for us and made this, who made this map. And then it's just to show you, so this is the province of East Java and these are our eight districts that we went to and these, you know, the kind of, the diarrhea colors villages are, our treatment villages and then the kind of, the purple are our control villages. But we have, you know, this is just to kind of show you that we're going all over the province and we have pretty nice geographic heterogeneity. So time frames. So we, you know, we, we've been working on this project for quite a while. We got participation and agreement from all the districts in, you know, May through July, 2008. Then we went in, after we gave them the list of the treatment and control villages, we went in, we ran our baseline surveys starting in August, 2008. Ending in September, 2008. And then right in October, those facilitator guys go into our treatment villages. They do their triggering. And then we kind of let these households sit for a while, right? Because we're interested to see if they're actually going to build a toilet or what's going to happen. And so you can't kind of go follow up really quickly because it might take households a while to build these toilets. And so we, we sort of sit for over two years and then we go back in, you know, December, 2010 and we run our end line survey to see what's happened in between. And I should mention that in between we were going back every few months to collect a few, you know, little indicators. You know, did the program actually come into your village? Were you guys even treated? We also collect some diarrhea measures over time because there's, you know, diarrhea changes a lot depending on seasonal issues, et cetera. And, you know, and now we're just basically working on some of the final analysis. Sorry, I missed it. Were you able to give them the $50? No. We were not. We were not. But the findings from the study will be good motivation for why we should be able to go back and do that because I'm basically going to show you the main reason that people don't build toilets is because they don't have the money to do it. So the outcomes I'm going to show you today are primarily health related. And so the idea here is we collected various levels of health outcomes. So the biggest movement you would quickly see with kids under five is diarrhea, right? And so we have, you know, we have diarrhea. We've also collected fecal samples to get at this issue of do we see any changes in, you know, in parasites. And then if you, for example, observe these changes in diarrhea, well then you might think that you would also see changes in anemia, right? Because if kids are keeping more of their food in and there's more nutrient absorption going on and iron is increasing, then maybe you'd see sort of improvement in anemia. And again, similarly, you know, if there's less diarrhea happening, we collected information on high weight, arm circumference, head circumference. Again, with the idea that if, you know, these kids are getting healthier, you should see some movement here on stunting and wasting measures. And then this last, this is a much more longer term measure, right? So there's some evidence that, you know, you would see improvements in cognitive and motor development, again, if kids tend to be healthier. And so we had a psychologist working with us who developed this, there's this ages and stages questionnaire that's commonly used in rich countries. And so she sort of adapted it for the Indonesian version of it, okay? And so we, you know, we have, in terms of our baseline and end line survey, we have collected lots of information. We have all the general household level basic demography, labor market outcomes, income. And then of course, you know, things on water supply, sanitation facilities, sanitation behavior. And I'm going to show you lots of data from these surveys. For the kids under five, we collected all the health outcomes that I just mentioned in the previous slide. We have, as I mentioned, we also have this longitudinal data set where we went back every few months to see if the, you know, how kind of treatment and control communities. You know, you'd be worried, right, that we were picking up in this longitudinal survey that all of our control communities, for example, have been triggered or been treated. So we just wanted to keep an eye on what was going on there. And then we have the end line data set, which is basically the same as our baseline household survey, but we actually were able to also get some funding to collect fecal samples. So we didn't collect fecal samples in the baseline because it's pretty expensive to do that, but we were able to do that in the end line. And then we also have a community level survey, which again, I'll show you a few indicators from that. But here the idea was we just wanted to get a sense of, you know, at the community level, what kind of public goods are already being provided, what sanitation looks like, what water looks like, and what other existing programs are, you know, going on in the communities. Just to make, you know, because again, you'd be worried, say maybe there's other sanitation programs going on that are affecting these outcomes. And so we just want to be able to make sure that what we're measuring as our treatment effect is actually the effect of this program and not other things. So I'm not going to show you, you know, we have pages and pages of table in the paper showing that we do actually have very nice balance between our treatment and control households. You know, so one thing you would be worried about is if at baseline before the program happens that you have big differences in treatment and control households. But, you know, we don't actually, in fact, have very many significant differences. And so this is just to give you an idea, you know, before the program goes in, income of these households look the same. Access to sanitation looks the same. Diarrheal rates look the same. Anemia looks the same. All these other Z-scores that we're going to generate from our heart and weight measures look the same. And so it seems like we have pretty good balance across our treatment and control villages. I thought you picked the non-control, the therial color part of the map. Was it because they had higher incidence of therial? No, no, it was just, it was all... Exactly, we didn't... So the only thing we observable that we picked on was we needed to make sure that the households had children. And that was the only thing we stratified on. And then I think, you know, the other thing just attrition is a potential issue here, right? Is if we have... We clearly will have some attrition, but it matters again if you have differential attrition by treatment control status. So we spend some time looking at that in the data. And again, it doesn't seem to be the case. You know, so we have about 8.5% of our households we lose when we go into the follow-up. But we look at characteristics of these households that we lose and they don't look different across treatment control status. So it doesn't seem to be the case, you know, that a lot of our control villages are moving or something like that. That doesn't seem to be the case. And then, you know, we also... So we end up replacing these households. In the end line, we do another sampling of new households. And then the other thing we do is because our kids have aged out, you know, we're going in over two years later. And remember, we're interested in looking at these health impacts of 5 and a lot of our kids have kind of aged out of this group. So we also resample and get some more households with young kids in these same treatment and control villages. And so again, we need to make sure that these households don't look different than our households that are in the panel sample. And again, you know, I'm not going to show you all the tables right now, but we do find that the differences in characteristics of the new sample are not significantly different than our old households. Okay, so this is just to show you guys some data on, you know, where our treatment village is treated and where our control village actually controls. And so we have three different sources of data we use to measure this. So we have, you know, so the WFP data, this is actually the program guys who are going in and implementing the program. So they can tell us which programs they treated and which programs they didn't. So, you know, this is the first column here where, so each district, we have 10 treatment villages and 10 control villages. So according to their data, about 83% of our treatment villages were triggered and 4% of the control villages were also triggered. So there's 4% contamination here. But we also went in, you know, they're the guys implementing. And so in some sense, you kind of want an unbiased view of, you know, where who's being treated and who's not being treated. And so we have two different data sets. We have that longitudinal data set that I told you guys about where we were going in every few months to ask households. So if we look at that data set, we get that about, you know, 63% of our households in the treatment villages were treated and about 15% in the community villages were, sorry, in the control villages were treated. So we're getting both higher rates of contamination and lower rates of triggering in the treatment villages. And then we have another, you know, in our end line survey where we went in at the very end, we also asked households about whether or not they were treated. And there we get about 66% say they were treated and here we get about 14% remain control. So, you know, not bad, but clearly we don't have kind of 100% compliance here and 0% here. So they're, which again is, you know, in a real life program you would expect that. And then the other thing, this is just, you know, in this end line survey we asked households if they knew about triggering. And again, you'll see that, you know, there's not, this program isn't, it's not that, I don't know what the word is, not terribly strong and that only, you know, on average in our entire sample only about 15% of households report to knowledge of triggering. There's, you know, significant variation within each district. But you will see that it's not, you know, that say 100% of households that were in a village that were triggered actually even know about the triggering. Because part of this is, okay, the guy shows up for a few days in your village, but you might not be there that day, right? You might not even attend. And so part of this is also, in some sense it's a low exposure program. Okay, so this is our first, you know, before, okay, I go, I have until one, right? So before I get to any of these health impacts, you first need to know if there were any sanitation impacts, right? Because if I show you that there were health impacts, but there's no sort of first stage in that you don't actually observe anyone building toilets or doing any change, where these health impacts are coming from. And I've already told you that, you know, this is in some sense a kind of low exposure program. And so if we look, you know, our main, you know, sanitation outcome here is did you build a toilet or not in the past two years, right? And so this is basically, this top row here is that question. And so in our treatment villages, about 16% of households report to building a toilet in the past two years. And in our control villages, about 13% report to building a toilet in the past two years. And so the idea here that, you know, and then we just look at the difference in these two means and the difference, in fact, it is statistically significant. So households in treatment villages are about, you know, three percentage points more likely to build a toilet. And so that's about a 20% increase, you know, in toilet construction. So it's not big by any means. And, you know, it's fairly small, but I think given the program, it's sort of what you would expect too. And then what we do is we break this up. So by the way, these are just the numbers of toilets that were actually built. So then we say, okay, let's look at these households that built these toilets and break it up by people who actually had no sanitation previously, look at people who had unimproved sanitation previously, and then just people who had improved sanitation previously. So these are the guys who already have good sanitation but are just kind of building a new toilet, maybe because they're kind of the richer guys or they have the money to do it or whatever. And so here you'll notice that the greatest number of toilets are actually being built by people who already have improved sanitation. But the biggest difference across treatment and control is actually happening amongst those guys who did not have sanitation previously. So these are the guys, you know, in the treatment villages who are, you know, so here it's around 4% in the treatment versus 2.5% in the control. And this difference here is statistically significant, as is the one for guys who had unimproved sanitation previously. All right? Yes? So could you differentiate between no sanitation and unimproved sanitation? Mm-hmm. Yes, great question. I should have clarified that. So for improved sanitation we're using, there's this WHO UNICEF joint monitoring program definition of improved sanitation. And they basically say that it's improved sanitation when you have either a flush toilet or a latrine that either flows into a septic tank or a sewage system when it's either a ventilated improved pit latrine or when you have a pit latrine with a well that's covered by a concrete slab or when you have a composting toilet. So those are all improved sanitation and unimproved is basically anything else. And so in the context of Indonesia it might be. So a lot of people will have what they call hanging toilets where they'll build like an outhouse that hangs over the river so you go, you poo and then, you know, your poo goes straight into the river and it flows away. That would be an unimproved toilet. Or, you know, if you just have a latrine and the sewage is not going into a septic tank or kind of, you know, going somewhere where it should be. Okay. And so it is the case that, you know, a lot of people have unimproved sanitation but part of the program might be to then kind of get them to improve their sanitation by installing maybe a septic tank, for example. Okay, and then the other big sanitation outcome we look at is open defecation, right? So as I mentioned, the goal of this program is to get people to reduce rates of open defecation. And so here we also see some movement, you know, not huge but some movement. So in this first panel we look at everyone. This is everyone, this is control, this is treatment. And so what you can see is that men are now significantly less likely to defecate in the open as our children. So overall we're seeing this decrease in open defecation but it's mostly being driven by men and children and not the women. And then we actually, you know, separate this by river and not on river because as I mentioned a lot of the open defecation is actually happening when you have access to close rivers and you'll see that most of this change is actually being driven by these guys who live on rivers in that, you know, again, men are significantly less likely now to be defecating when they live near rivers as our kids. There's not much movement going on with the women we're not sure why but I think part of it is, you know, women go to the river as more of a social thing. They wash clothes there, they hang out there together and so, you know, because they're doing other things while they're there as well they might also be, you know, defecating but part of it is that they're still, they still have to go to the river to wash clothes and so maybe if they're going to poo they're still just going to go poo there, right? Whereas men and kids maybe now or it's easier for them to change their behavior. Just a matter of question. Yes. How internally valid is this question in terms of accurately relating to their behavior? You mean like because it's just a self-report and what should we believe with? Sure. I don't know. No, I don't know. Is there anything that people look at it? I know, I think that would entail some alterations but I mean I'm just... So, I mean the one thing I can tell you is there's actually not that much stigma associated with open defecation and so I'll show you in terms of attitudes towards open defecation. People aren't embarrassed by it because it's just something that they do. So it's not a social desirability of reporting. Exactly. So sexual behavior, alcohol... Exactly. So I don't think, you know, exactly. So in the context of, yeah, like sex and condom use you would definitely worry that people know you want them to say that you use condoms. Here I think it's less of an issue just because I think it's not that stigmatized but look, it's definitely... It is a self-report and so... And we didn't do any observational stuff for this. We do observational stuff for hand-washing which I'll tell you about in a minute. Quick question. Yes. So kind of on that end, you said you took long-term data and you also took, you know, snapshots of data at the end and the diarrhea rates you guys have got... Were those self-reported as well? Yes. Okay. Now, because open defecation may not have a stigma with it but diarrhea may. So did you find... Yeah. So I'm going to spend a while talking to you about our diarrhea results and whether or not they're real and we should believe them and this will be one issue that I will get to. But you're absolutely right. The diarrhea is also self-reported. Any other questions? Okay. We also, you know, there's a lot of data that I'm not going to be able to show you today but just to give you some guys some idea, we also collected information on hand-washing and drinking water. I should note that hand-washing and drinking water, drinking water definitely not part of the program at all so we shouldn't expect to see any big differences by treatment and control. And hand-washing too, it's not a huge part of the program. The program focuses much more on, you know, sanitation and building toilets but there is a little bit of, oh, and you should also wash your hands after you go to the bathroom. And so we did collect all that type of stuff and, you know, and if you ask people if they wash their hands after they go to the bathroom, 98% of people say they do. And then, you know, but for some of this we actually had. So they're, you know, and I think part of this might also be this issue that Tim brings up that everyone knows you should say, you know, that you wash your hands after you go to the bathroom but not necessarily everyone does every time. But we did have our enumerators go and actually look at hand-washing facilities. So, you know, we would say, do you have a hand-washing station? And a hand-washing station is just somewhere where you wash your hands after you go to the bathroom. And then if the household would say yes, the enumerator would actually go and say, can you show it to me? And they would look to see if there's soap there and if there's running water and what kind of stuff is there. And, you know, and water was available in most of the stations. So in less, you know, again, which is what you would expect. But there's no significant difference in hand-washing behavior or hand-washing infrastructure by treatment in control villages. And so, again, you wouldn't expect, if you see any differences into these health outcomes, it doesn't seem to be the case that it's because hand-washing is driving it. And the same goes for drinking water. Drinking, you know, interestingly, in rural Indonesia, households actually have pretty good access to good drinking water. So these guys aren't drinking water from the river. They have, you know, access to various sources of improved drinking water. And, again, we don't see any differences between treatment and control communities in drinking water. So, again, you know, if you see differences in health outcomes, it doesn't seem to be the case that it's because of access to drinking water. And then here, you know, we have, this is, again, getting at some of the issues of attitudes towards open defecation. You know, how do people feel about open defecation? And we don't see, I think there's no, yeah, there's no significant differences across treatment and control communities. So this, we asked them at the end line, after the treatment has already happened, and you see that there's no big differences in how people feel about open defecation. And you'll notice here, you know, that around 30% of people strongly agree or agree with the fact that it's acceptable to practice open defecation if you don't have a toilet or a train. But they also do seem to know that if you do have a toilet or facility in the village, it'll benefit the community, as environmental pollution will be lessened. You know, most people I know normally defecate in a toilet latrine. And so it seems to be the case that people do understand, you know, that defecating, that open defecating is bad or unhealthy, but that it seems to be fairly okay if you don't have a toilet or a latrine. What else? So we also asked Caregivers' Perception about causes of diarrhea to see, you know, is it the case that our treatment village is just no more about causes and perceptions of diarrhea? And this, so here we actually asked this question in the baseline as well as in the end line, and so we're able to do a difference in difference. And again, you'll find that in general for most of these, there isn't any significant difference in Caregivers' Perception about causes of diarrhea. Most people seem to have a fairly good sense about what causes diarrhea. The only two significantly different measures here, our treatment villages do seem to know that drinking unclean water is a cause of diarrhea. But if you believe this result, the treatment villages also think that exposure to sun is a cause of diarrhea. So these might just be kind of spurious results, but in general it seems to be the case that the treatment and control guys know about what these, you know, they have the knowledge, and I think they probably had the knowledge before this treatment even went in. In terms of, you know, at the end line we said for those guys who didn't build a latrine, why didn't you build a latrine? And, you know, close to 50% of households report the high cost. And so this, you know, we've sort of been pushing this result to WSP as a reason in terms of why they might actually think about, you know, giving these households some money to build a latrine because in fact it's not very costly. So WSP estimates that it costs between 50 and 90 US dollars to build one of these, you know, toilets, which isn't that much money. And what's interesting is we asked in the end line, we wanted to get a perception from these households like how costly they actually think it is to build a latrine. And, you know, on average they're reporting numbers around 135 dollars. So it does seem that households believe it costs a lot more than at least WSP estimates. It also, you know, some of the districts where we saw more construction or sort of more successful construction were providing access to cheap credit. And so this was happening, you know, within the community where community members were getting together and doing these sort of rotating savings credit unions where different village members all contribute and then every month someone gets to take on the pot of money. And so that was one way that villages dealt with the fact that they might have credit constraints and not be able to build a toilet. Okay, so let me, in the last 15 minutes I have, let me now move on to some health outcomes. So just to kind of summarize, it looks like we've had, you know, some sanitation impacts that have been significant increases in toilet building, significant decreases in ODF, but they haven't been that big, right? They've been, you know, okay. So let me show you now. We have diarrhea. So this is inline data treatment and control. So treatment is green and control is red. And when you see stars, that means that there's actually statistical difference there. So there's no significant difference between RRI and ARLI for treatment and control. But you'll notice that in the inline data, we do see that our treatment villages are significantly less likely to report having diarrhea prevalence in the last seven days. And we actually use, so the WHO, they have sort of symptom based measures of diarrhea. So we're not actually asking households, did your kid have diarrhea in the past? We do this for two days, seven days, 24 hours. We'd have various measures of diarrhea. But I should note, we're doing this, we have this sort of child health calendar of health things. And so for diarrhea, you get that you had diarrhea if you either report to having your kid having three bowel movements in the last 24 hours, watery or soft stool, or mucus or blood and stool. So these are symptom based definitions of diarrhea. And you'll notice that in all three of these, our treatment guys are less likely to have three bowel movements, they're less likely to have mucus or blood in school in the stool, and they're also less likely to have watery or soft stool. The mucus or blood and stool is the only statistical difference, but the data looks the way it should if you believe the diarrhea results. So we did it similar for ARI where it's all symptom based, but there's not much going on there. And then we also, we just wanted to have some sort of placebo effect health outcomes, right? So you wouldn't expect to see anything happening with skin itching from probably from a sanitation program or like abrasion, scrapes or bruising. And you don't actually see any significant difference going on there. Good question? Yes. The last one? Yes. I mean, looking at that, I mean, it doesn't seem like there's... I guess it doesn't look that significant, you know, and therefore, I mean, by your conclusion it says, I mean, latrines are just as likely to increase your congestion and to have a runny nose than it would be to, you know, reduce your risk of diarrhea. So let me show you the regress. So these are all means right now. I'm going to put all of this into regression analysis and then basically what you're going to see is that the only result that actually holds up is the diarrhea stuff and this will go away. And so then, but you know, just, we look at all these other health outcomes, right? So we look at anemia, hemoglobin, weight for age, height for age. And these are all just means, by the way, but there's just nothing going on, right? So the, you know, in terms of just looking at means of all these different health outcomes that we've measured, the only one that kind of sticks with us is the diarrhea result. But then we say, well, let's actually put this all into, you know, regression format. And so we're going to estimate these intentions to treat regressions. And so the idea here is we're just going to estimate the program impact off of a comparison of those communities who were treated relative to those communities who weren't treated. Okay, so it's just going to be this sort of average impact. And our Y is going to be our health outcome of interest or sanitation outcome of interest. T is just going to be a dummy for whether or not your village was treated. So we'll run this first, which is, you know, and we'll put in, you know, sub-district community fixed effects because that was the level of our randomization. And then we're going to cluster all of our villages, all of our standard errors at the village level. Again, because we randomized it at the village level. And then in other specifications, we'll also put in, you know, baseline controls for baseline and other different control variables. And so when we do that, you know, this is the, this is where the, so each of these is a different Y. So this is the dependent variable for when you're, you know, if you built a toilet in the past two years. And really it's kind of the strongest, the red means that it's statistically significant. And so all these other sanitation measures, it doesn't seem like that much is going on. And I should note that each column is just a slightly different specification with different, you know, sometimes we control for baseline characteristics. Sometimes we include other controls. Sometimes we have fixed effects. And it's just, you know, to kind of show you how robust these results are. And the big robust result seems to be that, you know, treatment villages are significantly more likely to build a toilet by around three percentage points. And that's about a 20% increase from the mean. The ODF result, remember that I showed you the means where we had differences, but once we put it into regressions, we're actually losing statistical significance. The only ODF result which we get is, you know, if you are on a river, you're less likely to engage in open defecation. And hand washing behavior, again, we're not getting much and we wouldn't expect to get that much because it wasn't a big component of the program. So here's our diarrhea. So as I mentioned, we have, you know, two-day diarrhea prevalence and seven-day diarrhea prevalence. And this is where, and then this is the ARI and ALRI. And again, you know, you'll notice that the diarrhea results, they're there, they're pretty strong, and they're pretty big. So the seven-day mean of diarrhea is around 4.6%. And so this is basically saying that, you know, if you're in a treatment village, you're 30% less likely to report to, you know, your kid having seven-day diarrhea prevalence. And then the two-day result is even bigger because the mean there was about 3.4. So this is about a 40% decrease in two-day diarrhea. And so the, and so let me just say, so we've had these sanitation impacts which are not that big, this diarrhea result which is pretty big. And so what we're, you know, what we've been spending time thinking about is, you know, are these results real? How much can we trust them? And so I want to spend five minutes now talking to you a little about what we're doing there. And I guess, you know, before I get to that though, I should just quickly show you that in terms of all the other different health outcomes we have, there's not that much going on with the health outcomes. We do, you know, we do see the mucus or blood and stool, the refusal to eat. All of these things you would imagine are probably correlated with diarrhea. There is something here a little bit weird going on with increases in congestion or runny nose that I'm not sure, you know, what is exactly going on there. But again, none of the, none of the high weight, anemia, cognitive development, nothing is going on. So we've, the only real big health impact we've seen from this program, if we believe it, is the diarrhea. And so the, you know, the important question I think that you guys have been pushing me on is these are self reports of diarrhea. How much can we trust them? And so as I mentioned to you in the end line, we did collect fecal samples. And those are actually being analyzed right now in the lab. So we have like half the data. We don't have the rest of it. And so, you know, if we were able to kind of corroborate the diarrhea results where we are finding that the fecal samples look a lot better for the kids in the treatment villages, then I think, you know, you, again, you'd be much more likely to believe these big diarrhea results. We've also looked at, you know, medical usage. So is it the fact, again, you know, you're seeing these big diarrhea results. So do we see kids in these treatment villages being taken more to the doctor? Or are they actually getting more treatment? And so what we do is we regress the medical clinic use on a treatment dummy. And we do actually find that the diarrhea families in treatment groups are significantly more likely to actually use medical clinics when their kids have diarrhea symptoms. So, you know, that's one thing that sort of corroborates the story we're telling. We're also, and this is something that we're doing, we have, you know, another reason you might see these big diarrhea impacts if they exist because of this externality issue, right? It could be the case that, sure, not tons of people are building toilets, but if there's sort of this multiplier effect of this sort of externality of people, you know, that having, you could observe these big decreases in diarrhea due to externalities. And so we have GPS data on households, and so what we're, we're kind of, we're basically looking at things, you know, like distances within households, distances to river. And we want to see if we can use some of that to actually explain these big decreases in diarrhea. And then the last thing we're doing, which I'll show you quickly, is we decided to look within districts, right? You could imagine that there's a lot of heterogeneity within district in terms of, you know, how people build toilets and how they behave. And so what we want to do is we want to look in the districts where we see lots of toilets being built, that's where you would expect the big diarrhea decreases to come from, right? And if you're observing big diarrhea decreases coming from districts where no toilets were built, then again, you might think that's a bit weird and maybe these diarrhea results are just furious or something. And so we do look at that, you know, so we look at each of these as a district, and we have, you know, our red is our treatment and our blue is our control. And, you know, I guess the only district where we're getting, where the difference, the increase in toilet construction here is significant, is Mario. But remember, these are small sample sizes and it's, you know, doing statistical tests with small sample sizes within districts. You know, there's, it's a bit harder, but if you, you know, you look, you know, you'll see that in most of our districts, I guess, except for Bondo Hueso and Blitar, you do see that our treatment villages are building more toilets. We do the same thing, you know, for ODF. And then what we do is we just, we generate this table, right? And so we say, okay, in Zhongbang, you know, we do see open defecation going down, nothing much going on with diarrhea. In Blitar, we see open defecation going down, diarrhea is decreasing. And so we do that for each district. And in general, I think most of the districts look how you would expect. Banyuwangi is the one weird district where there's no significant sanitation improvement, but you're seeing big decreases in, in diarrhea. And, but though it's not significant, you know, if you look at Banyuwangi, it does seem to be, you know, it is the case that more treatment villages are building, treatment households are building toilets. It's just that the difference isn't statistically significant. And so again, you know, it seems that if you look kind of within district, we are getting results that we would expect. But I think the big, the big open question and the big thing that will sort of help us solve this question of whether or not the diarrhea results are real are these fecal samples, right? Because the fecal samples, they're not self-reports. And so, you know, for, I guess, for, you know, if you're worried that all the people are misreporting their diarrhea symptoms, for example, we hope that, that that wouldn't be an issue with the fecal samples. So in reference to the stool samples. I hope you're not going to ask me which parasites they're testing for. Goodbye. I was thinking I should write this down, but I... It's all right. So, I mean, stool samples are, it's really temporal. I mean, if you take a snapshot, you know, one day to the next can be completely different. So I guess, you know, my thought in it is that that information would be almost worthless. Really? You're taking one snapshot. And I mean, whether, and depending on the location, depending on whether or not it's rained within the past 24 hours, there's a whole host. And also because you can't test for the whole slate of everything that it gives you sick, it just seems like there's so many variables in there that one sample before and after can't really tell you anything. Well, so look, I mean, we, you know, this is sort of... We are people who are consulting on this project to know much more about like health and parasites than I do have helped us think about, you know, which things they should be testing for, which I could definitely get you information on. But again, you're absolutely right that it's just one snapshot in time. And at the end of the day, that's really all that we could pay for. It's, you know, because as I'm sure you know, it's pretty expensive to collect who samples from kids and get them to a lab and do all of that stuff. So it's, you know... And there also, I mean, on average, would there be raining in the treatment communities and dry and control communities? Oh, no. That's the issue, right? So it's going to be a ton of measurement error and a lot of confounders to the extent that it's not contingent upon any geographic, you know, if your selection was unrelated to the geography and unrelated to these factors. You know, that's the only potentially... I mean, that could imagine that also. But you also talk about externalities of the link toilets. Would that occur if a more than one individual, one household uses the same toilet? Or if you have, like, a super-shutter deciding to use the toilet? So the one who actually ended up infecting everybody else decided to own up and get the toilet. And how would the data actually differentiate if it goes in a variety of other externalities? It wouldn't. I mean, I think all we're going to be able to do or all we can really look at is this sort of more average externality issue, but we won't be able to disaggregate between story one and story two that you just pulled. Do you know which one might be more compelling? I mean, based on what's happening in those communities and I'm being close about that. I mean, we could, you know... Is there any kind of communal notion of a toilet? Or is it really like a personal property? You know, this household buys this, you know... It's definitely much more of a personal property. People don't want community toilets because there's this whole issue of who's going to clean it and they get dirty. And you know, so people do want their toilet. I think there is maybe a little issue of their sympathies associated with having your own toilet. So I would say for the most part it is private use toilets that people are building. In a few cases, though, you do have, oh, we share our toilet with our neighbor because they're like close friends and, you know... And so then you get rid of some of these pre-writing issues that community toilets interface. Were there still samples from every child? Or only those with diabetes? No, every child. Every child under the age of five within a household. So what was the age that you were looking for in children? Oh, so it was... We have all the health stuff is for kids under five. Okay, kids under five. Yeah. So I mean, one thought is that you could... They're under five, so maybe the others are probably some absolute... But you could look at school attendance to try to back up. Yeah, so exactly. So we are... These are things that we're also looking at things like we have things on school attendance, how much time maybe moms are spending caring for kids, is that changing, you know, labor market issues. So we're absolutely... The last thing is, when you showed that the graph of, you know, when you break it down by district, you know, there were two districts in particular that had significant gains. Did you guys look into what sort of maybe secondary factors went into them adopting improved sanitation? Maybe a baseline they had, or maybe they have, you know, a high percentage education. Maybe they have greater financial resources so that, you know, given when they realize, oh, LaTrinck really helped, you know, they actually have the resources to put them in place. Right. I mean, we haven't yet, but we... It's definitely on the list of things to do in terms of, yes, someone was telling us an interesting paper would just be... Exactly. Like, why are some districts successful when others aren't? Yeah, because then you could really tailor an intervention directly to that. And then you bring the point up, you know, should we give them money? I mean, you could say, well, you know, this group has an average of $60, $70 income, you know, per person per year, and maybe you could help direct the intervention that way. Absolutely. Yeah. Tell me a question, would it, or do you want to finish? I can keep taking questions. I'm basically almost done, so... Yeah, Ben? Well, I have a quick question that might help with the difference between the communities. Yeah. Different facilitators were there. Oh, so that's a great question, right? So we... Because you're absolutely right in that you could think that facilitators are going to drive success and that if you had, like, a really charismatic guy coming into your... You know, because we actually observed a lot of the interventions, and it was true that some of these guys were great, and I thought, God, I would build a toilet, and then others were sort of like... And so we collected information on these facilitators in terms of, like, how charismatic do you think he was, and we haven't actually looked at that data yet, but that could also be one of these things predicting, you know, why people are building toilets versus not. Because we did... There were a lot of different facilitators, you know. I mean, every district has its own set of facilitators. It's not that these guys are going, you know, all across the province or anything. Jumping out that curious question, how many... Was there, you know, a ratio of, you know, facilitator per population? You know, because obviously going and talking to 100 people to close to 1,000 won't be a lot more effective. Yeah, yeah, yeah, no. There were something like 25 to 30 facilitators across the province. But I don't know. I mean, I could easily find out, though, you know, how many were in each district, for example. I just know the province number. Mm-hmm. I was struck by the difference between access to improved drinking water and lack of access to improved sanitation. Mm-hmm. Typically, they go ahead and... Yeah. So I assume that the drinking water is communal and it's chlorinated and so people go out with their buckets to a central location. And that's... It's free to them because it's provided by the government. Yeah, so the Indonesian government, for some reason that I don't know why, there's been lots more water stuff happening than sanitation stuff. And so you're absolutely right that water is not... Bad drinking water is not a huge issue in rural Indonesia. Because running water, some scholars have said that's the most important thing in the area for them. Mm-hmm. That's not supposed to whether you're using a pit latrine. If you have a pit latrine and you can push your hands, you're much better off. Right. And you'll notice, I mean, no one's raised this, but the diarrheal prevalence in East Java, actually, we were surprised at how low it was. I mean, it wasn't that high. That's not true in other provinces in Indonesia. You could go to poorer places where you would observe much higher rates of diarrhea. And part of, I think, why WSB decided to roll out the program first in East Java is because it's just an easier place to work in terms of the government actually functions and institutions function relatively well. And in some of these places where diarrheal prevalence is a lot higher, these places are also a lot poorer and it would have been much harder to roll out the program, which are things that they're going to have to deal with now, though, as they actually do go into these poorer rural areas. Any other questions? I guess I'm over time, so I should probably wrap up. But, yeah, I think I've, you know, said everything I needed to say, so. Thank you.