 Okay, I think we'll go ahead and get going. My name's Andrew Wilder. I'm the Vice President here of the Asia Center at the US Institute of Peace. And on behalf of USIP, welcome for taking time out of your busy calendars to come join us today for what I think will be a fascinating topic, a discussion on a topic near and dear to my heart. In a previous life, I actually had time to do research and interesting things like look at issues of stabilization in particular in Afghanistan. So I'm actually also very much looking forward to hearing about the findings from the study in greater detail and for the following discussion. For those of you not familiar with the US Institute of Peace, we were founded in 1984 as an independent national institute dedicated to the proposition that peace is possible, it's practical, and it's also essential both for national and global security. And we operate in conflict zones including in Afghanistan, which is one of our biggest programs where we basically work primarily with partners to provide resources and training and education and assistance to try to prevent, mitigate, and resolve violent conflict. Today I wanna just start off by thanking USAID for not only supporting this study, but also supporting a lot of the work that USIP's been doing over the years in Afghanistan as well as elsewhere. And I also wanted to give a particular shout out, I think, to USAID's willingness to really take on this study and basically allow a critical assessment and open up and provide a lot of the data that hadn't really been available before for such in-depth research to the ESOC team to look at to really try to understand the important findings. I mean, given the billions of dollars that have now been spent on stabilization efforts in Afghanistan alone, not alone in other contexts, I think it's really an important and timely study. I'd also like to thank the Empirical Study of Conflict Team, ESOC, for their work on this, a very valuable contribution, not only in terms of the quantitative empirical analysis, which ESOC is best known for, but also all the effort they put into reviewing a lot of the qualitative studies that have been done on this topic over the years. And I think the collaboration between USAID and USIP and ESOC has really been a fabulous one. And I hope you'll find the findings of the study and this collaboration to be interesting and fruitful. And with that, I will introduce our great panel today, starting with Radha Iyengar Plum. Thanks for being with us today. Radha is a senior economist at the RAND Corporation. Previously, she was on the Department of Energy staff team and prior to that worked at the National Security Council as the director for defense personnel, readiness and partnerships, amongst many other things she's done. She'll basically present some of the key findings from the study and then we'll have comments and additional presentations from Jason Foley, who's the assistant administrator for Afghanistan and Pakistan at USAID. Previously, Jason served as deputy assistant administrator in the Asia Bureau and as the director of strategic and program planning in the Bureau of Policy Planning and Learning. And after Jason will have Rob Jenkins, who's the deputy administrator for the Bureau of Democracy Conflict and Humanitarian Assistance in USAID, I know better known as Dacha. He also serves as executive director of the agency's task force on Syria and previously served as the director of the Office of Transition Initiatives. And then last but not least, we'll have our very own Dr. Bill Byrd, who's a senior expert on Afghanistan at USIP. Bill's had a long and distinguished career at the World Bank, including opening up the World Bank country office in Kabul in 2002 and serving as the first country manager for the years that followed 2002, I believe it was 2005. I see some of his successors in the audience today, so I'm glad you could join us as well. And then actually, when we get following the presentations, we'll go into question and answer. And for there, I'm gonna also ask Dr. Jake Shapiro, who's with us today to join us so that the really tough questions we can all point in his direction. Jake is a professor of politics in international affairs at Princeton and very relevant for today's presentation. He's also the director of ESOC, so welcome, welcome, Jake. And with that, I'll turn it over to you, Rada. So thanks so much. I'm gonna just slide this a little closer. Thanks a lot for that. And so what I thought I'd do is kind of run through the study findings and some key themes, but before I do that, I just wanna take a moment to say that research like this doesn't happen in a vacuum and we're very grateful for USAID's partnership and support in this and I'll actually talk a lot more about that during the discussion here, but also for the whole research team at ESOC. So Jake provided a lot of guidance and support, but also Manu Singh and Ben Christman, who are sitting all the way in the back as far away as possible, so I can't point to them during Q&A, but they're back there and they did the lion's share of tremendous amount of data cleaning, working, processing, and analysis on some really tough datasets and some really tight timelines. So I'm gonna stand up here and take the hard questions, but really they did a lot of the hard work. So as was already mentioned, we have spent a lot of money in the US government and actually as a global community in places like Iraq and Afghanistan, actually in lots of other places, we're increasingly in North Africa, other parts of the Middle East, and what we're trying to do is figure out how do we support the promotion of peace, stability, and legitimately elected government so that we have some sort of political architecture going forward, right? We have a lot of tools to do that, and we have a lot of tools that we try to address a number of underlying issues with. We have military tools and we have operational activity, we have diplomatic and political tools, and we have increasingly this sort of development and stabilization toolkit. So given that there's a lot of time and money that goes into all of these different operations, and certainly over the last 15 years, a lot of the focus in Afghanistan, there's a lot of questions on how do we think about what we did with those dollars, right, was it effective, what can we learn from it, and in general, I like to sort of summarize it as what worked where and why. And there are a lot of open questions there, right, that we're still grappling with, what do we do in Afghanistan, how do we think about this in the strategic review context, but also what lessons can we port from Afghanistan to other places like Syria or West Africa, North Africa, are there lessons to be learned in for future conflicts into where to invest and not? So that's really where we sort of started this project, was thinking about these open questions, and we were very fortunate to have a willing partner in USAID on this. So they did take the proactive step to open up a tremendous amount of data, both that they had and that were, they served as a sort of conduit with partners to help us get access to that data, which was a lot of effort on their part. But they also opened up their staff expertise to us to talk us through what do these data's actually, what do we see in this data? How are we collecting it? What were the challenges in collecting it? What were the challenges in implementing? How can we interpret these data? Why didn't you collect this kind of data? What are the realistic constraints? And I think on the back end of a lessons learned study where it's easy to sort of poke and point, it's really important to understand a lot of the feasibility constraints so we can think about what lessons can actually be learned and implemented. I'm gonna talk about that on the back end of this. So what we did was compiled a range of data from lots of sources. So I mentioned already administrative data on stabilization programs. Those are US programs like USAID, also programs from the Department of Defense like the CERP program. Also programs from international actors like World Bank's National Solidarity Program. We also got survey data on attitudes among Afghans, largely from the NCAR survey. We collected and, or we processed and analyzed satellite and imagery data on things like nightlights and land scan data to get better senses of population estimates. And we also processed a host of data sources with information on geographic features like where military bases were located, what the road networks looked like as well as sort of local demographic and ethnographic information. So that's the quantitative piece but we wanted to sort of buttress this with two other things. And the second thing we did was review about 90 studies, 89 to be precise. And these ranged from government reports to think tanks to academic journals, really looking at the broad range of things that were aimed at assessing, evaluating or reviewing stabilization in Afghanistan. We actually code these up systematically and try to analyze them in the context of what we're seeing in the quantitative data strut. The third thing we did was interview about 15 different program staff that were involved with implementing and executing USAID programs between 2009 and 2014. And again, I really wanna sort of tip my hat here to USAID because nearly to the person, everyone we talked to was open, candid, clear in what they think worked but also really clear in what they thought the constraints and difficulties were in the operating environment that they were asked to execute programs in between 2009 and 2014. And that really shaped and informed how we thought about the quantitative trends data and how we thought about the match between the trends we were seeing and the research review, right? So this is not an easy thing to do and it was not something that would have been feasible without a very sort of candid working relationship. I'm gonna talk through our results here and to do that and since I didn't wanna have a bunch of slides and be like a super boring economist, what I thought I'd do was kind of talk us through in three big theme buckets. So the first big theme that I'll try to talk us through is that stabilization is just really hard. Like it's just, it's in difficult places and violent conditions and even in the best cases it's buying you a little bit of time and space with games that are relatively modest relative to the macro trends. Now, if I were to opine a little bit on that and I'll take the liberty of doing that at the very end, I think that means that stabilization programs can be necessary for a strategy but they're not sufficient and they need to be integrated into some kind of overall plan, a political strategy, a military operation, a broader development project, something that allows you to sustain and build off of those gains otherwise you're just gonna get these modest gains and they're gonna retreat. So I'll try to weave that into the results. I hope we get to talk about it a little bit more in the Q and A and I know that that's something that my colleagues at USAID really think about a lot when they're thinking about these programs. The second big theme is that we're operating in a dynamic environment, right? So this is not you do a program and you've got constraints that are static and physical constraints. These are dynamic, responsive actors who want to win just like we do and that's really important because that means that insurgents are gonna try to do things like destroy the projects or take credit for the projects and so projects which are really hard to execute, see theme one, are actually pretty easy to target, right? Being destructive and sort of cheating around here is not necessarily, it doesn't necessarily always face all of the same constraints and can be really harmful even if the destruction isn't fully successful. And so, and that point gets even more relevant when if insurgents are really clever and what they do instead is just try to take credit for the activities, right? So the implementers go through all of the hard work of actually getting a program that has an impact and now it's helping the other team. And so if you have a theory of change that relies on things like hearts and minds or community support, you really wanna think through how are we marketing or branding or getting the credit where credit is due on these types of programs and that's a sort of broader strategic implementation aspect that can't be the responsibility, day-to-day responsibility of the people who are already doing an enormous task of implementing programs on the ground. And the third big theme and I'm sure that everyone is gonna be shocked that I'm a researcher up here and I'm gonna tell you that we need better data and measurement and research Not surprisingly, I think we do and I think a key to this project working was having an enormous amount of support from USAID to get the data and that happened sort of after the fact and with a real eye towards trying to figure out what lessons can be learned and in part that was made feasible by the fact that USAID actually collects a ton of data. They do lots of monitoring and evaluation on a day-to-day basis in addition to this sort of accounting fiduciary responsibility data that they have to collect and that's great for your sort of day-to-day programmatic reviews. But there's a sort of higher level sort of 10,000 foot strategic review that can happen not on a day-to-day basis but on a more regular and real-time strategic basis that can help inform where the priority should be in this overall campaign or fight. And this really links to this first question of stabilization is hard. It needs to be nested in a bigger strategy and this type of strategic review of how a stabilization fitting into our broader strategic goals can happen. But to happen effectively, it needs some slightly different characteristics than what the types of things you would collect or analyze for either program reviews or accounting purposes. So with that, let me try to turn to talking about the actual key findings. Just before I do that and a little bit of a housekeeping point in some ways but I think one that's gonna be relevant for our discussion later is defining stabilization is not actually a trivial process, it turns out. It changes a lot over time. What counts as stabilization and even how specific programs are framed changes over time. We took a relatively agnostic approach trying to look at the literature and see how it defined it. We actually came up with, I think about 200 different indicators that are used across all of the different studies to measure stabilization outcomes. We end up picking sort of broadly five which are violence, public support for the government, Afghan government, public support for anti-government elements, community cohesion and economic outcomes as measured by nightlights. We also include access to health services as a sort of proxy for improvements in health outcomes. So I wanna be very sort of gentle in thinking of that as health improvements or even wellbeing improvements. I do wanna say that overall, we find that stabilization programming was actually correlated with a small short-term improvement in lots of things like support for the Afghan government and economic activities, despite the fact that these are operating in pretty violent areas. Now, this positive correlation is primarily observed for relatively low-dollar programs and in some cases those programs are executed nationwide, but they're small programs. These aren't huge infrastructure expansive programs. I also wanna emphasize that these are trends and we can't and really we shouldn't interpret these as sort of the causally identified precise impact estimate for any individual program at any individual point in time. What we're trying to do here is tell you, here are sort of the broad macro trends in these big outcomes. Here are where programmatic activity was more or less intensive at various points in time. And here's how we can think about interpreting that temporally in terms of overall improvement. So the correlations that we see look like they're largely consistent with existing research, which also suggests that smaller-scale programs are easier to administer. But also we get this kind of additional finding that comes from the research, which is part of the success is driven by the fact that these smaller programs are less likely to be targeted by anti-government actors. So the key trade-off here is for government and decision-makers you can reliably execute these programs, but you typically can't affect the key drivers of instability. And that really sort of takes us to that first key theme, which is that, and this is not really news to folks here, that stabilization is really hard. And so in practice, that means you can expect stabilization programs to have huge impacts. We're talking here about maybe a fraction of a standard deviation on the overall macro trends for indicators that are happening. And you can expect the effects to be super persistent. So we're talking six to 12 months of effects here. So, and that's a shockingly consistent result in both our findings and in the research, right? You can look at well-designed experimental and quasi-experimental approaches. There's the National Solidarity Program. USAID has the MISTI study. And those effects are, whatever those effects are, relatively transitory. Now some of these effects happen because of the sort of local community-driven community attitudes about the programs. But all of the programs that show effects are relatively small, right? And so by small, I mean these little programs with smaller activities that are locally spanned. And in part, that's because those are harder to target. They don't tend to disrupt local systems quite as much. And it's harder for anti-government elements to take credit for them. And I'm gonna come back to that. But these projects are also really hard to administer. And so this is a non-trivial trade-off for decision-makers in terms of thinking about the timeframe and scope, but also in terms of thinking about the value, the cost-benefit value of investing in stabilization, which is their valuable investments, their short-term investments to value the time and space to get to your political or development or military goals. But you're wasting your money if you don't nest them into that broader program. Now I said something about how these small programs are relatively less disruptive. And I mean by that, that they disrupt local political bargains less. They provide fewer high-profile, really high-value targeting. They also provide smaller things that insurgents find less individually valuable to take credit for. And there's also, sorry, a little bit less opportunity for bribery. So part of the strategic decision-making on nesting this in the overall political or military or development strategy is thinking about how to market or brand this. And that's a tricky question that likely evolves over time, likely changes with where we are in the strategy. But I'd point out that USAID has some pretty successful examples of this. There's the ASI program that ensured really local government and non-governmental organizations were centrally and visibly engaged in the implementation process, which really helped in not just making the programs themselves successful, but the overall implementation. And the last theme that I will shock no one again is that it does need to be this sort of broader strategic view to monitoring and evaluation. That is not, does this individual program work or not? And not, can we account for these funds and sort of survive an audit? All of these things are, again, extremely detailed monitoring and evaluation data that has been collected could help answer that. What I mean is at a higher level, at a strategic level, how are these stabilization programs and the gains they're achieving and the money that's being spent meeting with our overall program or operational objectives? And how do we think about how to fit any changes, adaptations, reprioritization of resources to get the biggest policy bang for our buck on stabilization programs? And again, we're suggesting here a really pretty difficult task of creating common outcomes, collected over time, analyzed on a regular basis as a non-trivial cost investment and it's a non-trivial time investment for the implementers. We think that that sort of idea of improving the monitoring and evaluation could really help get some of the insights we have here on smaller is better, what types of programs are operating, how they're contributing to what outcomes and how that matches with the strategic objectives could really help get folks kind of across the goal line. And especially help make sure that the dollars and efforts spent are captured and capitalized upon in real time. So I want to kind of wrap there and turn it over to Jason and Rob, but I want to end with a little bit of a human note. So we tend to talk here about programs and I did it here in spending and USAID is a behemoth and implementers as a group, but my one sort of biggest takeaway from the study after hours spent with data and interviews and research is just the tremendous number of really hardworking people that are trying their very best to make things better under nearly impossible circumstances. So there are folks here in DC, there are folks that mission at headquarters, there are civilians and military and contractors and a lot of these people have been doing this for a really long time. And I think when we sit here and we sort of talk about programs and I make a bunch of recommendations on smaller is better or this corrupt or at a macro level it's worth calling out that they're doing this good work and not just for like they do good work let's all applaud them and like thank USAID but this is a really hard problem and these aren't stupid people. These aren't people who haven't tried to learn by what they're doing. They're not like hanging out in Kabul with their feet up on a chair drinking a coffee. Like there are folks working really hard in difficult circumstances. And I think as we think about that having these sort of outside approaches on how we can feasibly feed information into the system to make them feel like what they've done and what they've spent a tremendous amount of time on is getting the biggest impact possible is really I think the sort of big takeaway from the study that that is the value of trying to have a strategic level review. So I hope that some of what we do here will be useful for them. And I hope that some of this information will be taken on board to improve systems for USAID. So those folks can get the support they need as they continue to fight the good fight. So I'll wrap there. Thanks, Radha. Jason. Great, hi. I'll just take two quick minutes. But first want to thank USIP and particularly ESOC for all their dedication, their assistance, their time and effort. And when we made this decision to do this we took a hard step which is let's actually learn from what happened. And typically in Washington I don't think that's common because people are afraid of what will be found but we've spent too much, we've lost too many lives, spent too much sort of treasure not to be honest for ourselves, the American people, the rest about what happened, the best we can tell and importantly what does that mean going forward in other contexts. I want to touch on a point, a couple of points that Radha made. One is this notion of the report did find some gains in health, economic, improve of the perception, legitimacy of the government and that these were short term generally. And I think that's a really important point which the study then didn't get into which is how were those gains leveraged and for what impact by broader national security policy goals and that assumes that those are clearly defined. If you have 200 explanations or indicators of stabilization well you're sort of trying to, well there's this big thing, this big policy objective and how do we measure it? If you have 200 it's questionable how clear and defined the policy goal was or is. And third then is this point on how do we learn best? And so we do a tremendous amount of monitoring evaluation as Radha mentioned but we do think there's a lot of value in working across this government with partners to better understand going forward in these environments how do we pull up this data in a more strategic way so that policy makers have evidence to make their decisions? I mean that's what we're trying to do here so I'll stop there. Thank you very much. I find myself in a familiar situation where I'm a generalist among experts so I'm gonna focus a little bit not on Afghanistan specifically but what does this study tell us about what we did or did not know and then looking forward and how do we use these lessons for places like Syria and Libya and the Greater Lake Chad Basin, Somalia and other things. I just wanna note that I often say I've been elevated to my level of incompetence but I got a further promotion today from Andrew. I'm not a deputy administrator. I'm a deputy assistant administrator. I don't know why anyone would ever make that mistake because there's so much that makes sense in our titles but. I think you deserved. Thanks. So I do think, I think the study actually validates quite a few things that many of us have been focused on for a long time so as I go through a quick list of those things it might sound like they're just truisms but I think sometimes truisms need to be repeated because they're often not listened to when folks are saying but this problem set is different and as you noted this is really hard, right? But as a great American once said we do these things not because they're easy but because they are hard and the only reason we try them is because it's of incredible national importance that we try to get something right. So all of this is in that sort of general context. My quick points would be all right so you're gonna do this in another country. You wanna stabilize it. Already said, we haven't even agreed what that means sort of you know when you see it until you don't know it and then but I think we spend way too much time on definition so let's just take it as you guys did and said stabilization is when we try to stabilize some place and leave it at that. You gotta be realistic, you gotta be modest, you have to be local, you have to be smart and you have to be a team player. So being realistic, Rome was not built in a day, neither was Kandahar, neither was cert, Raqqa will not be rebuilt in a day. So let's just be realistic about what we're trying to achieve. No liberal democracy was built in a day or six months or six years. So let's be very clear about what our objectives are. Wide opened eyes on the problem set. What are we trying to do? As Radha said, these are programs that just try to get us a little time in space. So let's not try to expect them to solve a problem like X you fill in the country, right? There was a few years ago in the last administration, clear guidance that came out that we needed to have a plan to fix Libya in six months. I don't know how to fix Detroit in six months. I don't know if anyone does. I don't know how to do Libya in six months, especially if you can't get into Libya. So let's be very clear about what we're trying to do. That ties into the next thing, which is let's be modest. We're not in control. And the moment that we think we're in control, that means we're probably out of control, right? Everything is gonna be by, with, and through. If we don't have local partners with the political will, it will not succeed. We can keep trying, but it's not gonna succeed. If we don't have those local partners, let's find them. If they don't have the political will, or if they haven't bought in to what we think is a mutual set of objectives, then we have to do a better job of trying to convince them. But maybe we also need to shut up and listen to what they're saying about how they want to stabilize their country and take a few lessons from the real experts. The real experts are obviously the people that live there, that know the place. Local ownership is key to all of this and local political will. So therefore, let's be local. We have to think locally and act locally. Communities are the center of gravity in all of these places. If you're gonna get anything done. And if you buy what I think is a definition we're working on now within the government, which is stabilization is an inherently political endeavor. It's separate from peer development. It's definitely separate from humanitarian assistance. It's political. Well, trite but true, all politics are local. All security is local. I would argue all legitimacy is local. No matter what you're hearing over the radio about how great things are in the capital. If you're not seeing that in your community, real change today, or if you don't see any reason to hope that realistically tomorrow might be a little bit better. No matter what's going on in the capital isn't important for you. It's gonna have to be local. So focus on the local, fit that into a national context, obviously. But it's gotta flow up. It can't flow out from the capital. Especially in most of these places, the reason that these problems exist is because they never had a national anything that was effective. If so, they probably wouldn't have the governed space that we're in there trying to fix, ungoverned space. So let's not pretend that we can do things telegraphed out of an embassy. They're gonna reach into these dark recesses that are very different in each place. Which brings us to being smart. Everything we do has to be informed by real time analysis. We need to be quick and flexible and highly adaptive in what we're doing. As I already mentioned, there's no one size fits all. Everything is different village by village, neighborhood by neighborhood, the actors, the drivers of instability, the forces of conflict in that place. It's not a society necessarily, it's a neighborhood. Who are the good cops? Who are the bad cops? Who are the people we can work with? Who are the people we shouldn't work with? Who are the people we, they're not ideal, but we need to try to bring them into the process because they're legitimate in some way. You need to have local staff who know that place, who are telling you these things because we stumble into these things and when we come up with the answers too quick, by golly, we're almost always wrong. That's okay, we're gonna make mistakes. So then we have to be able to identify those mistakes and adapt to fix those mistakes. And when something's working, let's double down on it, but then you gotta keep watching it because it might not work for very long and then you gotta change and try something else and we can talk about what that looks like in questions. But everything has gotta be hyper contextualized in time and space and micro-targeted and that's when you get some results. And then lastly, you gotta be a team player. Which is sometimes hard in these places. Our government can be quite unwieldy. It can be very large. We like to say the 3Ds. It's not just defense, it's not just diplomacy, it's not just development. But even within those 3Ds, in certain points in time, many of us have different examples. You could go to the embassy compound in Kabul and spend all day trying to identify all the coordination mechanisms. But you gotta be a team player. Because again, as Radha said, these stabilization things are a very small piece and a very larger US government strategy, international community strategy and then the whole strategy that that host nation is trying to execute with supposedly our help and hopefully our help and assistance. I'll end with that and just again, all hard questions go to Jake. Thanks. As the last speaker, it's really hard because I think virtually everything I had jotted down to speak has already been said by one or more of the other panelists. Let me try to just do about five quick points which hopefully will move toward the Q and A. With regard to the process, just to second all the positive comments and praise, let me just add one more thing. USAID insisted on and asked USIP to convene an advisory board for this project which met several times during the process of the project, included outside experts with different skill sets who provided advice, inputs and guidance. And I do think that is another aspect of this study which is a good practice model for future. Second, and again, this has already been mentioned, but as a somewhat qualitative person myself, I like case studies. I like many of the qualitative studies. ESOC really went out of the way and did a systematic review of other studies and this is like going to be the valuable state of the art in terms of trying to bring together a lot of other qualitative as well as quantitative work and putting it in a digestible form and drawing out systematically the main themes. So this synergized very well with the quantitative part and not least I would say proved that ESOC is able to do good quality qualitative work, not just the big data empirical analysis it's known for. There now maybe this is a slightly different take. My third point is that my take on the message about the quantitative work is a little different and I really don't think it should be taken to mean that having more data is always better and certainly the metrics fetishism that was rampant during the surge period in Afghanistan, particularly the obsession about collecting data and tracking developments on a quarterly, monthly, or sometimes even weekly basis, that's something really should be avoided for the future. The time horizons where outputs and impacts are felt are just not that short. And I think the key point that comes out of the study is collecting even a small set of data but doing that consistently over time and starting from as close to the baseline, i.e. the start of the intervention as possible is crucial and just to steal an example from ESOC and ask forgiveness later, if you are taking regular thermometer readings over several decades will probably give you a much better view on climate changes in the climate than a very sophisticated analysis of a lot of data at one point in time or over one or two years. So I'll leave it at that but I do want to caution because data collection is expensive, it's not easy in some of these areas. So you need a core set of indicators that's not too big and not sort of a compromise between what academics would like, which is always more and what is sort of feasible in the situation and what most important can be collected consistently over time. And in that one I think the point of the study that the tracking of expenditure data needs to be much better is really critical. That is important that at least you really have to know where money was spent, when, how and for what purposes or what investments. Fourth, I think there's a need to be realistic about what can be accomplished in these kind of studies and this study I think went as far as it could and there are just many factors which create a lot of noise in addition to the message. There are the exigencies of a stability operation which puts a lot of pressure and short termism, challenges of data collection which I mentioned. And differing and sorry, differing and changing definitions of stabilization and what's included in that. I mean, just to give two examples coming from my background in the World Bank, public health and community development and it's actually called community development are quintessential kinds of development programs yet they became whatever you wanna call it appropriated into the stabilization and some of these programs are actually included there but they certainly the national solidarity program started out as a long term development program and then coming from that development background the projects are five years and impacts are longer than that and that's sort of like five to 10 years is sort of what the lowest, shortest definition of the long term would be for the stability study 18 months as reckoned as the long term and I just wanna highlight that. My final point and this is actually echoing what Radha said so maybe this will help tee it off for the Q and A but it seems to me that underlying some of this ambiguity about the definition of stabilization is a more fundamental lack of clarity about the underlying objectives and were these objectives at all feasible in terms of the timelines allowed? What was a sequencing? Sort of a theory of change which would have a sequence of getting some initial level of security allowing the government to come in and working on development you could sort of see that kind of sequencing actually it all happened at once because it was under enormous time pressure and this idea of a preset limit of three years or as once was called government or governance in a box I mean these are just patently unrealistic and I think people recognize that at the time and then political strategy and backing when you have a surge as in the case of Afghanistan that the president of the country basically was at the very most lukewarm about and not even lukewarm about much of the time and this highlighted the underlying political problems so what were the domestic political dynamics? Iraq I don't know at all well but the surge was to bring the Sunnis into the fold and then promptly the political backing for that fell apart with the subsequent election so this is really critical and I think Radha emphasized I mean if the objectives are unclear and unrealistic the theory of change is flawed and the political dynamics are adverse there's really nothing that aid either development aid or stabilization aid can do beyond sort of playing some of a mitigating role in its own right so I would just end with saying that the aid is neither the villain nor the hero in the Afghanistan stabilization story and you have to look elsewhere for those. Thanks. Thank you, Jake. Do you wanna come take my seat and I'll stand here and I, oh. Thank you. Yeah, here my mess. Thank you very much for very interesting presentations and discussion. I had a couple comments and questions I think I'll lead off with and then we'll open it up for questions. I mentioned earlier that this is a topic that I was fascinated at when I was at Tufts before I came down to USIP and was looking at and coming at it from my background is 15, 20 years working as a humanitarian and development worker in Afghanistan and Pakistan in the 80s and 90s and then post 9-11 lots and far too many discussions about the shrinking of humanitarian space, the blurring of lines, all these debates about the growing militarization and securitization of assistance, especially in the Afghan context which sort of got me to try to look at this question but I was interested in maybe approaching it more from an effectiveness angle than from a humanitarian principles angle that certainly the number one demand and desire of Afghans with security and so if indeed military development assistance if you used it to try to pursue military objectives and it was effective in promoting stability and security in regions, I think you can make a strong rationale for securitizing, militarizing your assistance to achieve that objective. But if there wasn't a strong evidence base that the militarization of development assistance was effective and that indeed actually in many cases it seemed to be fueling more instability than insecurity then it made a lot less sense to pursue that and just to give you a sense of the scale when I was doing my research sort of in 2008, 2009, 2010 on this issue, more than 50% of all USAID assistance in Afghanistan in those days was spent in four of Afghanistan's 34 provinces down in the south because of this assumption that focusing our resources in these most insecure provinces would pay security dividends in terms of, especially in terms of the coin strategy at the time. But in addition to USAID and other donor assistance, the SERP budget, the Commander's Emergency Response Program budget was about $1.2 billion which is a phenomenal amount of money to be spent in some of these very insecure areas and especially if you put that in the context that at that time USAID's global education budget was $800 million. That's why this kind of study is really important which is again, if it's effective to use your money that way, okay, use it that way. But I think there's been now lots of evidence over many years of lots of study really coming to very similar conclusions which yeah, there are some effects. They generally are more effective with small amounts of money and then even then they generally aren't sustained. And I think for a lot of the reasons that have been pointed out already which is again, a lot of the main drivers of conflict in these situations have nothing to do with the lack of roads or clinics or schools or facilities but they are political. And I think that's where I often say the US, we have two tools in these situations, our military and our money but the underlying drivers of conflict are often political and we're still not very good at doing the politics in these kinds of situations or understanding the politics or engaging in the politics or interfering in the politics. I mean, we don't hesitate to come up with the security strategies for these countries and go in with great force or even economically with vast sums of money but on the politics, often we don't invest so many trying to understand and engage in those which is why I think a lot of these initiatives have not been very effective. So that's my commentary but moving on to a question I thought I'd try to ask a couple of questions that started off and the panel, you can self-select in terms of who wants to answer what but so I guess starting off with another study showing again the relatively limited short-term impact to stabilization effect impact is it time to retire the stabilization approach? You know, a lot of the money spent indeed did not have stability effects but many of those funds did have very useful and powerful development effects and this is where I'm wondering is development assistance being set up to fail by expecting it to do something that it's not very well suited to do which is achieving the security benefits and I think a lot of the funding that has been spent in Afghanistan, we undervalue the tremendous gains that have been achieved. I mean, in the health sector for example just the incredible improvements in reducing maternal mortality rates. To me, as a former Save the Children director there I think that's a good in and of itself but increasingly I don't think we value that because it didn't necessarily improve the security situation so I guess maybe that's just one question throwing it out. Where should we use stabilization? Where shouldn't we use and are we overusing it in the current context given the evidence base of its limited impact? And I guess the second question that Bill already touched on I think he nicely put it the metric fetishism which is again, lots of studies showing so many results, lots of qualitative studies that did it and I worry as someone who's on the practitioner side for many years about the growing demands for collecting more and more and better data. So what is the point of diminishing returns because quality data collection as you know more than anyone is very labor intensive, very expensive to do well and very difficult to do in these insecure contexts like Afghanistan. So absolutely monitoring evaluation is important. We need to do that, certainly not saying trying to determine impact isn't important but where is the point of diminishing returns when actually studies done 10 years ago were pointing to very similar conclusions to what we've done with a much more sophisticated methodology with much better data which has definitely brought more rigor and nuance to the discussion but with very similar findings. So over to whoever would like to talk about Rodak. Why don't I will start and then summarily kick it over to other folks to answer the harder parts. So I think it's useful to kind of group this into a couple of buckets. And the first is I think the question of whether development assistance or stabilization assistance or whatever frankly are doing in Afghanistan is set up to fail is fundamentally a question of what are you trying to get it to do? And then did it do it? So I think in some ways and we've all said it kind of in our own different words the goal of stabilization hasn't been precisely defined or maybe even fairly defined in a lot of contexts. So it's not even well-defined to say that it succeeded or failed. It achieved some outcomes. It didn't achieve other outcomes where the outcomes it achieved the right ones where the outcomes it didn't achieve the right ones. And so that's where this sort of frame on what are we, what is our overall campaign strategy? Where do we think stabilization fits in? What are we trying to buy with stabilization? And then you can answer the question, did you buy it? And I start at that point because I think that also helps us cut against what I think is a very real concern and that's why I closed on this sort of human note of programming which is collecting data is really hard. Collecting in a real time is really hard. And the people who are implementing it, collecting and monitoring and evaluating their program is a tiny, tiny slice of the total set of things they have to do, right? So it's not, it's easy for me to say, hey, I need you to collect 85 more things because all I have to do is analyze it. I don't have to figure out how to get a program to operate safely and effectively day in, day out in a conflict zone. So I'm mindful of that and I think in some ways it's incumbent on us here who have in some ways a luxury of time and space, such as it is and it's not a lot of time or space but it's maybe more to think about what are the campaign objectives? What are the stabilization objectives? And then what do we really need to know? What do we really need to know for the programs to self-evaluate sort of in real time as part of what USAID does is its ongoing monitoring evaluations? What do we need to know to be able to report to Congress or oversight or whoever? And what do we need to know to be able to do these strategic reviews? And let's put that all together, make it as streamlined and minimalist as possible and do it. And that's not to say that more data is better but it is to say you wanna define that complete picture before you decide what metrics you do or don't need. Awesome. And just a quick point on this issue of stabilization and whether it makes sense going forward. I guess my view is it's a necessary but again insufficient approach and resources. If it is part of a political objective, political strategy, I think it makes a lot of sense. If on the other hand it's tried to be used to achieve military objectives, I think it would be very difficult for it to succeed. And I'll agree with both of you, that seems like a safe thing to do. No, but you're bang on. The question is what do we want to achieve? And I spend a lot of time these days with our DOD and military colleagues coming up with what's the plan. And I'm like a broken record, I just keep saying USAID will be there but we will never be as big, we will never be as fast or we'll never be as effective as the military thinks and wants us to be because they just think and plan and execute on a scale and speed that is impossible for civilian actors doing things by within through local actors. That doesn't mean not to do stabilization. It means there's an education process. All the points that I went through, we were saying to people in Afghanistan in 2002, we were saying to, in Iraq in 2003, four, five, six, and seven, be realistic, be modest, be small. But that's not what the plan called for, the plan called for billions of dollars of reconstruction that then became stabilization. And a lot of very smart people were doing a lot of programs that a lot of us weren't sure and probably pretty sure weren't gonna work because I think a lot of those lessons have now been learned, maybe over learned. I think one of the things in the summary document said, just because it didn't work in Afghanistan doesn't mean it's never gonna work and just because things didn't work out like we wanted to in Afghanistan doesn't mean everything was a failure. We do have to have studies like this and others to tease out what do we actually know, what actually works and then be really realistic. What is the objective? If it's a military objective, guys, that's not gonna, our bridge is not gonna make a difference. It's that political objective though and I think that's one of the things that you touched on, Andrew, is that we do not put enough resources into the hard part of the diplomacy and the political because it's hard and because it's really just so complex. It's easier to focus on the money and the stuff and it's easier to focus on the airstrikes but that's not gonna necessarily get us to where we need to be. On the fetishism with metrics, we need less data and more information and more analysis of that information. I'm a big proponent of more third-party monitoring. Get people that are really smart in the field while it's happening. Let them decide what the data they need is and get that started sooner. Get that, as you mentioned, whether there's a lot of things that these operators have to do and these practitioners have to do. Take that away from them. Have someone else doing it and the data's gonna be more attuned to what you need it to be. It's gonna be more accurate. It's gonna be more objective and you can get real-time analysis from a lot of people that hopefully are back here with more time and space, like you said. Yeah, so just really quickly, I wanna pick up on the link between what Rob said and the question Andrew posed about are we at the point of diminishing returns and in terms of data collection and the obligations we're putting on people and I'd like to think about this in two different dimensions. One is how precise are your data? So are you really nailing down like exactly what the inputs were in a particular time and place? And then the second is how consistent are you being over time and how flexible is your system to let you aggregate that up in different ways so you can look for effects at different time scales and different geographic scales. And I think this bears on Rob's question because getting that consistency right is much more important for being able to achieve the objectives Rob laid out than getting the precision right. But looking at US government programs in Afghanistan, Colombia and Iraq, looking at government of Pakistan and government of Philippines programs, most of the effort on the data side to Andrew's question goes to precision and not to consistency. And so I think we're past the point of diminishing returns on that one dimension but we're not investing enough in the staffing and the labor to make sure that we have things consistently measured and aggregated up so that we can look at them in flexible ways after the fact and do the strategic learning. So we're kind of putting the effort in the wrong place when it comes to data in a lot of these settings. Thanks, I'll open up for questions first off the mark right here. If you, we're webcasting, so please identify yourself and wait to speak until you have the microphone. All right, thank you. My name is Shikib and I'm from the Voice of America. I have two questions. One is directed to Mr. Jason. The second one goes to Mr. Robb. I've read like kind of two, three reports, special reports about the UCI projects in Afghanistan and the more emphasis is on small projects. One concern about the small projects is that the impact is also small and as well as the sustainability is also like limited and small. The people of Afghanistan like one like kind of bigger projects that can affect their life ultimately. And the second question is about the budget card. The Trump administration has started like federal budget card and ultimately it will affect the State Department and USA as well. Will that affect the projects in Afghanistan? Thank you. I'm gonna take two or three questions and come back to the panel. Alistair. Thanks very much. I'm Alistair McKinney, Overseas Development Institute London and former World Bank Country Director for Afghanistan. I very much welcome the study because there's a lot of work going on about conflict prevention and the role of development interventions on violence. And there's a recent study came out in the last few weeks on South Sudan and that study as others raises the issue of legitimacy. How do people in the country see legitimacy? What is the relationship between legitimacy and ownership? And that also leads into the question about is stabilization the right concept? Is it something we do to them rather than looking at the experience of successful colonial rules whether it's the British, the Turks or even the early spread of Islam which is very much indirectly influencing a country rather than trying to take it over and shape its institutions. But looking at this through a legitimacy lens what role does public service provision play? I mean, is the stabilization really using the wrong instruments? Are the issues really much more of the other dimensions of legitimacy like justice, the way in which the state represents the values of the population? What is the role of the distribution of rents in some kind of market for loyalty, for example? So the question is doing the analysis of stabilizing interventions, how they turned out as one thing but what was the rationale for these interventions originally? Was the theory of change valid? Were these considered legitimate by the local population? And I just wonder whether we're scratching the surface and not really drilling down deeply enough into what drives conflict in Afghanistan. Thank you. One last question, the woman in third row here. Hi, my name is Salma. I'm just an intern with the House of Representatives but I'm Afghan, I'm Pashtun. And my question was just sort of bringing up this history in the 70s, the 80s, the US was funding the Majaheddin in a proxy war against the Soviet Union. So how do you get, I mean, given that history of the US, US's involvement of the US funding the Majaheddin to get the Afghan people to trust US organizations like the Institute of Peace and then also given the fact that the US is still allied with Pakistan, that the US has kind of interesting relations with Saudi Arabia right now, Saudi Arabia, which funded madrasas in Pakistan in Peshawar specifically that trained these Majaheddin and then sent them to Afghanistan. So how do you reconcile that past and this present of having these alliances with Pakistan with Saudi Arabia that funds terrorism and getting the Afghan people and the Afghan government to actually trust more US interference? Okay, thank you. We'll go back to the panel and again, and anyone want to start off with Rob, you seem about to jump in. No, I was about to tell him to answer the questions that were given to him. So on the legitimacy argument, I think there are a lot of filters on all of this and one of them is the need for speed which emanates always out of Washington and so we start getting into what is possible in three to six months. That's not that we don't understand that the underlying tensions that really need to be fixed are ones of basic legitimacy. You point to justice, justice is almost always the first, second or third thing that people point to in these places that they want. Many parts of Afghanistan have benefited from courthouses that have been built and rebuilt several times because if you want to set up a rule of law program or a justice in quotes program, the only thing that you can really do in a justice program in six months is to build a courthouse. I've spent a lot of time in South Sudan where a lot of people spend a lot of time sitting under trees every day listening to court cases and the courthouse is the least important part of a justice system. However, if you want an effective justice system, you need justice system, you need judges. For judges, you need people that went to law school. For law school, you need people that have a bachelor's degree. For a bachelor's degree, you need someone who went to high school. For high school, they had to have an elementary and primary education. That takes 30 years. When the need for speed is we need something to happen within six months, no one in the situation room wants to be presented with a plan for a 30-year justice system. That doesn't mean that we don't think that's a problem. It just speaks to let's be realistic about what stabilization programs can do and that let's be realistic that they have to be part of a larger, more comprehensive and much longer-term plan. That plan might change. But if we get obsessed with just the immediate stabilization needs, it's very easy that we can actually undermine the legitimacy that we're trying to build in a lot of those things. On the trust, trust is earned. Trust comes about not by word, but by deed. In a lot of these places, we find ourselves in very difficult circumstances given all the different players and neighboring countries and all the different relationships that we have to manage over time. And I think one of the base reasons you look at when we step back, all the things we're talking about. It's really good people, some of whom I see in this room, Hainu, have spent a lot of time in Afghanistan, working really hard at the risk to themselves to help the people of Afghanistan. They're not there for, you mentioned, I don't like to take a lot of lessons from colonialism. They're not there for colonialism. They're not there as some part of invasion. They're there to try to help the people of Afghanistan. It's through those programs, it's through the attitude that we bring, it's through what we can do in support of the people of Afghanistan, for them to decide if they wanna trust us or not. Coming at a couple points on the small versus large, I think there's a role for both. And particularly on longer term development, you can have big projects that make a big impact on people's lives. Key on the stabilization is why is the better understand locally, why are people grieved? Like what is it that they have an issue with? And if you're gonna do some programs, you better be sure that that's addressing the grievance. If you build a school, they're not grieved because they don't have a school. Well, yes, it can sort of have development impacts, but it's not gonna fundamentally address the driver of the conflict. And so I think what the study has found is that these small projects can be helpful to address some very specific immediate grievance issues. And in terms of legitimacy, I think one of the things we've seen here in other places is it's not just the service provision. Just serving, I mean, just having the government serve services in rural Mississippi is not gonna make the people sort of say, well, great, my government's taking care of me. It's how is the community involved in determining what's needed and how those services are delivered and is their accountability built in? If you don't have that local involvement in the service provision, it's gonna be very difficult to actually make an impact. And finally on trust, just a point on, I agree with Rob that when we're there, we are trying to do for the good of the Afghan people, but also to serve Americans' national interest. I mean, let's be clear about that. But when you do go in, if you go in as part of a military and you start doing development work or stable, you will be viewed that you are part of the military apparatus, no question about it. And then if you sort of, as the policy changes, you begin to say, okay, well, we're not military. We're sort of development people because that's how we will see ourselves. The locals are not gonna see you any differently. You will always be part of the military apparatus sort of for years and years because that's where they foresaw you and that's the assumption that they have. So I'll stop there. Anything else? Let's go back for another round of questions back here on the David Sedney. David Sedney formally with the State Department, NRC, DOD, and most recently, Acting President of the American University of Afghanistan for the last nine months. First, I wanna say how much I admire the report along with the RAND study of SERP. These are the two best data-driven analyses that I've seen of this issue. What I really hope is that there's more coming out, a book even, because you clearly have a lot of data behind that. And that's why I would love to see even more of what you're doing. Secondly, on the overarching of stabilization, I agree, the fact that 16 years after we first went to Afghanistan, we're still talking about stabilization. To me, it throws very clearly the question of the intersection of overall policy versus what we're doing. You stabilize a patient and then you work to cure the illness. We're still stabilizing. We lack an overall policy. Everyone has, I think, made clear. One comment about the military, is I've worked both with the military and with the State Department and others. The problem here is not the military at size or its capabilities. The military is that way because the people of the United States have made the decision to have a big military and made the decision to have a small aid. We've elected three straight presidents who don't wanna do nation-building. My question, really, there is, which is one you may or not wanna take on, can we succeed addressing a big problem with two small resources, which is what I think lies at the heart of a lot of the issues we've had. Thank you. Over here on the left. Jared Godby, a recent graduate. So I've noticed there's a couple kind of parallel, probably, problems that have not been identified. One of them is the distance or the distance between any type of developed strategy going into these situations and the particular practices that are taking place on the ground. But academically, it looks like there's also probably a distance between any type of theory of stability that exists or that is being relied upon to influence or to test using the quantitative methods that a lot of these studies, like the one currently has been used, they tend to rely on a lot of previous quantitative studies that use factors of identification. Would you, I guess, agree with those characteristics? Is there anything that we can do moving forward, I guess, to create a theory of stability that can be used to influence both strategy and the empirical studies afterwards? And lastly, the gentleman here. Okay. Okay, my name is Sergey. I used to live in Afghanistan for four years. And now I'm consulting for different programs in Afghanistan. So my understanding of stability, stabilization, is country could be stabilized if there is a willingness for local business to invest money. As if there is no interest, it means that country is not stable. So now businesses are hesitant to invest money, they're a local business, hesitant to invest money because of security concerns. They're not sure that tomorrow it will be ruined and that trashed. So, and I know this by doing assessment of grants that we are willing to give to different businesses and we say, okay, we will give half 50% of the entire cost, invest another 50. And they are still not willing to do this. How to mitigate these concerns, security concerns? I mean, how to be realistic in this? And do you think there is a way because we should encourage business to do more, to invest their own money, not just rely on donors, but engage their own capacity. Thank you. Thank you. Bill? Yeah, could I come back? I'll come to your question on investment, but also briefly on a couple of the previous points. You know, I think Bruce Riedel said very cogently that the U.S. was on a unique position of being on both sides of a civil war and counterinsurgency over at different times during the last 30 years. So I think it sort of gets to your point. And actually we did some work earlier on lessons from the Soviet withdrawal for the current or for the 2011-14 transition. And I think that's important. On large projects, I couldn't agree more. I think one of the failures was not to start, you know, each year you have a short-term approach, but then you go 15 years without a major, say, irrigation project. That is just wrong, you know, and some other projects related to context and also thinking about places like Syria. You know, urban areas are different and the failure to get electricity to Kabul for the first about seven years I think was a failure. And so, you know, actually if you're looking at a place like Syria for the urban parts, I'm not, you know, a lot of the lessons or thoughts will be different. It may not be that small, it's beautiful. You may wanna make sure there's water supply, electricity, buildings are rebuilt. It may be like a more classical reconstruction program which worked extremely well on a short-time horizon in Europe and Japan after World War II, but that was because there was politics and institutions behind it. And so, I just wanted to flag that. I think on the business thing, your point is well taken. I would just, you know, it sort of gets to the primacy of security concerns and I think there was another very interesting ESOC paper using cell phone data which suggests or one of those policy implications you better make sure the urban areas are secure because that's where the business investment is going to happen. That's where you might be able to attract some. I think it's not only security though, it's the political short-termism. It's the, you know, Republic of Korea, look at that. That's the biggest success story I'd argue, political development, everything. The US security guarantee for the Republic of Korea was never in question as far as I know since 1950, basically. Look at Afghanistan, you know, complete withdrawal by 2016, shift by 2014, all kinds of deadlines forever on the security side. And then what do you expect aid to accomplish? So I want to come back to David's point which I think is a good one, just a little bit of a, I mean, there's really only one answer to can you succeed with a big problem with two little resources. But I think it's a little bit like how you eat an elephant, right? It's one bite at a time here. And I think a lot of the frustration and sort of perceived failures with stabilization in Afghanistan, but with this sort of broader project of is stabilization a thing we should be doing? Is this misalignment between what we think stabilization is for and what it's actually for? And I don't disagree at all that if you've been in a place for more than 15 years, you could probably do a couple of big development projects and development projects do have longer execution times and longer time horizons, but 15 years can kind of fit into that. That's not saying we haven't done that in Afghanistan, but that is saying that those things weren't necessarily called stabilization and they shouldn't be called stabilization, right? Those are longer-term development objectives. There are longer-term political objectives that we can debate and I'm sure folks here know plenty more about that, but these stabilization programs are not gonna be all things to all people and they're not a silver bullet. They're not magical, but they can provide a necessary space in this overall arsenal of tools and I think what's worth kind of thinking about is not are we resourcing stabilization enough, but to David's point, are we resourcing the tools that we need for this overall campaign or overall strategy in the right proportions to get to where we wanna go? And I think that piece has been missing in Afghanistan for a long time for a lot of reasons that are too long to get into, but it's worth just remembering that this is not a stabilization critique. This is a strategic question about what we're trying to achieve here and what resources are needed to achieve that. Okay, I think we'll go back to the audience for another round of questions. Anyone here? Oh, sorry, my name is Chris Binstrom, a social psychologist. I had a more basic question really about how you measure commitment by the local community in these projects. And so I guess it's sort of the hearts and minds and the local focus. I'm assuming that the more the local community is actually committed to the project, the more counterproductive it would be for insurgents to actually destroy the project. And secondly, I'm also assuming that in terms of your point about it being a dynamic context, as politics change in the US, the perception of the US changes, then levels of commitment to US projects are gonna change. So I'm just wondering how you measure commitments, you know, local commitment to projects over time or how you think about that or is that something that's done? Mine's a basic question really about steps. We'll be right next door. I'd like to pick up on a writer's point and also Bill's and others about the lack of clarity of objectives. I mean, by the way, Nick Kraft also previous World Bank country director in Afghanistan. I think there is clarity of objective in Afghanistan. And we need to understand the objectives conflict. There are the objectives of the development community, which we all think we understand and know. I'm not sure we do, but we think we do. And then there are the objectives which were set down at the beginning, which was basically to ensure the security of the United States and fight against anti-terrorism. And if you read some of the Barney Rubin stuff, you'll see how important that is as the overall objective. Those objectives completely and utterly conflict. I think we need to understand that and operate within that environment because we'll never get clarity of objective. The problem with when you do understand that, and of course then you get the U.S. military and others working with warlords and border posts supporting military groups which are then present at customs posts and then you think you're gonna bring honesty and anti-corruption to the customs things. It is completely antithesis of building up a legal judicial system. And my own sense, having been with the World Bank, is that one probably ought to accept what these conflicting objectives are and then operate within them. And in my opinion, frankly, trying to build up a legal judicial system within that context is frankly just blowing against the wind. But there are plenty of areas in which we can operate, education, health, NFP, and the rest. And we ought to understand that better and then operate within it. I know it's very difficult, but it does mean also you can't build up a legal judicial system. Just one sort of more minor level point. I just give one a specific example. I remember a time we were discussing about building up, taking on somebody's point about how it's all about communities and building up from the bottom. There was this discussion that went on for probably two years around the time of Karzai's re-election about district level councils. There were two sets. One was the appointed district level councils, which was done by the Ministry of Local Government. And then the other was using NSP, the community level councils, having a number of them agglomerated together and then building those up to the top. Of course, Karzai and others wanted the first because that allowed people appointed in positions to support his election. And the donor community, well, basically led by the United States, went with the first. I think most people must have understood that that was the wrong thing to do. But we did it because that's what Karzai wanted. And they're sort of interested in, you know, when one talks about what we need to do, what the government wants and built. How do you deal with a situation, something like that, where what you're doing is you almost for sure know it's not really in the interest of the country? There's one question in the very back. My name is Kyle. I'm from Creative Associates. There's a question in the report about whether addressing economic conditions should be a necessary element of stabilization activity and whether the effects or success are unclear. And then there's also a mention about a lack of evidence linking economic conditions, instability and conflict. So kind of a loosely word question, what indicators are missing? What things would you have wanted in the data to draw those conclusions and develop those linkages? And then building on that, do you think that addressing economic conditions can be done on the small scale or does it need to be longer term with a longer time horizon? Thank you. We're going to go back to the panel and I think we'll just have time for sort of maybe starting with Bill, work our way back, ending with Hugh Jake and take a couple minutes each to answer the questions and any concluding remarks. Okay, yeah, I'll sort of... Can you turn your mic on? Turn the mic on. We've together Nick's point and also going back to the earlier one. I mean, I think you don't want to retire stable stabilization because the development, I mean, let's be quite frank and David said that aid was under resources. That may be true globally but I don't think it was true in Afghanistan. There was, it was true before 2006 largely and after 2006, Afghanistan was pretty well resourced in terms of civilian development and stabilization assistance, it was pretty high. So you have this sort of revealed preference. Nobody wants to do nation building but they end up spending a lot of money on civilian aspects following the troops and I think it's important that USAID maintain a seat at the table and not be, we talked about being a team player but not be a sort of very junior subordinate member of the team that basically just follows what the DOD suggests and I think that has got to change and how you change that politically it gets to the incentives within Washington. I mean, one sort of probably bold and totally unrealistic recommendation is go back to the USAID having a cabinet level position in the government. By the way, just to highlight DFID, my impression in Kabul and where it was DFID had actually a much stronger position vis-a-vis the Ministry of Defense in the UK, not necessarily equal but at least able to argue points for example on counter-narcotics. My impression was that USAID was more of a victim or unfortunately sort of cornered into being a follower and I think actually having USAID fully at the table is really important because the other thing I learned in Afghanistan as a development practitioner was how shockingly bad the diplomats were at political strategy and so you may even need the development people there to be able to bring in the points about the need for a longer term and political kind of strategy. So you navigate the different objectives as Nick was saying but I do think it's really important to have a seat at the table on these bigger strategic questions and that's really critical for success in the future. Thanks, I won't weigh in on any of that. Back to David's point and as Radha and Bill you pointed out, in these particular contexts lack of resources wasn't the biggest issue, it's what are we doing with those resources, what are we expecting to achieve and how are they aligned in a nutshell. On how do we measure commitment? I'll leave that to the academics. I'm the generalist who says you know when you see it but what it looks like is when you go in you provide a community the opportunity to be committed to that project from the start, from the get go. It starts with shutting up and listening and saying what do you want. And we often with all the best intentions say we have a water program so then you enter into a village and say we're here to work on the water system. As you said, you can have a school program, run around doing schools. That's great if you want to improve the number and maybe quality of schools but if you're trying to get at whatever grievances or whatever issues are there you're gonna do that based on want, not on need. And that starts with asking people what do they want? That's one, two, who decides? Well you get the community to decide themselves. And then that's where you really have to get into what's the actual dynamic in this community and that's why this stuff is very high touch and it's hard to take to scale. Is that actually what the community wants or is that what the local wannabe warlord wants? You learn that only through doing. And a follow-up project and a follow-up project and then you will see if these people are committed to fixing the well that you helped them build. Are they actually putting in their own resources and if it's not money, is it time, is it commitment, is it emotion about these things? And then if you're not getting that level of commitment you drop them and move on because you can keep trying to do stuff and if you don't have the commitment it's not gonna work. Lastly, Nick, on your issue, I won't speak to that specific thing although I remember that debate and I saw some smiles of others in the room who remember that debate. That's where the teamwork comes in. That's what happens at the situation room. That's what happens at country team. That's what happens when we're trying to work on. We are USAID, a player, whether we're a bit player, a substantive player, changes with personality, with country, with context, with the plan. Our job and responsibility is to argue based on what we know. Sometimes we will win. Sometimes we will not. But the same can be said for our military colleagues. I see a lot of military plans that are never executed because the aid or state people say, here's the secondary third order or fourth order effects of you doing that and they get overruled. So that's part of the game that is government and policy making and it hurts sometimes. But it's compromise. Just one quick point on Bill's comment. I mean, I can't speak personally to what it was five or 10 years ago but I wanna assure folks we are at the table particularly in talking about Afghanistan and we're being listened to. So I will take a bunch of the sort of quantitative and evaluation questions that sort of came through and use that as an excuse to wrap up on my- One tough one for Jake. That's right. Well, I'll leave all of the tough ones. I'll just take some surface gibberish. So just as a reminder, when I was talking about the results, we had these three themes of stabilization is hard. It's a dynamic system and we need to think about these strategic evaluations not just sort of localized program evaluations. And I think sort of in thinking through the fact that sort of returning to a really early question on the distance between sort of theories of stability, theories of change and what we're quantitatively measuring, I think we've got actually a pretty robust body of evidence both on the quantitative, sort of pretty well-designed experimental stuff with the National Solidarity Program, non-experimental but pre-well-designed matching study with MISTI and a number of other programs looking at Sir Rand has a report such that we know the kinds of outcomes we can measure and what types of changes happen or don't. I think the gap isn't so much in measuring those outcomes and the theory, measuring those outcomes and saying what they mean. It's talking about whether those outcomes are meaningful and if they're meaningful for the theory of change that's operating. And to some extent, that theory of change comes from bigger picture. It comes from overall strategy and it comes from a different and sort of separate set of research and evidence and policy-making and an experience on the ground that we don't certainly review in this study but certainly exist in multitudes related to the case of Afghanistan. So I think that gap of really teasing out what do we think the theory of change, what were theories of change and which ones do we think were operative? Frankly, I suspect the real question is which ones were operative when and where? It is an open question for research and one that's worth thinking about. But I think that body of evidence kind of highlights that there are a number of sort of low-hanging fruit conclusions here that we can draw that relate to the two of the questions that were asked both on economic measures. The short answer is there's lots of survey measures and economic activity. We use nightlights, but I think the key sort of rehashing Jake's point is to pick something and measure it over and over again so we can see improvement. And similarly, we have some senses on what it looks like after the fact when a community has bought into a program. I don't know, we have great measures for it, but we could keep measuring those after the fact outcomes and seeing if the programs are successful, but we've got to do that over and over again. And so that kind of consistency and strategic level view of monitoring and evaluation is really important. But look, we're not offering up magic bullets here, right? We can do, we can review almost 100 studies and we can look at lots and lots of data, but at the end of the day, the lessons from this analysis aren't gonna make stabilization efforts more effective in and of themselves, right? Setting up future programs to collect sort of the right kind of data at the right level and have the right kind of analysis is helpful as is planning ahead, as is sort of taking advantage of new technologies that make data collection or program delivery more advantage. But at the end of the day, the best practices have to be incorporated into this sort of strategic view that is realistic and fair to stabilization as a project in its narrow role, but the campaign that you're looking at on a broader scale. And I think that's still true in Afghanistan, but it's true in lots of other places too as we think about Syria, thinking that we're gonna have a stabilization that brings government in a box in six months in Raqqa. It's probably not a fair pitch for stabilization and also probably something that's not gonna usefully resource or make good use of detailed measurement and evaluation. They could do the greatest measurement and evaluation possible. I can tell you the outcomes of the study right now. So I think at the end of the day, what we're hoping these results show is both that, hey, there are some modest gains to be had, but on a bigger picture sense, we can think about these gains and lessons after the fact to enable better design and implementation of stabilization programs in this broader nested strategy in Afghanistan and other conflict affected places. And let me just stop there. Yeah, so just to take the last word here, I think something that comes out of this study and comes out of research that ESOC has done and other people have done in more than 15 other countries around the world where development assistance is being used as part of a tool to rebuild social and political order is this idea that's been emphasized again and again up here, which is what can you do with this type of programming to advance the political and state that you want. And across all these places, what you see is that there are ways in which even in the most violent places, aid can contribute to opening up political space and creating incentives for settlement. And a wonderful example of this is Columbia where not development assistance, but actually military aid and loan guarantees overcame a funding constraint that the Colombian government had to enable it to enact a military policy that after 10 or 15 years created the space that you could get a bargain between the FARC and the government that was not on offer 15 years previously in Iraq, the combination of military action and development assistance enabled a series of local bargains that opened up tremendous political space by 2009 for a settlement of long running disputes between the Sunni and Shia populations. It wasn't acted on by the government, but the combination of aid and military action clearly opened up that space that didn't exist in 2005, six or until the very end of 2007. So there's clearly great potential for synergistic effects here. What I think gets in the way of that in some cases and is unfortunate is these unrealistic expectations and to end on a point Jason raised earlier, the conflation of the two goals for development assistance in conflict areas. And so the ways in which you would spend money to advance that political settlement, that comes from some theory of political change, that may or may not be the same as the ways in which you would use development spending and put the efforts of the people that both Rod and Rob have highlighted to work for long-term development. And so I think if we can separate out those two goals and think about development programming as having a role in both, but a distinct role in both, we can do a lot better at applying this tool of the government to help rebuild social and political order in these environments. Thank you. I'm gonna conclude with another quote. I included in a paper I wrote called Money Can't Buy America Love in 2009, which may reveal some of my bias, but it was a US-age study that said, the use of aid for short-term political objectives tended to weaken the longer-term political interests of the United States. Aid is a tool of diplomacy has its limitations when politically motivated commitments are at much higher levels and promise more than can reasonably delivered in economic returns. Some of you will be familiar with this quote, but this is a quote from a study I dug up in 2007 or 2008. On the USA assessment of its large investments in Afghanistan from 1950 to 1979, and it was written in 1988, just as the Soviets were about to pull out of Afghanistan, USA, many of us working in Peshawar in those days started our planning to move into Kabul and rebuild Afghanistan after the Soviets left. And USA actually thought, let's take a minute, reflect on what we've done during that 1950 to 1979, when it's important to remember, Afghanistan was the largest per capita recipient of donor assistance in the world due to the Cold War, when effort to win hearts and minds in Afghanistan during part of that period. And so I think just to conclude, I think maybe the next ESOC study should be to look at why, again, we've learned lots of lessons, but why are lessons not learned or applied? I mean, we're learning them, but others have talked about this, but I think in particular, an interesting question, is the incentive structures against sometimes learning these lessons, especially in an environment where security, everything has to be justified increasingly in security teams, terms to be legitimate, or to get resource. So there's some very real incentives to say what you're doing is stabilizing or having security benefits to get more of the development resources. And so I think that's a real problem we have, again, because I do feel, having worked in Afghanistan a long time, there's been tremendous development gains from the investments in Afghanistan, which again, don't get valued because they failed to achieve the security objectives, which I think they weren't best suited. So thank you very much for coming, giving us valuable time. Thank you for those who joined us online today. And last but not least, please join me in thanking our panelists. Thanks. Thank you.