 Good morning, everyone. Good afternoon and good evening to those who may be joining virtually. My name is Julie Werbel. I'm the division chief for leadership and learning in USAID's Center for Conflict and Violence Prevention. I'm joined today by an esteemed panel of experts to explore what we know about what works to prevent violence and counter violent extremism. Their bios are in the agenda, but let me briefly introduce you. Mark Summers is an international consultant author and specialist in issues related to youth, peace-building, PCV education, gender, and conflict. I'm sure he is known to many of you in the audience. Patricia Campy is a principal researcher for the American Institutes for Research. Jessica Baumgardner Zuzix also probably known to many of you is the deputy executive director for research and finance at the Alliance for Peace-Building, and Chris Inman on the end serves as a senior program officer for countering violent extremism at USIP. One of the biggest challenges that we face as peace builders is to demonstrate effectiveness. There are a host of reasons why we find ourselves so often unable to answer a deceivingly simple question. What works to prevent violence and prevent violent extremism? There are a host of reasons. We work in complex contexts where multiple factors affect program outcomes. The drivers of violence and violent extremism are often hyper-local and defy replication. We lack the resources to consistently invest in evaluation. One of the few sectors I would add in the development space that hasn't had the resource base to invest significantly over time in evaluation. And in the violence and PCVE space in particular, we're repeatedly asked to prove a counterfactual, often by our own leadership and by Congress. How did your activity stop a terrorist attack? How did you interrupt violence? Despite these challenges, we continue to make progress identifying programing approaches that effectively interrupt violence. This session aims to bridge theory and practice, taking us from theories of change to programmatic approaches, lessons learned and measures of success. I'm pleased that the Alliance for Peace Building has dedicated so many of this week's sessions to measuring prevention. And at USAID, we are contributing to this effort through our own learning agenda designed to identify what works to prevent armed conflict and violence. In partnership with the Alliance for Peace Building and the American Institutes for Research, represented by our two principal investigators, and ACLID, we set out to identify best practices in six areas, six areas reduced from the hundreds of questions that were put forth by USAID staff globally. Those areas are conflict and violence prevention, conflict sensitivity and integration, peace building, peace CVE, climate and conflict, and maybe most importantly, in monitoring evaluation and learning. That is to identify the evidence-based approaches for measuring the impact of our work across disciplines. Today, our four panelists will share their thoughts from their own efforts to gather evidence to tell the collective story of our impact. I had the opportunity to bump into Mark this morning who gave me a taste of the presentation he would share today, and I'm very intrigued how he's gonna tie peace CVE to reggae. With that, let me turn the floor to you, Mark. Okay, I'll do my best. So thanks, Julie, and thanks everyone for coming. What works for peace CVE? I guess the starting question before I talk about my book and what we can learn from it is, what if we looked at this situation, not from the perspective that Julie's describing, but from the perspective of violent extremist groups? What if we looked at it from the perspective of alienated youths? Those are the two perspectives that I'll be proposing. And I think one of the things is, if you're a violent extremist group in most of these countries, it's like the governments are rolling out the red carpet for you. What could be easier than to start to enter a country? In so many places, they're winning. So with that, I wanna draw from my new book to talk about terror and peace CVE today, and I mentioned how this started briefly to Julie when we were in line, and this all started in the year 2000, talking to Sierra Leonean refugees in the Gambia, and I asked them, how did you get here? And people jumped up and started talking about Tupac, and I thought, gee, they didn't understand the question, what are they talking about? And they were talking about Tupac Shakur, and that rebels were coming into Freetown during their January 6th invasion, wearing Tupac t-shirts, listening to Tupac music, and bursting out and carrying out terror tactics. Others talked about that in other parts of the city, they came out with Bob Marley t-shirts, and with Bob Marley songs, and started carrying out massive atrocities. And so I was trying to figure this out in 23 years, whatever, 23 years later, here we are. The last thing I wanna say is, I don't think I would have ever gotten this book done if I wasn't a fellow here at USIP, to get it started and to get money to go back a few years ago. So let's see if that works. Okay, so the book itself, it's a long book, this is pretty brief, but what it tries to do is look at war and peace through a pop culture lens, and I think it'll become clear why that's so important. So you have the big three in the book, and in Sierra Leone, during this period, were Bob Marley, the rap star, Tupac, I'm sorry, Bob Marley, the reggae star, Tupac, the rap star, and the movies featuring John Rambo, the Rambo films of the 1980s. And what did they have in common? Well, they were all blamed, but it wasn't their fault. So that's what links those three, and that's the power for alienated young people. That's what resonated, and that's why they were so important. So when the war started with diamonds, marijuana, and extreme terror being featured, it was almost impossible for military leaders not to exploit the attraction, the power of these three pop culture figures. So Rambo became military training videos. To Fode Sanko, the main rebel leader, he quoted Bob Marley, people said, people I interviewed said, every single speech. And the biggest song was Get Up, Stand Up. And then once the war finally ended, finally, youth turned to the same pop culture figures, not so much Rambo, because the war was over, but certainly Marley for inspiration and Tupac for friendship. So these were very important people. So just to mention about the book, the pre-ordering starts, it was supposed to start, but it's not ready yet. What can I say? It starts in mid-May, if you have a cue, there's a QR code I was sent from my press. If you wanna give me your email afterwards, I can send you the information from my press about it. So it'll come up for pre-ordering later this month on Amazon and it comes out in October. Okay, so the Sierra Leonean context, I'm gonna mention the three aspects. The one is the fake state. So a fake state is a state that's really, they're not even trying to govern. They're running the country as a business. And it was really blatant under this one very successful, in his mind, leader called Chaka Stevens. Chaka Stevens took all the diamonds in the country. He took almost zero of the diamonds, went through the government. He starved the government on purpose. So when people would come and say, during the war and say, oh, this government has no capacity. Yeah, that was because it was, that was the intention. So I guess people would say it's an extreme case, but I do think that when we go into countries, we think that the Ministry of Justice is there for justice, for example. And if you ask people in the country, they might have a different idea. And that's not really the locus of where justice takes place, just as an example. So then you have youth alienation. And this is a long part of the book. So I'm gonna try to go very fast, but male youth were trapped and didn't have, most of them didn't have a future. They could go into the city and find some way to survive. They could mine in the diamond fields or they could be really controlled in the so-called communities. This community-based idea in Sierra Leone was a joke for most youth before the war. And they were seriously emasculated. They were trapped people. And they were known quite often as thugs or tug, which comes up in a moment. And then female youth, this goes back to the era of the Atlantic slave trade, had a level of subordination and there was a level of sexual violence in this war that I, it was so intense. It's almost, I do dig into this in a chapter called The War on Girls to try to understand why was it so obsessive? But to just briefly, during the Atlantic slave trade there was, men were taken and there was a surplus of women from it. And so many of them were enslaved and there was this feeling that women had this wild, dangerous power they had to be controlled. And so this obsessive focus on controlling women arose from that period and it is true to this day in many respects. And then the issue of peaceful dissent. You could not say anything. There was no dissent allowed against the government. At all, it was harshly repressed. So just to give you a sense, actually these three, it starts in 1960s with the entrance of marijuana. It went into the soils and absolutely took off in Sierra Leone. That it became a crop. It's very powerful. So that's why I put it in Marley with the spleef in the 1970s. So if you can't express yourself in your country you look for people who can help you. Marley, the reggae stars in the 70s, they were the guide. And as I said, get up, stand up, stand up for your rights was the biggest song. Burning Spear was there, Peter Tosh. But Marley was the most important and one of the things you had to do was smoke marijuana and listen to reggae music, particularly Marley because it would help you to become conscious of what was going on. And this takes you back to this idea that we didn't do anything wrong, but we're being blamed. And so that was, you were supposed to analyze the situation. Marley sings about this a lot. Don't let him fool you or even try to school you. You've got a mind of your own. All those kinds of things are all over in Marley's music and it was really taken as a practice. So in the 1980s, this is an amazing thing. These movies came into Sierra Leone and were played everywhere to a degree that every single youth appeared to have memorized these movies. So during the war, you didn't have to show the film. You could just say, do you remember that in this, that scene, all the kids knew. They'd seen it and memorized it. It was, these were extraordinarily influential movies in the 1980s, the three, the Rambo trilogy movies. Across the world and to a level of depth in Sierra Leone, which was really quite stunning. And then you have Tupac. Now, I got a picture of him with his tattoo thug life. This is a man who says, I'm a thug. Bring it on. I'm not embarrassed about it. That's who I am. You see that the eye in life, that's a bullet. Tupac is fearless. Tupac is a thug. He's saying to Sierra Leone and boys, I'm a thug and I'm proud of it. All eyes on me. So he was, it was like a thunder clap. Immediately he was a hero. He's a hero in Palestine. He's a hero, Marley and Rambo actually across in all kinds of war zones. Those two are together, are linked. Okay, so the other part of the context to fill the alienation void came these pop culture superstars. And states and elite society perceive marginalized young people as the problem. But what do Marley, Tupac and Rambo say? We are blamed and we didn't do anything wrong. Those in power are exploiting us and forcing us to resist, right? Rambo says in the first film, but I didn't do anything. And then they shoot at him and then he starts his war. So that's the movie. So the idea is that the world's upside down. Tupac wraps about this constantly. That the world is upside down and the universe is perverse. Why is everybody attacking us? What did we do? Nothing. And that's what the three tell them. So to get the context about the sort of the terror war recipe, there's a lot of parts, but I put down five ingredients. So the first is you exploit government weaknesses. How easy was that, right? I mean, Shaka Stevens set up basically a business and didn't try to provide any services. The corruption was so obvious. And if you complained, you got beat up, you got vanished, all kinds of things. You exploit alienated youth. Again, everybody's alienated. They're trapped. Sexual slavery for girls and female youth. They were captives. And then you drug and you manipulate the boys to become terrorists, basically. And then you attack civilians. And this is very important, because this is very common. You attack civilians, and you avoid engagement with conventional warfare. Now, you tell me why CT doesn't work, because they want terrorists to fight on their terms. And they won't do it. They melt into the forest or into the deserts. And so they don't want to battle. They're going to lose. And the last one is you control natural resources. Obviously, diamonds and Sierra Leone. So what are the similarities with today? Well, you invade fake, predatory, incredibly unpopular states that don't deliver services. There's blatant inequality. They put you down or arrest you or vanish you if you say anything. It's the same context. You exploit youth. And I put up the example of alienated youth with ISIS, because ISIS is doing exactly what the rebels in Sierra Leone did. They have sexual slavery. I don't know if you remember the Catholic or the Christian women were slaves in Syria. And they drug their fighters and also run the drug trade. But they drug their fighters before they go off and fight. It's exactly what the rebels in Sierra Leone did. And the same thing, you target civilians and you don't engage in conventional warfare, because obviously you'll lose. And you control natural resources. So gold is a big one in West Africa. One of the big issues in Mozambique is it has the third largest resource now or holding of natural gas. And then there's sometimes the drug trade, where the drug trade is something that you can use and also sell. So if you look at PCVE taking place in an upside down world, which is what Marley and Rambo and Tupac teach us, it seems to me, what do you see? That the world's upside down. These governments are the problem. They're attacking us and we didn't do anything, which is what opens the door wide open for insurgents and for violent extremist groups. I think it's a mistake to start with the ideology of violent extremist groups and instead look at how governments run their countries and how they treat their own citizens. And then you analyze, and these are my suggestions for people, is the first is to analyze the situation from two perspectives. From the perspective of being trapped in an upside down world, being blamed for, I don't know, being alive? What? Not having a job? Being forced to sell drugs to the police so you don't get arrested? Yeah, because they run the drug trade? Things like that. They're bad people. In Bujumbura as an example, where in order to go to secondary school, what do girls do on the weekends? They're prostitutes. How else are they going to make money? So this is post-war. And are these bad people? Well, if you look at it from their perspective, the whole world looks different. And I really learned a lot from young people in Sierra Leone and elsewhere about this. And what if you see the state as an occupying force? It's not legitimate. What if that's the case? That's what a lot of people feel in these countries that are being attacked. Not that they like the violent extremist groups, but does that government represent me? Often no. So the strategic actions, right? This is not easy to do. CVE is actually, if you really do it, the chances of being attacked, if you really get close to a violent extremist group, they can kill you. They are terrorists. So real CVE is actually dangerous because you have to get to them. So how do we know when they were saying what works? Somalis told me NGOs and Kenyan NGO leaders, how do they know if a PCVE project is working? Because Al-Shabaab gets your phone number and texts you and says, we know where your wife's going to work. We know where your children are going to school. And that's a sign of impact from their perspective. So I think you have to mainstream the towering significance of predatory national governments. I know there's a tendency to focus on the local. There's nothing wrong with that. But that's not the root cause. And that's the way it is. I think what Carolyn Bonson said of FCV in the first session was actually really important that you have to work with governments. OK, what's wrong with that? Find ways to engage. But believe me, most people are going to want to talk about rule of law. I mean, it's almost predictable that they're going to mention that there's no rule of law. And the second one is, and I'll end here, Julie, is what really matters in PCVE programs is who gets in. Because it's a world of alienation. It's a world of exclusion. So everything that matters is the symbolism of programs. Because most people will never get in your programs. So who gets in and who doesn't? Everybody's watching. The content of the program is quite secondary from the perspective of people in these places. The key is who gets in, because most will not get in. So if elites get in, you go through the traditional leaders, the community leaders, we work through the community. If you work through the community, alienated youth will not get in your program. Because the leaders don't like them, and they won't get in. And so that's not going to work. That's actually pushing them closer to violent extremist groups. Because they're now even more outsiders. So I actually feel there's a lot of good work that we can do. But I think we have to know what we're up against. Because it's a lot. Thank you. Thank you. Thank you, Mark. I think you highlighted a lot of themes that we're probably going to touch on today. Certainly, many of the grievances, a lack of legitimate state, lack of rule of law, lack of opportunity, lack of voice. And I think the role of gender and gender-based violence, which has become so pervasive that we now have a term for it, conflict-related sexual violence. And then some of the pull factors and the importance of narratives. And as well as some of the failures of how we respond to it through asymmetric warfare. And then maybe some glimmers of hope as we all talk about increasingly localization and understanding local voices and the role that they play in the work that we should all be doing. And what IEP calls positive peace. And what are the structural factors that actually are required to make our targeted interventions with respect to violence or violent extremism take root? We have two more seats up front. If anybody in the back wants to fill them, maybe three. So please feel free to come on up. And with that, Trish, let me hand the mic to you. Okay, fantastic. Thank you so much, Julie. And thanks for giving us started. So what I'm gonna talk with you about today is the state of evidence, what we know now about what works for preventing, reducing violent behavior, and then update you on current work we're in the middle of to kind of deepen that around causal pathways. So what we know right now about violent behavior, there's a variety of ways researchers go about this. One of the most important is to collect all the research together that other researchers have done. And it's called a systematic review. And so the last one of these that we did, my research team did in 2019 globally to look at the evidence base, we basically found that most of the interventions around the world are focused on outcomes for individual people in a program. And I mean, the lion's share is 75, 80%. Which means that if your program is one that's trying to reshape structural things in a community, whether it's economic opportunity or green space or safe passage to school, these studies are basically not capturing that. They're looking at programs where somebody walks in the door, sits in a seat, has some kind of intervention and you measure outcomes over time. What we also found is that most of the studies are conducted in high income countries. So about 65%. There's a lot of countries represented in that over 40, but they're mostly high income countries, some middle income countries. There's very little that's done in low and middle income countries in terms of rigorous research. It's very difficult to conduct those studies in low and middle income countries because the capacity on the ground is often not developed enough to do monitoring and evaluation. So you may come in as a researcher with this great idea to evaluate this thing that you've been doing for five years and you find that the program has never collected information on who they're serving, how often they're serving them, why they're serving them, the training of their staff, all of that. Which then your study turns into a qualitative study that can't really look at the counterfactual that we've been talking about, right? The difference between your outcome being in the program and someone else's outcome not in the program who's very much like you in terms of risk for violence. We also found that these interventions with individuals are done primarily with individuals who are at very low risk for violence or might be a little bit at risk for violence. But the individuals who are really committing violence, there isn't as much work done with that population in terms of direct intervention. And there's lots of reasons for this. One is goes back to what you just heard that it's very hard to engage those young people. Either they don't wanna be part of the intervention or you have no access into them because the community is not, you know, why are you working with them? They should be in prison or whatever the case may be. Or you may try to partner with law enforcement and maybe they are part of the problem, right? So it can be very difficult to work with those populations and once you get them in the door, are your staff able to negotiate what might happen in that program when I'm in there and then a person from a rival group is sitting right next to me because we're causing a lot of the trouble. You need to intervene with us, but how do you have us in a shared space together where we are, we're safe with each other and we're actually gonna learn something that's going to help us desist from violence. In terms of theories of change for what works, there's three sort of broad approaches and they are tied specifically to these levels of risk that I talked about a minute ago. So the first one is intervening at an early age and I'm talking about, you know, as soon as a young person comes into this world, there are these psychosocial interventions that work with parents. To help parents understand the proper way to bond with their child, show affection, supervision, appropriate discipline, instead of discipline that generates a lot of anger and violence in a young person growing up, all kinds of things like that around human development. Then once children get into school, hopefully they have some kind of an educational environment, it's about teaching them things like emotional regulation. Like how do I manage my emotions when somebody wants to take that book from me or calls me a bad name or like bumps into me in a funny way, right? How do I manage my emotions to deal with that? Then getting a little bit older and still not really showing any signs of antisocial behavior, it's about really connecting those young people with a safe and supportive pro-social environment. So not an environment where they're listening to, you know, thug life sorts of things, right? But where they're actually contributing in the community, they have relationships with supportive adults. They don't have to be teachers, they could be other adults in the community, but these become trusting relationships that help them develop their identity as positive pro-social human beings who care about other people. So that's one broad sort of umbrella around theories of change for just any young person who comes into the world, right? They haven't demonstrated any issues at all. The second broad theory of change umbrella falls to those young people who have started to demonstrate antisocial behavior. So they may not have hurt anybody yet physically in some way, but maybe they're lying, maybe they're stealing, maybe they're skipping out of school and doing it in a way that is causing duress to other people. So they've started to show that something is not quite right, there's something going on. With those young people, they're normally in the pre-adolescence to middle-adolescence phase, somewhere between 10 years old and 14 years old, they're becoming much more influenced by their peers. So if they come from a healthy family environment, they're by about 10, it's normal and natural for your adolescent development to break away from your parents a little bit, right? And figure out who you are so the influence of peers becomes really important. So the interventions that are effective with young people at this stage are managing peer relationships. This is communication. How do I communicate with my peers when they say, oh, look at that girl who's over there, she's by herself, I think we should go over there and give her a hard time, which could end up in like a sexual assault. If I don't wanna lose credibility with my male peers, how do I communicate in a way to say, I'm not okay with that? If I just say, I'm not okay with that, maybe I'm now gonna be the one victimized, right? So a lot of the adolescent programs for youth that are at risk for violence focus on navigating these conversations with peers, the peer pressure. And oftentimes the victims on the other side of this, like the young lady who's walking by herself, she is involved in programs that help her communicate with someone maybe she's intimately involved with. You know, maybe there's some kind of a relationship that's brewing, which again, completely normal and natural. But what do I do if I'm in that relationship and I feel uncomfortable and something is going on? So empowering victims, potential victims in the adolescent phase is really important for decreasing the threat of violence. And the third piece for this population is social norming. So things in the community that influence the way we, what we think is normal about the way you treat girls, about the way you treat each other. Whether it's media types of things in art or it could be things that politicians say, it could be things that you see on advertisements for liquor or whatever the case may be. So the things that are effective is to put up pro-social things like eight out of 10 young men still treat their friends with respect. It's trying to provide those positive messages that say, oh, so most people are not doing this thing that my friends want me to do. So I'll be okay if I say no. And then the third category of theory of change is really that group that is already committed violence. And they're a small percentage but they produce the large percentage of violence, right? And they're doing that oftentimes with groups, sometimes informal groups, clicks, combos, sometimes very organized criminal gangs and organized crime groups, right? Those folks have disconnected from all the things we talked about before. That primary prevention did not work, right? The family wasn't a safe place, the community wasn't a safe place, the friends weren't a safe place, they're not in school. So once they're at that point, then you have to understand the interventions that you're gonna need to do are gonna take a long time to kinda reset the clock for that young person. And I mean years. It's a matter of internal changes for their identity to structure the way they think and behave in different ways so they don't turn to violence the same way they did in the past. But it's also the external world in the community has to be ready to receive them as different people. And work I've done around the world, I mean it's always an issue where you can have this program and you're giving a young person, he's like, I'm ready to come out of MS-13, I'm ready to work, I'm ready to provide for my family. He can't get a job. No one wants him to live in the community. He can't get a driver's license, whatever the case may be. And it takes that to have that person go right back in and say, I might as well be back in the gang. At least they protect me, I can make a living, I can take care of my family. So again, the political structure, the community structure could be really working against those folks at the highest levels of violence. So I think I'm at time, do you want me to say more about the future work? Why don't we come back to that in a minute. Okay, it's perfect. So if you have a chance. Thank you for your remarks. I really appreciate the way you've laid out the different levels of intervention from primary prevention to secondary prevention and tertiary prevention. I'm struck by a truism that I know, but it still hits home that victims become perpetrators and that that early, early intervention actually has an impact on tertiary prevention and stopping the violence before it begins. Also the challenges of reintegration, which I know are shared across disciplines between violence prevention and violent extremism. I tried to explain to someone once that trying to integrate a former violent extremist is akin to having a sexual predator move back into the neighborhood. Nobody wants them. Nobody wants to talk about it. The amount of work that has to be done at the community level to actually receive them as well as the individual rehabilitation is extremely high and it's difficult and lengthy and lengthy work. So thank you for your remarks. Jessica, may I turn to you? Great. Well, I have a small presentation I'll move through pretty quickly. I'm gonna let you know right away I do not have fun slides like Tupac Shakur and Bob Marley, although I know that's what we all want. So we can pull that up here. So just to kind of get a start as we're kind of working on this, as Julie was saying, I'm one of the co-PIs for the learning agenda implementation team that the conflict of violence prevention unit at USAID is funding research for us to kind of better understand what works and where in relation to violence prevention in relation to PCVE where I'm focused on complex sensitivity integration, modern evaluation, all those fun things. So for all of you nerds in the room I have a research heavy, nerdy presentation. Are we able to pull that up? Do I have to push anything? Oh gosh, I don't even know how to use the tech. Hey, my own conference. Okay, so I'm gonna start by telling you right away that there's very few things we know because it's so incredibly complicated. One of the biggest issues that we're running into is what even is violent extremism? What and who is a violent extremist? At what point do we intervene or engage? I think this is a completely repeated schematic here in conversation. Where are we focusing our prevention activities? Where are we focusing government intervention to respond to or create laws or legislation or policies around containment or interdiction? Where do we focus on disengagement and de-radicalization? Are those the same things? Are they partnered together? Does one happen first and then the other? What about rehabilitation and reintegration? Do you have to be fully rehabilitated to be reintegrated? Does that happen at the same time? Can you be in a process of rehabilitation while you're also de-radicalizing? These are really difficult questions and as a researcher when we start to try and think about, well, how do we group these to even understand what we know? We fullheartedly said, oh yeah, we can do a systematic mixed methods review on PCVE. We'll do one, actually we're doing five because each of these are very, very unique and very different. So I'm not gonna go deeply into our protocol other than to say we'd love to talk to you about our protocol. We'd certainly love to hear about the issues you're running into in your protocols. My lovely colleague and I have a lot of fun conversations around protocol issues, but I do wanna point out that something unique in ours is relation to our relevancy criteria. We really are trying hard to, instead of utilizing a traditional systematic review method where we're really looking at high level fixed effects and really considering quasi-experimental and experimental methods, to look at a much broader gamut all those other studies that are trying to understand qualitatively mixed methods approach. We did some sort of descriptive quantitative study. What do we actually know? Because the field is really maturing and I can tell you when we did a very brief one of these reviews in 2018, we had like five studies and we're like, yeah, yeah, we'll be able to do that. And then we realize, no, we actually have thousands of studies. So how do we actually go deep into those? So we really have a very broad mixed method approach to ours and we also have a theory of change analysis approach. I won't go deep into here other than to say that with a very targeted scraping of close to 2,700 resources we are doing full text coding of just under 200 full evaluations. For anyone in here who does research you can appreciate why my team, I have to give a mad shout out to Allison and Shazia who are in here. We're like blinded by numbers right now of lines and lines of code when we were like, yeah, yeah, we can do this. Some of the really fun full text indicators that we're looking at is trying to develop a better corpus of understanding what's the state of programming? What are the types of documents we're seeing? What are the goals of these programs? What are their outcomes? They're actually saying they're going to achieve. What about demographic data? Where is this happening? With who, when, why, how many times? Target group descriptors. I've heard this a lot here. Who is at risk? What makes an at risk person? Who is a youth? Is a 27 year old a youth? Is a two year old a youth? We have such, and I'm gonna come back to this in challenges, such broad conversations and many of our studies do a disservice by not describing how did we target these people? What is a marginalized person in this community that we say we're working on? We know that early intervention is key. We know that target intervention is key. We know that working with current offenders is key. All these present their own difficulties. The same with research then. What are the types of evaluations being done? What methods? Oh my goodness, do they have a controlling sample or a controlled treatment? What size? How did they recruit them? Did they pay them? How did they keep them involved? How did they collect data? How did they analyze it? What limitations are they willfully sharing with us on all the mistakes they made in their evaluation that went from probably five years to, I'm gonna say if I'm lucky, 20 pages, but most likely some USAID 500 page evaluations that we're trying to read and review. We're also applying a mixed method appraisal tool to really think about what's the quality of this intervention. Just because you did something, even if you did it with a recognized method, doesn't mean you did it well. That's the same then with the programming. Just because you ran a program doesn't mean the quality of that program was well done. Are those results the same as a program that was very well done? How do you compare that? What does that mean? So really trying to understand what quality looks like. We are utilizing a mixed method appraisal tool that's been developed in Canada, exciting neighbors to the North, that tries to kind of give some systematic approach to assessing quality across each study. So we can bring that into our analysis. Not just saying, on the one extreme, we know nothing because everything's localized. To the other extreme, we know all this and we've just combined it together and pretended that all the quality was the same. We developed 19 right now theories of change in partnership with a variety of experts to try and understand what do across these different targeted responses, what are we presuming the programming should look like? If we are addressing in prevention, X, Y, Z, what are the expected outcomes we think are going to happen? We find this very helpful when doing types of larger meta components or reviews like this because the individual activities are often different, but they may be going in with a similar theory that we wanna broaden that aperture so we're not just focused on if you do this specific intervention at this time with this group, you're gonna succeed or you're gonna fail or that's the important. Trying to kind of again, like I said, broaden this. So we have a better information because for me to tell you as an implementer, well if you do this in this context with this person at this time, it's gonna work. And they're gonna come back to me, well I'm not in that context or I can't touch those people or all the other reasons why they can't do it. So how do we broaden that to at least get us closer to better than perfect? I am not gonna go through these theories of change mainly because there'll be a lot of haters in here and you all wanna pick apart the words but we are definitely gonna share this in the document and something I'm really excited to go across these different five kind of be building out and thinking about because we put these things out there so that they can be refuted and we can do better not because they're the end point and the only way to do it. I do wanna touch very quickly on my last time on some of the challenges which anyone doing research in here, you're gonna know these. Definitional boundaries, I already touched on this a little. What does it mean to be in a prevention scheme? What does it mean to be in a reintegration process? Who is being reintegrated? How do we define those terms and how do we deal with all that blending together? Intentionally or unintentionally? Programmatic constructility. I'm keying this term probably not the first but I'm gonna say so we can all put a little asterisk Jessica Baumgartner's music. One of the big issues we deal with is the intentionality of a program. If you say that you are directly working to reduce violent extremism, to prevent violent extremism, to reintegrate a person but your program is not actually dealing with that at all, is that the same amount of evidence as a program that went in intentionally to work with that risk individuals to prevent an attack? How do you compare results like that? When sometimes we just are like, oh, I'm working in a violent extremist context therefore I'm doing VE and that's gonna come up with my nom de jour, give me funding for that because people wanna work on that. Is that the same thing? Do the results and the intentionality of the programmatic design matter? I would say yes, because I'm an evaluator. But what does that look like in a construct where a lot of times we're just trying to do our best effort with very little programming and design intentionality. Integrated PCVE programming and theory of change evidence gap mapping. As I mentioned in the beginning, disengaging, de-radicalizing, reintegrating, rehabilitating. It's very hard. Most programs don't separate those concepts out or they're attempting to kinda in peace building either, I'll say the kind plant a thousand flowers, see what blooms or what I like to say is throw the kitchen sink at a problem and hope something comes out from it and then we've succeeded. How do we separate that out with intentionality? We know that theory is so important and we're doing research. We're trying to understand those underlying causal links. Our violence prevention colleagues have done so much better in some ways and they have a longer history of evaluating that and understanding those theories. But when we throw everything at it, it's very hard to understand what do we replicate? What do we promote again? What do we invest in? Not because we're trying to nickel and dime, but because we all have limited resources, whether it's time, capacity, money, funding, blah, blah, blah. How do we do the best we can for the people we're hoping to serve? And then finally, evaluation reporting. My last very quick little shtick, which I know my colleagues who have done full text coding. Everyone reports differently. Everyone shares things differently. Not only, you know, how they talk about it, but do they discuss their sample? Do they discuss their targeting approach? We again do a disservice to our field when we don't have some unity around that to also help us improve. And I know a lot of it's because we're trying to hide things that probably didn't go well. Whether through our qualitative components, we didn't fail. We learned how not to do this. Or this was in a great adaptation phase. We ran out of money, we decided to do blah, blah, blah. How do we be truthful enough that we can help grow the field and not replicate mistakes, but also be open enough, especially as researchers, to acknowledging there's no perfect study, particularly in prevention and areas like this, that we can at least go hmm, how do I do that in my program? So I think learning how we can better report on our evaluations for some greater consistency to help us grow is a really big challenge. And I do often see that it's either one extreme or the other. We're trying to hide a lot, so we throw 500 pages at it. Or we try to hide a lot, so we spend five pages on it. So how do we hit that kind of sweet medium to kind of acknowledge? So I'll leave it there and hand over to my colleagues. So many great points. I'm just gonna tease out a couple. The first day, I really appreciated your comments about intentionality, and I think in the CVE space, that's one of the most challenging. That if you look across programs, most often you'll see things like social cohesion or community peace building. You won't see CVE even when the intent is specifically to reduce the pull of violent extremism. And so that intentionality question, I think, will persist, particularly as budgets specifically designated for countering violent extremism strength and other things like strategic competition kind of rise up as important. Sorry, not sorry, about the 200 full evaluations. Because I think the, I'm actually delighted that there are that many. And I think one of the things, if you look across the entire peace building field, if you look at gap maps, you'll see these tiny little bubbles, right? Because they're one evaluation deep. We as a community have not invested significantly or sufficiently to really understand what works. And that is a see away from other development sectors, water and health. They can tell you what works and what doesn't work. And we started out with some of the challenges, but it's incumbent on us to find those opportunities to use creative methods to make sure that we can tell the story of our impact. And so I'm actually very pleased on that. And then the third point I just wanted to highlight was this question about reintegration and where do the lines stop? Reintegration into reinsertion into rehabilitation, into repatriation. It reminds me, for those of us who worked on these issues of the DDR of old, where the reintegration component was often given short shrift because the focus was on the disarmament and the demobilization. And then it was like, okay, well, you guys will just find your way back to society. We didn't measure then and it's serving us ill now because we don't have the lessons that we're trying to apply these models to a very different phenomenon. It's almost RDD, right? We're not asking extremists to disarm. We can't really anymore. They're hardly demobilized. It's the question of a gang leader. You're MS-13 until you die. Whether or not you act on that is a different question, but your identity is shaped. So we're not demobilizing necessarily. We're not breaking chains of command. We're not disarming. We're reintegrating and we don't have the lessons of history to help us figure out how much of that really needs to be done. And there's a lot of focus on de-radicalization when many of the adherents to moniextremism were never radicalized to begin with. They were forcibly conscripted into these movements. And so de-radicalization is completely ill-placed for them. So I think we have a lot to learn, particularly about what is tertiary prevention in the violence prevention space and what is disengagement in the CVE space. Chris, can I pass the mic to you? Yes. Well, thank you, Julie, and thanks for such great panel presentations so far. I will try to put an exclamation mark on what they've already said. And I've been staring at Robin Nelson here in the front because, especially during your presentation, Jessica, because, and we were laughing, everything you were talking about, we were like, oh yeah, we've had that problem. So I'm going to start my presentation here. It's entitled, Hyperlocal and Wicked, Challenges and Opportunities for PCVE, Evidence, Generation, and Use. Hopefully, if you're here, you know what PCVE means, preventing and countering violent extremism in case you don't. This is really a 30,000-foot view of the challenges that we're up against. And I'm going to try to speak to both those of you in the room who generate evidence and those of you who use evidence or some combination thereof, to hopefully give you some hope about what I think are some simple, maybe not so simple, but at least simple in concept, things we can be doing to improve our practice overall. So I'm going to briefly describe what we're dealing with, talk about three ways to overcome some of the challenges that the panelists have brought up and three opportunities for improving our evidence, generation, and use, which some of the panelists have also already touched on that, so thank you for teeing it up nicely for me. So what are we dealing with? Riddle and Weber coined this term wicked problems. And on the slide here are the 10 characteristics they use to describe a wicked problem. And VE and PCVE are wicked problems. I'm not going to talk about all of this because we'll be here all day. I'm just going to focus on number seven. Wicked problems are each unique. In PCVE parlance, we call wicked problems hyper-local. That's really what we're talking about. What does that mean? And what challenge does this present to us as researchers and practitioners and policymakers? First, as Jessica said, even the definition of violent extremism varies by context. And I used to think that was stupid, but now that I've been around long enough to see it on the ground, I realize actually it makes a lot of sense. That also means that what PCVE interventions work in one place or even time varies. It will change over time and it will be different in the context as we've already discussed. Third, we are dealing with multiple causal complexities. Things like equifinality, multifinality, bidirectional causality, which I can define those later if you're unfamiliar with those terms, but they also really form a challenge to us trying to do these systematic reviews. In fact, these challenges inhibit our ability to compare across studies in many, and I applaud Jessica for your effort because it's the most robust thing I've seen to date and I know how hard this is because I've had to try to do this, so really applaud you. It necessitates ongoing data collection because the complexity, the hyperlocality means that these things are changing on the ground often rapidly. If you don't have ongoing data collection, you're going to be missing out and you're not gonna necessarily know that you need to adapt your program, right? And it also requires methodological innovation and upskilling of our monitoring, evaluation, research and learning staff. Across the international development sector, both here in the States, I count myself as someone who's had to reskill, upskill, learn new stuff, as well as definitely in the field where we're working. Finally, just this 30,000 foot view that I'm gonna talk about, I'm really talking about challenges and opportunities for developing the body of knowledge. That would include things we think we know or are trying to know about programs, policies, places, spaces and time. Okay, so the first way I think we can overcome these challenges is to when we are generating evidence, think about how we can design our research from the beginning to facilitate better comparison. Sometimes we forget to even put in our, what we mean, our own definitions and our own study or our own evaluation. That being on the other side, trying to do systematic reviews would be a huge help if we just make that a more firm practice. Jessica talked at length about the challenges to comparability and ways to overcome them, so I'm not gonna rehash that. I'll just point to the bar graph thing on the slide here. This is I think another thoughtful way or innovative way we can think of comparing studies. This is a recent study by Lauren Van Meter and Thomas Scherre called preventing and countering violent extremism, assessing missteps and promising community approaches. The way they chose to compare studies was to look at the quality of the research. I know it's really small, but you can see the quality indicators that they used on the graph. The second thing we can do is contextualize. Both Mark and Jessica have already talked about this in slightly different ways. Jessica, in terms of the program construct validity, Mark, in terms of, which I completely agree, the perspective of the violent extremist organizations and the people joining them. So when we're thinking about contextualizing our evidence generation, I think we should be asking questions like what was the violent extremism context that the program sought to address? How were VEOs recruiting? Not how did recruits say they were recruited, but actually, do we know how VEOs are trying to recruit? Was an assessment done? And if so, what were the findings of the assessment and how well did the program design respond to the assessment findings? I have almost never seen an evaluation take that soup to nuts. So we are kind of in the dark when we're thinking about programs that work because we don't really know the assessment that produced the program design in the first place. And I could go on and on about context, but those are just some highlights. Another way we've been thinking about how to overcome some of these challenges is to rather than figure out, okay, because I agree, Jessica, telling someone this is gonna work here with this group of people in this context, that is probably not gonna be helpful. But what we can do is generate, we have this document at USIP called the violent extremism disengagement and reintegration action plan. It's going to be published this summer, so hopefully we can share it out soon. It's a menu of options of what could work, not what will work, what could work based at the individual community and context level. Now, we don't intend to give that to practitioners and be like, here you figure it out, but I think there are interesting ways we can do action planning with local stakeholders and FHI 360s sitting here right in the front row. They have, we have in our toolkit ways to do this that have been tested in many locations and many sectors that we can be borrowing so that if you had a menu of options, you could engage in these local action planning activities and let the local community, including youth who would be presumably receiving these programs, let them decide what is a priority and what programming needs to take place in their communities. Finally, I want to briefly go over three opportunities for improving evidence generation use. The first, which Trish already kind of touched on, is the need to strengthen local researcher and Mel's staff capacity. This would greatly, I think I'm preaching to the choir here, I'm sure, but it would improve the quality of rapid and regular data collection by people with local expertise and local language skills, which is really important. It would infuse their perspectives into the research conclusions and recommendations that are often missing or filtered through people like me who may or may not know what I'm talking about in a particular context and it would reduce the cost of international data collection. We can also do better at sharing, I think. So often I've been asked to do research or an evaluation and then we think about dissemination. I think planning for dissemination and knowledge sharing from the beginning would be really helpful. We need to ask how will this evidence be used and the sooner we can put a plan in place for the research sharing and utility, I think the better off researchers would be able to kind of like plan for that from the beginning and there would be resources for the road show as we like to call it. Finally, I think both on the evidence generation and youth side, there's opportunity to integrate. On the evidence youth side, I came from Catholic Relief Services and had to work across a lot of different sectors and I learned about this effort happening in the humanitarian sector. So I'm gonna tell you a little bit about it in the interest of time. So there's this interagency standing committee. It's the long, if you don't know, humanitarian sector like I didn't. It's the longest standing and highest level humanitarian coordination form of the UN. In 2010, this organization called for a quote, paradigm shift in humanitarian assistance in urban areas based on a community-based rather than an individual beneficiary approach. What this meant was basically let's stop doing sectoral individual responses during disasters and let's start thinking of the whole community, where the schools are gonna be, where the health systems are gonna be, where it makes sense to put them and like really think about building, designing cities and the way people will be accessing all of this stuff. I think for PCBE, we can borrow from this paradigm shift to be thinking more about PCBE programs within the broader development and political space. And we can also be thinking about how different reporting that like a USAID or other donors is requesting, how they, I know we have learning agendas, but I think a lot more could be done to plan through how all of the research is going to like lead into a body of knowledge. And then at the, just the last comment on the integration at the evidence generation level, I don't know, it's probably been 15 years that PCBE and CT researchers have been shouting the need to be more interdisciplinary in our research and we're not doing it. But I think there is a lot of opportunity to grow in our ability to be interdisciplinary. So I thank you and I will turn it back over to Julie. So many great points. I'm just gonna pick up on a few, particularly the last one about overcoming sectors. I think that was Rob Jenkins point this morning for those of you who are in the plenary about the importance of moving out of our silos. I also think it adds to the challenge of measuring progress in any of them. I really appreciated the several points you made about strengthening local capacity, particularly for data collection and the need to link assessment to evaluation. And here I'm just gonna offer a shameless plug for USAID's new violence and conflict assessment framework which replaces the decades old CAF or conflict assessment framework where I think we've done a pretty good job thanks to many of you of widening the aperture from the sort of narrow identity based understanding of conflict to one that incorporates narratives and interests and incentives and really gets better at this linkages between individual and community level violence with state level conflict. So it will be coming out for external comment as soon as we can figure it out to get in on our website. So please look out for that. And with that, we have about five minutes for questions. Please raise your hand or put something in the chat. Gentlemen up front. Thank you, thank you very much. Benjamin Petrini from the International Institute for Strategic Studies. I have two questions. One in the spirit of what Chris just said about breaking the silos. And so what are some lessons learned that all these practice and knowledge can be applied to other non-state armed groups beyond violent extremists? And the second question just for Mark, fascinating analysis about pop culture. Who is feeling the void, that alienation void that you spoke about today in some of the other conflicts? I'm sure you've made some reflections to the current context today. Thank you. Comments on the first question. The first question was for everyone. Oh, well, I'll try. I'm just gonna give you my personal view. I have come to, given the fact that we know radicalization is not really what we should have been looking at all along. I have come to really question the exceptionality of terrorism as an act of violence. So I think a lot of what we've learned, I think there could be a lot of opportunity for cross learning across violent actors in general. I mean, everything that Tricia was saying, I was nodding along, like, yep, that sounds familiar. And that actually feeds into work, new work that we're doing. And we actually, I'll see your 200 articles and raise you 500. We've got about 740 research articles that we're coding right now that try and look across the spectrum and get more to those motivations and incentives at the communal social and economic level, so those root causes. But I will put a challenge out there to just say we don't fund longitudinal research. We don't study these things over time and so it's extremely difficult to know if any of the prevention things we talk about as root causes are gonna pay off. So we need to fund longitudinal work. I mean, I would just flag that I think there's so many crossovers and peace building is just so much in its infancy in some ways in terms of research development. My background's actually as an economist studying pro-social behavior development and investments. So not saying that it's all about the money or it's all about how you're going to fund yourself, is it? But there are many studies and disciplines that are trying to look at what motivates us in different ways to behave rationally, irrationally, all those fun things. But I think there is a lot of overlap that we can apply both what we're learning in PCVE and peace building and violence prevention to other disciplines that are also questioning these concepts. How do we have cycles of corruption in a country that's been underdeveloped for many years? How is that related to investments in self compared to violent, as Mark was talking about, purposefully delegitimizing a government in order to fund your own pockets? What is the implication of that over time? So I think our problem is that we get very, we put blinders on and we look ahead because it is, you know, the good news about research is you have to clean the data. You can only work with clean data. The bad news is all the funds and the dirty data, but it's very hard to invest time and space into those complexities. So I think what's happened about filling the void today compared to before, that the previous era that I was looking at, so the 90s and the early 2000s, and I would say in the 80s as well, you had Bob Marley and Tupac were almost everywhere in these rebellions. I mean, they were really, really popular, not team or less, very interestingly. That was different. But I found that to be true. The two paired together in Eastern DRC in Palestine. Rambo was very big in the Philippines with rebels. So you had this triumvirate that really were linked in very different places. I don't see that now. I know that for many years, Get Richard Die Try-In by 50 Cent was, it took over Juba, South Sudan, to a degree that was extraordinary. And to this day, there was a song criticizing him saying why, by Emmanuel Jal saying, why are you fueling the violence? And 50 Cent, to his credit, went to South Sudan and started working with alienated youth with his foundation. So, which was really, really quite dramatic that that whole took place. But I think you'll find Marley to be popular everywhere up to this day. Tupac to be popular in some places and not others. Rambo's out. There's new films. And then I think you're gonna find local groups. So, I just wanna give you an example of why rap is so important as a medium. And why, I think it's important, you know, one thing I found working on youth that a lot of older people don't like rap. And yet to understand what's going on, rap is actually the mode of expression for so many people, young people globally. And one of the ways you can find out really fast what's going on, if there is a little bit of peaceful descent available, is to find out what people are singing about or what they're listening to. And if you ask young people to explain it to you, if they become your teachers on this, I mean, you can really learn a lot. And I think that the local themes would come out, but also the way global icons are being interpreted locally, that's the thing that's really so compelling. One thing I wanted to say is that you were mentioning de-radicalization. And there's, when you think about alienation and how great it is and how working through communities is such a perverse way and a way to reach the people that are being recruited by gangs and violent extremist groups because they're working with the outsiders and they're really good at recruiting. It's extraordinary how good they are. That we use words that make it seem like it's easy. So one of them is reintegration. How many of these kids were, that presumes that they were integrated before this all happened. That's bullshit for most of them. They weren't and yet we like that idea, reintegration for ex-combatants and so forth. It's just not true and for a lot of young people that they've been on an outside all the time. The last thing I wanna say is I did a study called Youth and the Field of Countering Violent Extremism and found that if there was no PCVE, most youth wouldn't join anyway. What we don't really look at is why they're so inherently, most of them, they resist engagement in almost everything. If you look at it, what they call, right, they're like rats, right, gang infested neighborhoods, they're called. Generally speaking, it's about 5% of youth are members of gangs, another 5% might be affiliated and 90% have nothing to do with gangs in the neighborhoods where they're active. And I found this to be true with violent extremist groups too. I think there's this feeling because we don't know youth and especially alienated ones and we think they're bad, especially the boys and a lot of the girls too, right? They're pregnant and all these things, they're all having drugs and so forth and they talk back and all this stuff. But why don't they join? I wish we knew more about that. I mean, I think that that is inherent in most young people's lives is they resist engagement in violence. There's something inside of them worldwide that says, I'm not doing it. And it's something for us to think about, we're not saving anybody. We might save a few, it's hard to prove prevention as we know, but I think the bigger picture is is we have so much to work with with young people because elementally, no matter, I mean orphans that have been treated so badly, their whole life, they're inherently peaceful, most of them. And it is something that I think we could really build on. That's a great place for us to end today's session. Please join me in thanking our panelists. Please enjoy the rest of the conference. Just a flag for everyone. We're gonna be moving into lunch. We do have a lunchtime book talk that'll be held in Carlucci with Dan Rundi. It's not the full time, it's scheduled for 30 minutes. So please make sure if you are going to attend the book talk, there can be no food or drink in Carlucci. So I encourage you to either wait until after to get your food or to kind of skedaddle right now. But thank you all. Thank you, Joy.