 Right, we now turn to this, which is launched today. We're very excited to be proud to launch this report for a long time coming. I think all in all, it took a year to develop this. It's only 70 pages, which I'm sure you'll read page by page tonight over a glass of wine. But now I'll present to you some of the highlights, some of the key results in the methodology. I'll full present this with Philippe Boulanger, it's in first row here. We hired Philippe, who is working for the World Bank at the time, as a consultant to work with us on this project. Develop the methodology, implement data collection, analyze and do the whole thing. So I'm really happy we're going to be here today, we're here today, we're here today. Quick thanks to Dona, the owner, who I already presented this morning. This is one of the donors, other donors that have contributed to this, to make the study possible with the event that we take, and the journey in terms of the contributions of staff time, yes. Right, so why evaluate the campaign? When we ended this field two years ago, or so, we saw a lot of activities. We saw a really sharp growth in campaigns that are implemented, are divided by a lot of other providers who are very active in this field. We saw an increase in funding by governments, by the EU, and others into the type of activity for where it's related to information campaigns. At the same time, we were wondering, what do you actually know about how this works, what aspects work, which aspects may not work, what is the evidence that is available to us in this field? We read the Dutch report with great interest, and so on. There's a lot of skepticism in this field, there are a lot of critics that are skeptical about the impact, about the assumptions underlying this approach. We said there's hardly any empirical evidence, so as a data center, we started working in this field and trying to improve the evidence. The start of the project funded by DFID is now moving in the growth of this. We also see a push for better evidence and better analysis in the GCN. The global public for migration really enshrines the data agenda, and the agenda to improve evidence, and actually in one of the goals it mentions information campaigns directly, it says we need to improve the evidence base for these campaigns. So this is sort of the context where we start up this work. And this is just a quick impression of all the campaigns that are out there. We have websites, we have radios, we have town halls, we have moving cinemas, we have a lot of activities online with billboards and flyers. There's a whole range of interventions here, but as we said, when we started the evidence, it was quite limited on what works and what doesn't. Now, before we started, we did a systematic review of the evaluations that we already have. So these were evaluations available or published in journals, but also in great literature funded by governments, evaluations that the IRF has done previously and others. And we not only look at the results, but also look at the robustness of the research, the robustness of the evaluations, and well, we came to quite a not so positive summary that a lot of the evaluations that we rely on, and the policy we're made to rely on, are limited in the way that we can draw conclusions on them regarding the impact of such studies. And that's why, well, let's get into this work and try to improve the evidence with a new study. We'll share the slides later and for all those listening online, by the way, my colleague who started will share the link to the study as well, systematic review was published last year. Every comment for all the attendees listening online, thanks for tuning in. I don't know if you can see us, I hope so, I hope you can hear us. Please tune your mic. Sometimes everyone here present can hear what you're doing at home, so please make sure to tune your mic down. Thank you very much. Why am I a semester? So we entered this field and we're really approaching this with limited knowledge about what's actually happening in the field. We were browsing for campaigns to evaluate and we're looking at the whole range, the whole portfolio, then I remember also, outside of IBM, and we came across this wonderful campaign that you just heard about, and it presented any amount of new presented on, and we were really impressed with this campaign and thought this would be an ideal candidate to actually do the evaluation on it. First of all, it's a very innovative approach. We just heard about it here, it's, there's an emphasis on the local community, local ownership, it increases authenticity and trust, and this we thought was an interesting mechanism to look at and to research more thoroughly. There's new contents of the videos that are to be used themselves. There are very targeted measures, such as town hall events that we'll talk about more in this study, and there was also the potential for scale up, and I think impact regulations are always useful when you're thinking, you wanna know whether a certain approach works and you're planning the long term to actually scale it up, and before you scale it up, you wanna learn about which aspects work, what are the aspects where you need improvement before you scale it up. Now, how do we measure impact? And you're gonna hand over to Philippe to explain how we approach this, how we design this study, and then make out the same results of the report. Thanks for having me today. So, there are many types of evaluations, exposed evaluations, process evaluations, before and after evaluations, and we argue that impact evaluations actually best evaluation method to evaluate the impact of the project. There's so-called randomized control trials of the soft form of these impact evaluations, so sometimes we use those synonymously, although it's a slight difference, but RCTs are basically the best way to do impact evaluations when possible. They try to answer the cause and effect relationships. Basically, the main guide and question is what would have happened in the absence of the project? That's the true impact is if we could compare the state of the world with and without the project. So, ideally, we would jump into a time machine and observe two states of the world, one with the Magnum's Messenger project and one without, and just see what is different between those two worlds if we hold everything else constant. Now, I quickly check time travel is not yet possible. So, basically, RCTs are the next best thing to simulating a time machine. And randomized control trials are also used in other fields. You know from the medical field, there are basically no drugs that can get, that can enter the market without first passing through a randomized control trial. And this now also cuts in fire and a public policy to make. And, yeah, with this report also at IAMM, so that's exciting. So, I joined IAMM as a consultant in the last year in July and initially my assignment was just for a month. And then at the end of that month, I talked to Amy and William in the car and said, I think this could be a great opportunity to do this type of evaluation. And at that point, Jan also had already talked with Jasper about opportunities to do so. So, I just wanted to commend the IAMM to do this, Amy and William Creavager, who was absolutely instrumental to make this work without him, this project would not have all this evaluation done, not as successful. And also the donors, of course, for being adventurous and doing all of this. And I also, I remember talking to William when we decided to do this, and I said, William, he was a project manager in the car. I said, you are aware that there's a risk that you cannot show any results, right? And he's like, ooh, okay, let's go ahead. I'm ready for that too. So, you always need, when you do these studies, you need project managers that are open-minded enough and also can accept failure because we also can learn from that a lot. This was not a failure, but it was nice. So, let me quickly talk about other types of valuations to then come back to why impact valuations is important. And if we think about how valuations are done at the United Nations or other institutions, it's often the form of, before and after valuations. So you measure your outcome value X before you start the project, and then again after the project, and you assign all the impact to that difference. And we say that's not the impact at all of the project because the main problem here is the problem of confounders, I think, which is that during that time, before and after, there are many other things that can happen at the same time. It could be political developments, social developments, the economy could go up and down, natural disasters, et cetera. So if we only assign all the difference before and after to our project, we make a mistake because we forget about all these other influences. For example, imagine a training program for unemployed youth, and we measure what happens before that people are not employed, and afterwards everybody's employed, and we say, wow, the project was super successful. We might mistake, that might have been cost by other circumstances, and in reality, our project might have done nothing to help. So the core, the problem behind this is what you've heard many times, probably the correlation is not implied in causation. Just because two things move in sync does not mean they're connected. Unfortunately, this thinking is hard-wired in our brains. It's very hard to get out of our brains. And we also read that in the news all the time, American president, this one or the previous one, he said, oh, I created X amount of jobs, so the big newspapers would write this also. Not in bad faith, but because there's a fundamental misunderstanding of these concepts, and it's very hard for you to do that. But this is why it's very important to think about this harder and go to the bottom of it. So what we need is basically a counterfactual, a state of the world, basically what I said earlier, what would have happened without the project. And don't get me wrong, other evaluations are also very important. Process evaluations are also very important because we still need to understand how many people showed up to the events, how many people were trained, how many, were the funds spent reasonably and can we account for everything? So this is not to say that other forms of evaluation, especially process evaluations are useless, not at all, they should go hand in hand. So the main reason to do an RCT is to create this counterfactual. It's a very, very easy concept, very easy. It's hard to implement, but it's a very easy concept. We make use of the law of large numbers, which is a reality of the world we inhibit. And it always holds across all contracts everywhere, if you, and that's the basis of the RCT methodology. You start with one big group over here, and you literally flip a coin and you split it up into two groups or more as you have a larger group to start with. And what gives you is two identical groups. And all the confounders that I spoke about earlier balance out. And this, as I said, this holds all the time since it's a basic truth of the physical world we live in. And the interesting thing is that it also balances out things we cannot measure, which might be even more important than the things we can measure, because we can measure the economy going up or down or certain political developments, but there's a million things we cannot even see that impact our lives. And so that's also very important to understand with this method. We do balance those things out as well. We can't quantify them and we can't see them, can't help them, but we do know they're balanced out in the two groups. So that's very important. Because if this is successful, then the only difference between those two groups is the project that we're rolling out. And even after the fact, let's say we have two groups established two groups that are exactly the same and now big earthquake strikes, we can also be confident that that earthquake affects both groups similarly. So the bias is limited, there's no bias, even though it might be big events happening in the pathogenization, this is all in the ideal case. And so that's how it happens. So this is the premise of this method. So then in the end, you have these two groups, you roll out the project and you collect data and you prepare the two groups. And because the only difference between the two groups is the project, all the difference in measure is the impact of the project. And this data collection happens to be the most expensive part of the impact evaluations, along with higher international consultants to help you do this. So that was my bit on the pathology. Now we want to work actually there. So we couldn't evaluate all aspects of the project. We focused on the town halls, which were a part of the project because they were a very important part of the project and also a controlled environment that we could use this methodology with. And we had two twin goals actually that are related, but not quite the same. The first one was, did the project work, right? This is what we wanted to know, this does this work or not. But the second one was also a briefly touched on this and the break of youngness, kind of to do a proof of concept. Can this work at all, right? Because as Jasper said earlier, we basically have zero evidence on any of these information campaigns. So it was important to establish that if we show this peer to peer content, does that move the needle at all? And I talked to William when I was there, I said, we use these tunnels to really hold down the potential migrants with this content, right? Because we have the town halls, we have the discussions, et cetera. So we need to throw the kitchen sink at them to see if it works or not. So because if we couldn't establish that and we were pretty confident that this is not a useful approach. So we created the, I don't know if this was mentioned. So the movie, we created the documentary movie out of the snippets that were already collected with the community response app. So in order to do the town halls, we put them all together into a 15 minute documentary movie, kind of the best off of these clips. This movie is available on YouTube. I think the link is in the report. I encourage everybody that works on migration to watch this. It's really amazing that the group and the card have a great job putting these together. And it's a very touching movie. It has English subtitles, so it's a great watch actually. Because without that, I think we always think about migration at very abstract terms. But once you've set through a 15 minute movie with all these people telling about their experiences, it really changes your perspective really further. So we had this movie, we made this movie and then we had the in-person discussion right after that. Yeah, I think this was mentioned before, our kind of theory of change was this emotional, relatable content of people that look like the people that are about to migrate would have a higher impact than just somebody lecturing from the top. And what we also saw is that in our data, and we had less data, we used one before, that people have a generally, very high level of awareness, but in an abstract sense. So the goal is to make this kind of more concrete, feasible. Briefly on the outcomes that we measured, we measured subjective information levels, factual knowledge about migration, risk perceptions about migration, perceptions of returnees, because there is stigma attached to returnees that didn't make it, and as more of you also mentioned, and lastly, intentions to migrate and just a little talk about the results of it, and then shortly about how we did, we selected eight neighborhoods in the car that were migration-prone, in which overlaps with poverty, I guess, largely, and we talked to local decision makers and politicians and asked them to collaborate with us, provide us venues so we could do these screens, and we decided to do 36 screens overall, and 18 screenings were of what we call treatment, our movie with a discussion with the returning migrants afterwards and in-person discussion, and then we also had 18 placebo movies unrelated to migration, so we showed them a movie and did it, always at the same day, two screenings, one treatment, one control, and we showed just a movie about the San Luis Society, and we did it in order to get people to these screenings with random walks in the car and invited over 8,000 people, 8,800 people, and each time we invited somebody, we flipped the coin, we did this on a tablet computer, but every time we invited somebody, we didn't know whether that person would be invited to the treatment group of a control group, and that way we created these two groups that I talked about earlier. The only criteria that we had for inviting people is it needed to be under the age of 35 and express a minimum interest in migrating and also in attending a movie because we wanted to make sure that our attendance is sign up, and then we collected, while being invited, we collected some basic information, social demographic information about them. And this is my last slide, so just to give you an overview of how it turned out, so we invited over 8,800 people, and I think 1,400 showed up in the screening events, so we really needed to invite a lot of people for people to come out because it was very, even though people said, yeah, there would be interest in a very low level of commitment, so we had to invite a lot of people to make sure we get those people through the screenings, and then we found basically 1,200 people three months after the events, and then we did them again, and we did the small comprehensive data collection, and because of some, you know, fishing as people opposing against different people or people couldn't be found, et cetera, or partial surveys come through, et cetera, we decided to be very conservative that we had to use a smaller sample coding analysis of 924 people that kind of went through the entire process of being invited, showing up to the movie, and being re-founded, and with that, Jasper, on the results. Okay, thanks, Mr. Lieber. For those interested in the methods and all the, say, no details, you know, then there's a whole section to work explaining how we did it exactly, and what most people will never probably see, there is an 80-page annex, a typical annex for those who get really excited about tables, and figures, and footnotes, but this is gonna be online and make this in the report. All right, so let's jump to the key results. As Lidia said, there were four sections that looked at how well people feel informed subjectively, how well they think they know about migration to Europe, the regular migration, so forth, about the risks involved. What do they know? Is there some sort of factual knowledge as that we can do, you know, comparing the information that potential migrants have against some information that we have with, yeah, some accuracy, and then, I would say the most important is the study is to assess the risks of the sections of potential migrants. How risky do you think it is to try to be regularities to Europe where the specific risks involved and how likely you think they will occur? This was one of the key areas, and then lastly, there was the intention to migrate regularly, so I'll go through that. In a nutshell, here are some of the results that we found. We found that participating in those events increased the how well migrants feel informed themselves. We found limited to very little effects on actually hard facts regarding migration. However, the campaign really wasn't about providing the number of people that have died trying to cross to Europe or the specific costs of, you know, this micro through the desert. That was not what the campaign was about. It was about emotional connection between the two years. Now, we see larger, very consistent effects on risk perceptions. We find a modest, but consistent effect on the reception of returnees that often face stigma, as we've heard, and the campaigns was able to improve the perception of returnees themselves. There was a limited effect on the perception of what are the economic opportunities at all? And lastly, we did find an effect on the intention to migrate irregularly. Now, to walk you through these four areas very briefly, I'll first present some context data that you also collected at A-signs, as I've said, to frame these questions, and then the second slide I'll present the impact estimate. So, and of course, there's a lot more in the board, a lot more details, so this is just some of the highlights, the questions that we asked. We find that, does that work? Yes, okay, sorry. So we find that 43% of respondents did not feel well informed about how to migrate to Europe. 37% they were not well informed about the risk associated with migration. So, in the discussion, this is high, or this is low, in the literature there's a, especially the critical point is to say, well, it's already perfectly informed, they know everything, why they're running information, and when we find a different picture of the data, actually have this one, so more than a third are feel themselves, not very well informed about how to migrate, and this is very consistent with other studies that we have in my two now minutes. Yes, can I step away a little bit? Sorry about that, is this more comfortable? Okay. 53% of respondents report that they decide alone, while 25% identify parents' key influence. There's this debate in the literature and well also in the GOLA evidence of who actually decides, is it the individual, or is it the family that puts pressure on the individual migrant? We see that against the common believers, it's not always a family that puts pressure on the individual, it's often the individual that decides themselves, often without the family knowing, and sometimes against the advice of the family. So again, you see a different mixed picture of the data and what we often hear in the literature. Now the information sources, this is very consistent with the other studies that we have, it's the largest source of information, if potential migrants have information, is family and friends, and this speaks to the whole rationale in this campaign that these are the primary sources of information, those are the messages that are most relevant. The internet's fairly low, there's a lot of activity online, but we have to assess what the potential of online communication is, to really reach out potential migrants and change perceptions there. There's an ongoing study that we're doing to kind of unpack how this communication online work and what's the impact of that compared to offline approaches like the tunnels. Now the impact estimate, we find that participating in the events increased the subjective information that will potential migrants by 19%. It's a 9% point difference between the MAM events and the control group. We find a 16% effect on how well migrants are informed about migration to Europe more generally. Factual knowledge, and I should say as sort of a word of caution here, there isn't, it's debatable whether there is actual, factual knowledge in this area, because we don't know accurately how much does it cost to migrate to Europe, how long does it take? There's such variation and most of the information that we have from the studies is quite anecdotal, I did not represent it. We also had a project that didn't actually just try and count the number of missing migrants. And of course this can only be a conservative estimate of what the true number is because we will never know how many migrants really go missing by trying to migrate to Europe. So it's debatable whether we actually can compare the knowledge of migrants regarding migration with accurate facts, because the accurate facts of course are debatable. But there's some questions we ask for example regarding familiarity and knowledge about the silent procedures and the legal context and 100% some results in this regard. So since 2014, an IR project estimates that there are at least 22,000 migrants that have died trying to reach Europe from West Africa and North Africa. 26%, when we ask potential migrants in Dakar, 26% say they don't know. And another 43% say they estimate that it's less than 1,000. Only 5% of potential migrants in Dakar estimate a number that's anywhere close to what we know from the IOM estimate. And the IOM estimate, as I said, it's likely to be a very minimal estimate because there's a lot of casualties that go unrecorded. Now the average asylum recognition rate for synac elites nationals in the EU between 2018 and 2018 varies somewhere around 10 to 25%. So it's fairly low. Yet approximately 73% of respondents did not know what the site was. And among those that report some economic reasons for wanting to leave. So those reasons that do not necessarily grant you sort of international protection and access to refugee status, 40% thought they would be eligible for refugee status. So we see regarding the legal status and the legal context of migration to Europe and some of the facts surrounding it, there's quite some misinformation. Now regarding the impact estimates, we see very limited effects on how accurately can you estimate the number of casualties, when some of the silent procedures. And of course we were a little surprised that we were thinking more about it and realized, well actually when you watch this 50 minute documentary and you listen to the discussions of the participants in those events, it wasn't about how many, are there 22,000 or 15,000? Or it wasn't about how you apply for asylum and what's the whole procedure. It really was about the personal testimonies and emotional experiences of potential life. So in that context I guess we wanted to surprise that we see a little effect on factual knowledge. We turn to risk perceptions. Now this is combined in context and the impact estimate. We, first of all, we find a very high baseline. What we mean by that is if you ask potential migrants in their part of how risky is it to migrate to Europe, you get a very high percentage of people, it's a very high percentage of people that say it's a very risky, very risky, critical risk, a very elevated risk. And here we only see the percentage on the graph here, the percentage of people that say there's a high or critical risk. And you see it's often over 50%, around 50% of potential migrants that say that, which is a fairly elevated base, I would say. However, we, one of the open questions that we couldn't quite solve with this one report is the difference between abstract risks that we think that could occur and risks that we think you might be exposed to yourself. So abstract understanding of risks more generally and the likelihood of you ending up in a situation of two very different things. So we found very consistent effects on various measures of risk perceptions and the different bars here are representing the control group that watched a different movie and the treatment group that watched the migrants' messages move in. You see across all of these outcomes, we see different solutions, in most cases, statistically significant, quite large and consistent. And here the risks of, so what is the risk of witnessing the death, experience forced labor, losing all your money, imprisonment, deportation, beating, food shortages, the risk of violence or the risk to life. All of these outcome barriers were chosen based on the movie and the topics that were discussed in the movie. And those were all risks, sources that potential migrants talked about in their own experience. And we see that participating in those events significantly increased the risk awareness. Can I ask a question, sorry? Yeah. Your conclusion from those distributions are there's significant difference. Right. But within errors, you just state the errors there. Right. Then to me within errors, there's not a huge difference between the two groups. Am I reading that wrongly? You mean the confidence intervals? Yeah. Yeah. Across, I think these are, we had seven, age of nine report, I think six were the city's native, getting up to 5% of all. Okay, because for me from here, just witness death and imprisonment are the only. Well, it's hard to see. For example here, it's hard to see from there, for example, but most of the confidence intervals are not overlapping for the, Ooh. It's not strict. It's not strict. It's not strict. Most of the confidence intervals are not overlapping and most of these differences are statistically significant with the exception of, I think it was deportation and food shortage. But you'll see all the tables with the significant confidence intervals, standard errors, we clustered on different levels with all of that nice technical, so we find it all in the NX and some of it also in the report. Perceptions of rich needs. We found that 30% stated that returning migrants should be ashamed. This question of stigmatization and it's often brought up in these debates. Well, we actually find that not everyone thinks that returning migrants should be ashamed of themselves, not everyone faces strong stigmatization. What is the size of all of the shame? And 30% think that migrants should be ashamed, which is arguably a very high percentage still. In the mental treatment group, 58.6% stated that her Chinese should be or could be proud of themselves, as opposed to 52% in the control group. We see it as small, but again, statistically it's gonna be an effect here on how positive you perceive the rich remains. Now, intentions to migrate irregularly. We find that while 50% report that it's probable, very likely or certain that they will leave Seneca the next two years, 60% report that they would not consider migrating irregularly. So, this is a bit of a tricky situation because the intention to migrate generally is very high, we found. Among our sample. Yet, to report in a survey that you would do so irregularly, which is a very solid question, of course, it's a very low, which asks the questions, well, they are remain legal opportunities to migrate. And it's also maybe a lack of understanding what the difference is actually between legal and irregular migration among potential migrants. Overall, we found that attending those events reduce the intentions to migrate irregularly by 20% relative to the control group. Now, I wanna talk about some of the limitations of the report, and I think this is very important, every study has limitations, and they should be made explicit and you gotta be transparent, and in a way to find, improve the next study, improve the design and have the opportunity to go forward in this project to fill further gaps and to improve our approach. And by being transparent about those limitations, we can provide more feedback and others to help us do a better job in the future. Some of these limitations are the same. So the timeframe is quite limited. We measure the effects of the events three months after attending the event, which is, some may argue, quite a long time if it's about watching a movie and attending an event. The event was, I think, very memorable. Most people that we surveyed clearly remembered the event. They remembered the key messages. They remembered where it was and so forth. So we would say clearly there is a measurable impact three months after, but what about the long-term effect? What about six months later or a year later? Has the events sort of produced further information seeking behavior that leads to, yeah, as a sustainable change in attitudes and perceptions that we don't know yet? Now, for this study, the scope was limited to Dakar for many reasons it was, but this was the pilot impact evaluation where we studied the effects of attending events in Dakar. Of course, our Dakar is a particular setting and Jan mentioned earlier the question of external validity. And then we have to do more to assess what are the effects of these events in rural areas? What are the effects on woman versus man? What are the effects on high school students versus, yeah, workers? So this, they were very much at the beginning with those types of questions and teasing out how generally this mechanism would apply in the context. Now, also, the study was limited as you've seen on intentions to migrate, not actual migration, which is of course a big difference. And future studies will attempt to look into that and see it's very tricky to measure, of course, actual behavior and to measure actual flows, but we'll, in future research, look into how to do that. Yet it remains, in general, as opposed to political question, or whether we want to know, as I went, whether people move or not. Last week, with every survey, there are some measurement issues. We asked some very sensitive questions, right, about irregular migration, about people's own personal experiences, risk perceptions. Those questions of course aren't easy and we have to tease out in the future how to best ask them to get the most better answer. We're pretty confident in our results, but we don't have much to work on and how to ask that question of would you migrate irregularly? I mean, it could probably make a difference on how you ask that question, how you frame it, what the wording they use in future research, we'll tease that out how to ask that question, and then this refers to measurement issues. So overall, we think this is really a great pilot study, a proof of concept, whether actually providing it near information to potential migrants, whether that mechanism can instill some change of perceptions, whether it generally works. We trust this evidence that it can and now we have to go beyond it. And I think in the next presentation, towards the end, we will explain some of the studies that we've now planned going forward. So thank you very much. We now have some kind of a question for you. A fantastic piece of work and really something to look up to and learning quite a lot about. And I have one technical question and then maybe one or two more methodological ones. So the technical question is, I didn't, sorry, if you mentioned that, what was the sham movie? What did you show people? What was the control group then? So what did they see? And then the question, and I thank you very, very much for being candid about the limitations in the study, especially around generalizability and those kind of issues. And of course, acknowledging that you recruited about 16% of the people that you intended to recruit and all that kind of stuff, but being very candid about that. My question is more about what would you, so thinking back on this experience, what would you have done, if anything, differently if you were to do it again? Thank you. That sounds good. Any suggesting to take three or four questions and then answer them together? Yes? Yes. So also thank you very much. Really fantastic. And that's clear. How many people did you have in your treatment group and how many were in your control? So how many watched the movie rest of the procedure? Six to 700 in each group. So we had a sample breakdown there, yeah. So you had 50 percent. It's very balanced, yes. It's very balanced. I'm sorry, we wanted to wait to answer that, but that was an easy one, right? Okay. Any other questions? Yes? Hi, I have a question on the findings, specifically on the honeymoon risk perception and on the self-reported level of information of respondents said they felt they had a lot of migration because what I found interesting was that as we could see from your findings that the risk perception was actually quite high about even before the event took place. But at the same time, respondents, self-reported, they didn't feel they had enough information of migration. So I just wondered whether you've got any sense what the information was that they still felt was missing considering that where this perception was pretty high. Okay, maybe we take one more? Yes. I'm feeling from J-PAL, maybe one last question. Could you tell us a bit about the challenges in terms of reaching people at end line? Both sort of where they may have gone or why it was difficult to find them and anything you know about who you might not have reached? Maybe I can start with the placebo muamadu. Do you want to talk about that? Yes, sure, okay. For the placebo muamadu we wanted to take something that is nowhere related to migration in order not to contaminate or influence the placebo. So we chose a movie on the Senegalese society. It was on street children. Yes, nowhere related to migrant children. So it was a movie about, the movie is called Cheval Nong? Cheval Nong, yes. It's about the Senegalese family structure and how parents treat their children. Yeah. It's also, I think we have a different report and it's on YouTube as well. I think it was, it's a UN-produced movie. I have to check, but nothing to do with migration. But it also has the same length. So the control of it was kind of this, everything was the same. The only difference was the movie was different and we did not have obviously returning migrants speaking to the audience afterwards. What would you do differently? I would say we would test how we ask questions. I think I mentioned that how we ask questions, we would run some more pre-tests and see what variation we get in how we ask questions. I think impact of innovation is often very technical, but what really comes down to also is field implementation and there you often need more time and you spend a lot of time in the field and the whole team in their current and amazing job there, but it was under serious time pressure. And I think if we do this again and we will have the opportunity to do so, we would play and we would plan way more in advance to give us more time to set up the study and do the data collection. I would probably also go with a slightly larger sample. I think this is good for RCT studies. This is solid, but we were somehow limited in some of the heterogeneous treatment effects. So the subgroups that we look at, how is this more effective for women or men? Is this more effective for young people? We explored some of this, but we were limited in terms of our sample size and how far we can go. So what do you think? I think we'll probably go even larger. We did not understand how many people we need to invite to actually have people show up, right? So we were surprised by that. We thought 5,000 would be good enough and then we ended up inviting more than 8,800 and to print extra invitations, et cetera. So everything was a little bit kind of, we built the airplane as we threw it, basically. So what Yasper said about the size is true. I think one other aspect that could have been interesting is to compare the migrantus messengers to a more traditional inflammation campaign, to have those three, like control, traditional migrantus messengers that would be super useful. When we, again, like when we designed this, we didn't know how many people would show up. We played with that thought, but then kind of decided to be kind of more conservative and have at least something on migrantus messengers. But I think that could be something interesting for the other phase. Also, what Yasper said earlier, longer term follow up could be interesting if there's more funding available to see after three months, six months, nine months, et cetera. Also, in other fields, you know, this edutainment kind of field where people get movies to change behavior, et cetera. Many people now have that, like, this is just a one-off movie. So in the future, it'll be also something that's in the form of a TV show or social media exposure where people not just get in touch with it like once, but maybe every two weeks, every month, and to see if that has a high impact on, those are just plays into kind of what can happen in the next phase, I guess, or in future studies. But, yeah, we did this really histically, so I think we're glad that came out that well. Yeah, just to add on that, and Slade speaks to the question from over there about what is the type of information that potential migrants actually want and need? I think we can do more there, and we will do more there, because, yeah, we see this gap in how well migrants feel informed, not so well. They're also, we find, in terms of sexual knowledge, they're very misinformed, but then the risk perceptions are fairly high. But this begs the question of, yeah, what kind of offers, what information should we provide even after the movie? Because the movie raises a lot of questions, and I think there's this vacuum that we can step in to say how can potential movies that are very migrants that are moved by this movie follow up with us? How can they reach out and ask for more information, maybe more legal information, procedural information, these kind of specific types of information? Right now, in the past, there were limited opportunities, and that's what we work on to provide more of a follow up. Because they did say, we see in the survey, people did ask for more information afterwards, and even as we have an impact with this movie, maybe that impact evaporates, that they go to their friends and they tell them the wrong stuff again, right? So I think this is an opportunity for IAM to kind of have resources ready that are trustworthy that people can kind of engage with. On the risks, I mean, that's a tricky question, an interesting question. What also we saw in the literature is that people do have kind of high abstract risk awareness levels, and that is separate from the kind of individuals as this has something to do with me, risk perception. I think it's a very interesting field to do more research on, but I feel this is kind of what we see here a little bit, is that people say, yeah, it's super risky, like very risky, but I'll go anyways, because I can manage, or it's not about me, and I think this was part of the, there's one strand of thought that says that only people die or run into trouble that are kind of weak, or it's not smart enough, and everybody thinks they're smarter than everybody is, so I'm just smarter than these guys, and I think this is what we try to do with this movie, is to show, look, these guys that are returnees, they're pretty cool people, pretty smart people, they are very resilient, and if you see that, you're like, oh, I thought only the kind of, the weaker people don't make it, and you see these people that are kind of cool dudes, and you're like, wow, this could be me, maybe, you know. So I think there's a gap between abstract and kind of concrete risk perceptions, but that's kind of conjecture, I don't know, that's actually the case, I think more research could be into that as well. Right, I think we talked a little bit about what information is missing already on that question, I think, particularly about, you know, legal context, the procedures, how long it takes, how much it costs, we know from previous surveys that this is one aspect that many minds are misinformed about. On the question from Jaypal on the end line, I think we were, well, I at least was very surprised how low the attrition was, and I think that's thanks to the great work of the team in Descartes-Riva and the survey company that we subcontracted, there were a lot of sort of initial efforts to keep the attrition, so the drop out from the study very, very low. We want to... Can you maybe go back to the table, where you show? This one. Yeah, so we did like, we did callback surveys after in between like the advance and the end lines, we kind of remind people like, hey, we're part of the study, but it will come at a different point than we do at some point, and we had, at the movie screenings, we gave out like snacks, et cetera, so we kind of had some good buy-in with the people, and yeah, so it was about 80%, the areas were pretty confined, and we didn't have to travel throughout the country, it was all in the car, and by this point, we had like three, four touch points with the people, because we invited them into the screening, they filled the survey, we had the telephone numbers, the telephone numbers of their friends, but I mean, in the end, it did take time to find everybody, we did some interviews over the phone over people that were inside the country somewhere, so the first, I don't know, first 800 were pretty easy, and then it took more time to kind of catch everybody else there, and we could have continued trying to find 30 people to work, but at that point, we can't find all these others. We have a table A4 in the end, next on the page two, analysis of drop-out, and we look at who's dropping out, how many, and what is predictive of dropping out, we find older participants slightly more likely to not participate in the end line, and also female potential are slightly less likely, however, this drop-out does not vary by the treatment or the control, so we see the same sort of pattern for both groups. Yeah, and actually on attrition and drop-out, which is a huge problem for the validity of the study, it's half of the sample you don't find in the second survey, you have a problem, and we're running another study in Guinea with my colleague, Marie-Louis, who you will meet her later on the panel as well, and we were amazed how low the attrition is, but in Guinea, you were not, we were having a debate about this, but actually we were sampling and surveying very rural areas, very small villages in remote areas that are very difficult to collect data in, however, actually if they don't migrate out of the country, people are not very mobile, so if you go to the same village, have you later, you'll find the same people very quickly, that's what I've learned there, and I was amazed by the low attrition and the context. Glenn's one follow-up there, sorry, and it just sort of, you mentioned that this is a study that's very focused on an urban environment in Dakar, is there anything you can say about how you might expect either from finding elsewhere in the region or sort of from Iowa and operations in Senegal, that you might think about how you might expect rural or urban populations to be different here or maybe that's a focus of future research? Many people also go, and maybe more and maybe know more about this, many people come from rural areas to Dakar to actually use it as a hub for migrating out, I don't know, this is kind of the plan. There are some, and how they, many people come to Dakar where they live from Tambakunda, which is at the border with Mali, and if they want to take the work, sometimes they do, but for the biggest part, they come to Dakar where they can find buses that are going to Mali and Niger to all the way to Libya. And there is actually, and there are reasons of Senegal, Tambakunda, where migration is a larger problem in other areas. This, we thought about doing this in rural areas, but then decided to just focus on Dakar, kind of to do something good rather than spreading ourselves too thin and then not be able to say anything, but obviously there, what's the other area, this Tambakunda, and those two regions are super heavily impacted by my outward migration, so it would be worthwhile to do something there as well. And we will, we have a study planned on looking at what are the differences between rural and different populations, age groups, gender, so we'll take a look. And one last follow-up on inline, so I work a lot in Nigeria on health issues and doing inline surveys, I'm always amazed how many people we can find. I always say maybe one thing is that these people usually do not get to talk about themselves, it's different like when you get a call on your phone, this is a market research area, but if somebody actually can talk about themselves for 30 minutes and they never had that experience before, can be kind of interesting for people because we ask detailed questions about how you feel, what you think about this, what you think about that, and it's not everybody's super turned off just because usually people don't have a voice and we say, hey, here's an opportunity for you to say something about your situation and people tend to want to do that. Okay, we'll collect another round of questions and maybe also from the webinar, but Frank? Yeah, I just wanted to highlight one particular figure that you showed. Relating to the awareness of the risks that migrants face. And that is the figure relating to the number of missing migrants. Since 2014, IOM with the help of MCD, which launched this idea, has been implementing the Missing Migrants project. We've collected data from around the world and showed that more than 30,000 migrants around the world have perished trying to reach their destinations. It is probably one of the most high profile IOM data projects. It's widely cited around the world by international media, UN agencies, governments. But what your study shows with this new data is that the people who probably most need to have this kind of information don't actually know very much about the risks because you show that I think something like one, they think that the number of those who dies around 1,000 whereas the figure you showed was something like 22,000 in one slide. So that suggests to us that we need to be doing much more to reach out to that audience in the Missing Migrants project in the future. Clearly, despite the fact that this is one of the most high profile IOM initiatives that we have developed over the last four or five years, we're still not reaching enough migrants and they're not fully aware of the terrible risks that they might face. Thank you. Yeah, Casey, that's it. Should we, Gustavo, are there questions from the webinar that we can answer? Yes, there are a few good questions from the webinar. So I'll just read two for now. This is from Anselia from LNU Munich. Have you tried to see whether the effect of the campaign differ for individuals with or without networks, family, friends, abroad? If it wasn't possible that the data collected was ignored. And the second question regarding methodology, to what extent did you manage to avoid response bias? Eventually, the participant asked you in a way they think you're expecting them to respond, which could impact the accuracy of the responses. Right, networks, response bias. Okay, we collected a couple more. Yes? We touched upon that topic now several times when it comes to misinformation of migrants. Did you maybe learn something also about the false information, which is, let's say, distributed by smugglers because the migrants are exposed especially to that. And it's always, you have to look at this dual sort that you have to make migrants aware but also combat the misinformation provided by the smugglers. So did you have any findings when it comes to that topic? Because most of the risks actually, which you've shown and the awareness when it comes to risks have directly to smugglers and what they're telling and saying and that's something that would be interesting. Mm-hmm, yeah. We have two more, then, Josie. Yeah, I'm just interested in your last point about the willingness to talk. And linking that up with one of the earlier remarks that you were calling the community journalists. And is there some way of measuring or talking a little bit more concretely about the willingness of people to talk in this context and what it tells us for the future about how to communicate better? I think that's a good point. So if we go around telling people that, sorry, 1,000 are going to die, 22,000 are not going to believe us. So you've got a cohort here to seem to be perfectly willing to be community journalists, trusted community journalists. So what could your study tell us about how we would recruit them in another context to pass that information more effectively? Ooh, many hands, of course. How did you choose the answer to the question? Oh, you already had one question, so that's good. Sorry, we'll try to include as many as we can. It's just a short question. It's saying that we've seen a difference between intention to migrate irregularly. Did you see a difference in intention to migrate at all and you said the number was high, but I didn't know if there was a difference. Because if people aren't aware of refugee asylum, maybe just their intentions in a great way might be interesting. Okay, five questions. Maybe let's go through those very quickly and do another round. Are we doing one time? Terrible. Yeah. You're going to be amazing. Okay. All right. The question from the webinar on networks, we have that information. We know how many people, how many contexts do potential migrants have abroad? Do they receive remittances from abroad? We know about their household size and the household structure. So there's a lot of exciting research to be done here, but the problem is these projects are always structured in a certain way. You try to get the main results out and then the project is over. So we're now in the process of trying to get more funding to tease out some of those questions. And for example, what is the relevance of networks? And there are many other questions that we were to look into. It's possible with this data, but we haven't had the time and resources to do it yet. To follow up on that, I think that ties in a little bit with what Frank said. And I said this to Jan earlier, and also in a hallway, is that we designed this to measure the impact of the project and to do this research, but we also have this unique data of potential migrants from the car. And I think that can be exploited in different ways. And one way would be to look at who are the people most prone to migration which then allows us to tailor our messages towards that group. Because right now we don't know a lot about that either and then that might mean that we spend a lot of resources on people that might not be even open to this. So the more we know about which campaigns work for which people, the better we can spend our resources. And I think that's another way how this data can be useful and make these things also more cost effective if you use the data in multiple ways. That's the answer for all of the purposes. There was a question from Vivenos on response bias. We have a section, 3.6 study limitations on page 27, where we write a whole page about response bias. It's an issue that every study faces are basically respondents lying to you, right? And what is their incentive to respond to your questions accurately? This is an issue that every study has. However, we have certain safeguards. For example, the interviewers were locals. They were interviewed several times over months. So there was sort of an identification and a relationship with the study team. There was no service or payment associated with it. So there was no conditional participation in an IOM project or anything like that tied to the survey. So these are some of the measures and there are others that we try to apply to reduce that risk. Let's first go through the other questions. On that, sorry, if you could comment on blinding because you can prevent response bias by basically blinding at different levels. So the observer, the participants, application for treatment, whatever. So if you could comment now or in the future. Right, in the future. We had some elements of blinding in there because we, the enumerators did not know who they were going to be with. That was a person in the control group or the treatment group. So every time they contacted a person they didn't know who they were talking to and then because some questions were not relevant for the control group, we had like a skip pattern program into the survey that later you would know. But we tried to do a little bit of that. Also at the invitation stage we had the coin flip implement in the computer that says this is a treatment or a control person. So the enumerators that we recruited were not deciding anything or were not aware fully about who's grown aware, et cetera. So this is not full to the total solution but it goes in that direction. Oh yeah. How to recruit messengers. I think that's a bit beyond the scope of this. Could be interesting, could be another interesting study to kind of look into who are the best messengers. We do use the movie to, well when we come together in the movie we try to do a best off because some messengers were just not as convincing as others so there's definitely a steep kind of differences, right? What we do is some are just amazing storytellers others are not so much so kind of this is a bit beyond but it's super important that this gets continued. We should definitely focus on people that are good at this and train those. I don't know how to best find them but no. We're running another lecture at Regine and give me and there's a sort of association of attorneys that organize themselves, that institutionalize themselves and I think created, and we work with them directly for the data collection actually employing them for data collection and it is an amazing job but I think this institute, this organization increases the confidence because they find like-minded people in a similar past and I think this helps them also to speak out and to be confident to share their stories and the more I am good support this type of association I think that would be a good way to go. False information from smugglers so this is also slightly beyond the scope of this study. There are other service out there in a detailed way show what is the misinformation spread by smugglers but for example here we didn't ask potential migrants directly what have you heard from smugglers so it's stupid for me to answer this but other studies have shown the level of misinformation and what exactly they're misinformed about and how they're getting that information and I'm happy to share those references with you. We did ask I think if they were in contact with smugglers and most of them said no and this is conjecture too this information is no lack of information there's like everything about migration UN, government, social media whatever, bots from Russia I don't know but so there's information everywhere I think if somebody decides to migrate momentarily you might know it's better than me but it's not like there's like a Facebook ad and you call a number and you go it's more like you make the decision to migrate and then you go through the networks and kind of get a referral and people because everybody knows somebody that migrated so you kind of get like personal referrals you know go with that guy or that help you know UN last year so it's more like that but I think the smugglers and this is also conjecture I think they come in after the decision was made when you kind of get into actually doing migration last question of regular migration right we did ask general migration intentions regardless of the regular, regular and we also asked have you made any preparations for a move not necessarily regular we do find an effect of a campaign on kind of producing the percentage of people that have been carrying the move goes down we find a small effect on regular migration but it's not significant and it's small and so yeah it's difficult to interpret those findings generally we think that you know this information campaign doesn't necessarily you know it doesn't do away with your general intentions to want to leave but maybe changes the form in which you want to leave and how lots of migration also regular migration happens in tra-africa so most regular migration is actually between countries in the region and for many people have interviewed regular migration to Europe is not even on the table but doesn't exist as an option you can't just let go and apply or something like that so anybody that wants to go to Europe has to go the regular people that want to go regular migration regular they usually tend to go to other countries in the region more questions or last round and you all yes one more hi uh yeah I have a common question first of all I think it's it's amazing that we're doing this kind of work now on project reviews I think it's amazing we have a donor that's supportive of this kind of process it's rare to find I think sometimes we're woefully blind to how effective these kinds of campaigns are so I really take my hat off to the to you guys who are doing the work and for the process and to our friends from from from Holland now my my question is on the front end so we've done this marble's shell at the back of the project to identify impact and I'm just curious to the front end because there's a it was a slide a little bit earlier to identify you know migrants we take this as a given migrants are trusted voices um but in my experience running campaigns there are many trusted voices within the community and so I'm just wondering if any thoughts have been given or what your thoughts are about doing like an insightful market survey product of the campaign to identify other influencers other other influential voices within the community and to bring them into the tent as well as part of the process so that people who are considering a migration option when they go looking for information or be looking for feedback are actually bracketed by members of their own community by members of their families by the shopkeeper by the school teacher so I'm just curious as to whether or not this is something we ought to be looking at down the road thank you thank you and I just make a comment that there's one slight thing about this campaign which we haven't mentioned which I don't just have a flow on the campaign but I think it made it easier is that everybody's coming back from hell as they've all been to Libya they've all had a terrible time so that's not the case necessarily for return life or some time so the study only focuses on the daily migrants it's not the entire global community some might be successful some might die on the way so this perception might be overestimated by some people I'm just wondering right last one so one more from the webinar this one and then one more from the webinar maybe is there is this one I was just wondering if you had considered or are considering using social media as another additional data source so Twitter, Facebook and monitoring in that side of things is there one more from the webinar from your yeah so this is a question regarding comments what communication tools were used to convince the target audience to watch the film was there a number of tweets Facebook posts, etc are used to inform the target audience to attend to those tweets okay if I can answer that so we invited them on the spot and then we collected their phone numbers we sent them text message reminders we sent them we called them we said there's food actually so when Moa did the first screening didn't go that well there was like five people at the screening and then we had to kind of up our game in terms of getting people yeah add a little bit of incentive for people to show up the incentives were always the same for both groups but it was definitely not easy to get people to take an hour or two hours of data to complete the screens so you invited as IOM? no no the other questions the returnees that were the messengers were their selective group right that obviously kind of drive the agenda and the messaging there that's true and what the approach was to counteract the misinformation by smugglers and lead to a more balanced balanced information balanced story by providing that one side of the story right I would say that you know the dangerous side and that particular perspective but this ties back to what Lenin said in the future I think these campaigns could be adjusted also to reflect broader experiences and not just Libya and the horrors of Libya but also other stories that could be included I mean I think as IOM we want potential migrants to just have accurate balanced information so that they can make informed safe choices and information that they that many have before they leave is a bit biased towards one area and the attempt here was to sort of unbiased towards the other direction but there's more to be done in this area Let me follow you quickly so the yeah it's definitely not a full picture of all migration experiences that was not the goal of this so this the movie is a bit biased I mean we did have returning migrants that went through hell and said they would go again tomorrow and they're like we've up and beaten my friend died and this is my third try and you know in a month I'm going for my number four and so we did not have those messages in the movie in the end the goal was to kind of have people not go on a boat and potentially die on a seat so just to follow up on them On social media yes I'm glad you asked me this because we're launching a new study that is looking into social media there are a lot of high hopes with social media because you know you reach supposedly reach a lot of people very quickly with low at low cost but then we're asking who do you actually reach who are these people that are reached by Facebook campaigns and you know how do they actually engage with the content and what kind of approaches to reach those target groups are more effective than others what kind of content is more effective this is a upcoming evaluation that we work on in the context of this study there was no social media the problem with social media is and maybe Yasper finds a way to do this is that it's hard to do get a control group because the message spreads so like fluidly through the network so even in our control group we cannot say that we're not exposed to Microsoft's messenger clips right but our theory of change was that you need to be at the movie get the 50 minutes get the peer-to-peer in-person messaging to actually internalize all the information but because there's so much information on the social media in the car everywhere there's nobody that has not been exposed to last one on other sources trustworthy sources I think we do have some data on who people can talk with friends family internet et cetera I think super interesting to especially if we want to do something like this cost effectively it would be very useful to once we have a message that we think kind of works to work with community leaders maybe religious institutions mosques or even like famous soccer players you know like kind of get celebrities local celebrities involved to kind of that would be useful I think we try to do here is to show kind of get the first mile done like can this have an impact at all and then down the road you know we need to find the best and most cost-effective channels to transport that message the way we did this is not the most cost-effective one because we had screening events inviting people running all over the place so maybe it's more impactful I think the fear I have is that everybody goes for gold in this game everybody wants to have a celebrity and where's the evidence that that works at all I mean you have evidence that this works right so why why jeopardize this for something that's just a nirvana that you can't do