 Mae'r cyfnod o'r ffordd yng Nghymru, ac mae'r cyfnod o'r cyfnod o'r technologi yn bwysigol, sy'n ei ddweud o'r cyfnod yn cyfnod cyffredinol. Mae'r cyfnod o'r cyfnod o'r cyffredinol yn Unifes i'r Pennsylfaeniaol. Yn ei ddweud, ddweud i'r ffwrdd'r cyfnod, ac mae'n ddweud i'r cyffredinol i'r cyfnod o'r cyfnod o'r cyffredinol. Mae'n rhoi i'n rhoi i ddech chi'n dweud i ddech chi gyfnod y next 30 ac 30, a ddim ddim i ddweud i'r cyffredinol o'r cyffredinol sy'n eitio rydw i'r cyffredinol ym mordau ond y cyffredinol, mewn ddillidatau o'r cyffredinol sy'n ei ddech chi'n amguedd i'r cyffredinol, ac mae'n ddweud i'r cyffredinol yn ddech chi'n inifes i ddweud i'r cyffredinol. The way we talk about civic tech. It's better than just looking at communications with citizens, with politicians. I know many of you study areasGONeed,âwa cei, I know that many of you work in areas other than going to countries. I hope that some of the lessons I can share with you will be relevant to the work of many people here. I'r Inspir yw werth ystwyff acted-syllfa'n blyny bod cymdr feddwl traf Reynolds cyntref, i fengyfrind grandes ym ddyivelyunt embuf functionalau i'r yr omlynyddoedd O holord Ysdorydd dyf tema yw'n sefydlu cydyliau mod i'r bobws gyda dda highilus cells. Bydd yn credu y problem yn gyfr tonnes melodasu vermiliaid, sy'n oesach yn y passtwch gyta parhau finan Mythicalian, O fe, o ddellolaethau i ddeudio'n gweithio, o'i ddweud o'r ddweud felly oherwydd ym Llyfr Saeddau, so oedd yn ddeudio'n dweud o ddweud o'r ddweud o ddweud, o faenny o'r ddweud o'r ddweud o'r cyfnogwadau cysybodaeth yn ni wedi'r rhan oed ddweud i ddweud o ddweud o'r cyfnogwadau cyfnogwadau cysybodaeth. So when we think about political communication, one way of thinking about political communication is thinking about the will and ability of citizens to communicate the interests, the preference, the need to the elected officials. But we know that in many, many low-income countries and maybe also in high-income countries may be many low-income countries there's barriers to articulate people's interests and preferences A ydych chi'n bwysig arbennig oherwydd bod ydych chi'n adeiladau a gweithio'r hollb. Yn cyfnod gyda'r hollb, mae wedi'i dweud o'r bwysig a dweud o'r bwysig. Ieithio arbennig oherwydd y bwysig, a yna'r bwysig arbennig oherwydd mae hi wych yn cysgirio yng Nghymru, a'r bwysig arbennig oherwydd yn cyfrifio. Ilych yn ymloedd i cael syrdu ddid lumbera iinybomas. Can ystafol gwyno dydyn ni ryngwst yngiellu ateddiant sgwrs wedi bod yna'i bywodol awsig. Mae sgwr i'n digyn. E ignol i, yn ech empez еndriadd ar y byddog contractei ar y ddweud yr crevhau課 procedurell ei ddaligol dar侵inkol hefyd i'r daf, а PMF, ...yny'r rhaid i'r iawn, yn iddo i'r unigion, ond yn ei gyfnod... ...yna'r cyffrith yn ei gael gynedd oedd yn eu hunain. Yr hyn argynniadol yn ysgrifennu yma. Pwy ydy'r hynny'n gweithio. Mae'n dynnu'r ffordd mewn cyd-i-dwyg... ...fyrdd yma'r ddweud cyfeithio'n ysgrifennu... ...ynd yn digwyddio'r ddweudio'n gweithio'n ddweudio... ...o ddweudio'n ddweudio'n ddweudio'n ddweudio'n ddweudio... 20 miles, 30 miles, 40 miles out of the district headquarters. Now the district might know, like in general, there's problems of service delivery, but they don't necessarily know specifically when a teacher doesn't show up to work, when a nurse doesn't show up to work, when a doctor asks for payment for a treatment that should be subsidized, or a nurse is asking for payment for medicine that should be delivered for free, okay? And these are not a small problem. I mean, if you will do a random audit in schools and clinics in East Africa, 30% of teachers are not present in the school. 25% of nurses are not present in the clinics, okay? The problem is that because of re-communication channels, the government doesn't really know which teacher didn't show up. They don't know which clinic is closed when it's supposed to be open. The problem also is that the citizens know that the government doesn't know about the teacher that doesn't show up, which means that because the citizens kind of know that the government don't know, they have very low expectations. Because the citizens have very low expectations, and the government knows that the citizens know that they don't know, the government doesn't have a strong incentive to go and seek for this information. So we're in a low equilibrium where the government doesn't know what's the problem, the citizens know that they don't know, they know that the citizens don't know, and we all stay in this low equilibrium, okay? Another problem which people don't fully appreciate of the problem of re-communication channel is parties that are not really programmatic. So parties don't offer programs, so they're not competing on a marketplace of ideas because they don't really know what citizens want. If they don't know what citizens want and what are their preferences, so they campaign on very general issues like, oh, we'll all fight corruption, we'll all bring development, okay? And when parties are not offering voters any differentiation, what voters tend to do is to vote according to tribal lines, and that adds a whole host of other problems, okay? So these are some of the problems of re-communication channels, and then there's another big problem which is political communication or political access is not distributed evenly. So we know women and poor, the less educated have less opportunities to raise their voice than the men and the better educated people. Okay, so these are the problems, and now we have this big revolution of in the developing world where we completely jump, we do this leapfrog from landlines that no one has, we have now a world in which also computers people don't really have access and we know only 10, 15% of Africans, for example, ever use the computer, but 85% of the population has access to mobile phones. So this is really exciting and the question becomes, can we use the fact that so many people are now connected with the mobile technology to address some of these challenges, okay? So can we use mobile technology to increase political accountability, to improve, for example, service delivery? Can we use them to flatten political access to marginalised populations? Now the second point is not a minor one. When I started trying to sell governments and donors and civil society groups in Africa on the idea of mobile civil tech, one of the pushbacks that I got is like, hey, you're just going to be privileging, they're already privileged, because if you're going to allow communication that is based on say mobile technology, you're going to just privilege the men and the better educated that are more savvy in using technology. So a big question into the research agenda is whether mobile civil tech can flatten access and be an equaliser, okay? So what I'm going to do after we kind of set up the stage of what the problem that we're trying to face, I'm going to describe in the next few minutes based on research that we've been conducting in the last four, five years, what we think we know and what we know that we don't know, okay? And use that to kind of chart going into the future, what do we think, what I think should be some of the work that people that care about this space should be focusing on. I just should mention before we go into what we know and what we don't know that at least in the context of the countries that I'm working at, we're still in the G2 world, okay? We're not in the G4. So the fact that we have 85% of the people 90% in some countries have access to mobile technology that doesn't mean that they can go online. It doesn't mean that they can contact their representative through Twitter and Facebook. We're still in a G2 world, okay? And the projection is that at least in the next five, 10 years we're still going to be to the most part in a G2 world outside the main cities. So a lot of the solution that we're trying to do are with the G2 world. So maybe a lot of the things that I'll be discussing will be less relevant once we all get connected, but we don't have the patience to wait 10 years till everyone is connected, so bear with me. Okay. So the first lesson that we learned, okay? And this was a big if, this was a big question before we started this like engaging this whole like research agenda. So we know, I think, or we think we know that there's an underlying demand to contact one representative using mobile civil tech platforms, okay? So this is based on several studies I'm going to give just two examples. So one example is we did a survey with a national representative sample in Uganda. We collected a lot of information about demographics. We also collected a lot of information about the engagement, the political engagement, using traditional forms of engagement, like participating in rallies, contacting the representative through mail, voting, okay? And at the end of the survey, the survey was with close to 4,000 people. We gave them a flyer that says, hey, if you want to contact your MP, here's a number you can text. We use the very basic platform. I think we use the frontline SMS. And we said we'll deliver the messages, okay? And we got 5% that send the text message. So 5% is, you know, sometimes I presented, we would say, yeah, that's not a lot. Actually, I mean, I think there's a lot of us here that would be happy 5% of the population would be using our platforms. But that's also equivalent to, you know, primary, the show of people that go and vote in primaries in elections in the United States. So that's not a small number, okay? Another project was a very recent project in Northern Uganda. We created a platform that enabled citizens to complain when, this is mostly about service delivery deficiencies. When the teachers doesn't show up, when the health clinic is closed, we implemented it as an experiment in 100 villages, there's 100 villages control, and we're following these villages in the next year or two. And in the first six months of implementing this project, we're getting about, we're averaging about 60 messages that are coming in from each of the 100 villages in six months. So that's about 10 messages per village. And these villages are, the population is about 100 and 150 adult people in each village. So that's a high uptake, and that's on a monthly basis, okay? So one good lesson that we've learned, there's an underlying demand to use these technologies. But we've also learned another lesson that, you know, and I think many of you in the room have learned this lesson in the past. If you build them, it doesn't necessarily mean that they will come, okay? So we have also experienced with platforms that we implemented that didn't catch. So for example, we implemented a platform together with the Ugandan Parliament, and there was another partner, was the National Democratic Institute, International NGO based in DC. The platform was very sophisticated platform that enabled citizens to send a text message, a taxi and MPs can open it on a dashboard that allows them to take all the messages that come in and look at queries, how many messages about health, how much education, how many in each month. Super sophisticated. In order to market the platform, the parliament heard 30 second ads in 19 languages over a year twice daily, okay? And they got less than 2,000 messages over a population that we estimate that heard the radio support of about 10 million people. Then there's another study that I was involved in Nigeria where the idea was to get people to report corruption. This was marketed through blast text messages that arrived in about two weeks period, sending out 700,000 text messages to people saying if you want to report a corruption incident, it's usually kind of a bribe when somebody asks you to pay for service that the government should be providing at no cost. And the platform generated 800 messages over the two week period. And again, this is less than 0.1% uptake, okay? So if you build them, it won't doesn't mean that they will come and a lot of the research that we're doing in this space is trying to understand why some platform catch and some platforms don't catch. So I will talk about this also in the next few minutes. Another important lesson that we've learned and this is a very important lesson that we've learned is ICT platforms, platforms that allow citizens to contact their representative in government have a real potential of flattening access, okay? So they could be reducing inequality in access to politicians, okay? So I'll give example of what do I mean by flattening access, okay? So you remember when I told you that we did this national representative survey, we asked people about the demographics, this allowed us to construct a measure of how marginalised people are. So, you know, women are more marginalised, less educated, more marginalised, poor people are more marginalised, people that live in distant areas. So we created an index of marginalisation, but then we asked them also about the political engagement. So we took all the forms of engagement and created the index of engagement, okay? And what you can see here on the left is the more marginalised you are, the less engaged, okay? So that's the negative trend, okay? These are traditional forms of participation. On the right, what we see is the same marginalisation as a function of the share of people that send a text message to the representative in government, okay? And what we see is that this is straight, okay? This is straight line, meaning more marginalised people are not less likely to send a text message, okay? So one way of thinking about, one way of thinking about whether this can flatten access is to think about the share of women and poor among civic tech users as compared to traditional forms of engagement. So the right way to think about it is not to say 40% of SMS senders are women. So we still are not in gender parity, but also say that when a politician comes to a village, only 20% of the people that get up and speak are women, okay? So if only 20% of the people that get up and speak are women, in the community meeting, but 40% of those that send messages are women, then we can say that we added another channel that flattens access, okay? Even if we don't get parity, okay? Now, we think that the reason that this is the case is because women and poor and less educated value this form of participation because a lot of politics in low income countries is about traveling to the district, traveling to the capital. It's very personalized, but marginalized population value the ability to stay in the village. They also value the fact that a lot of these messaging are impersonal in nature, okay? Now, we thought that this is about value, so we wanted to test that. So we ran another experiment, and what we did in the experiment, we randomized the cost of sending the text message. So for those of you who don't work in low income countries, people to the most part don't have data plans. Every time they need to make a call or send a text message, they will buy a scratch card and they will upload money. So sending text messages is costly, we randomized the cost, okay? And the idea was to see what happens when we vary the cost of contacting one representative. So what we found was very interesting. Two interesting findings. The first one is that when you increase the cost, we have less people that participate. So that's kind of normal. It's like any good and any service more expensive reduces the cost. But in most services and goods in the world, when you increase the price, the first people that drop will be the marginalized people, okay, because they're more price sensitive. What we found in our experiments is that that was not the case, that the marginalized people were not dropping first. Okay? They were not, so everyone was like messaging less when the cost was high, but it's not that the marginalized people were dropping at a higher rate, suggesting that they place a high value on this form of participation. So that's another good news, okay? That there's a potential of using mobile civil tech to flatten access. But after every good news there's some bad news. And the fact that marginalized populations value this form of connection doesn't necessarily mean that every time we will put in a platform, it will be an equalizer, okay? So participation of marginalized groups is far from guaranteed. So I want to go back to the experiment that we did with the Parliament of Uganda where they marketed the service through year-long radio spots, 30 seconds, twice daily, over a year, okay? So I wanted to focus on this graph, which is fascinating, because we're talking about the same population, just different marketing. On the right we're gonna look at the shelf women that send SMS when we gave them a flyer, we'll contact them directly, we gave them a flyer and said, hey, you can contact your members of parliament. We're talking about the same parliament, the same country, the same population. When it was a direct invitation, almost, I think you can see it here, almost 50% of senders were women. When the marketing strategy was radio spots, the shelf women was 18%. Okay? That's kind of a big deal, okay? So we were trying to understand why is that the case, okay? Why is the marketing channel so important? So it could be that women don't have access to radio, okay? That could be the case. So we tested that, so we did another big survey nationally representative to try and understand who had access to radio, who heard the messages, and what we found is that men and women were equally likely to hear messages on the radio spots. The big difference was that whereas men, when they hear a radio spot that says, hey, there's a new service, now you can contact your MP. They're like, oh, this is relevant, I'm going to contact my MP. Women heard the same message and didn't think that this is relevant for them. Oh, there's a new service. That's for the guys, they should do that, okay? So we're learning that the type of marketing is important and specifically the importance of personalizing the invitation to participate in these contexts is super important. So we thought that this is the case. We thought that personalizing the request is important based on this information, but we wanted to study the deep, okay? So Becca was talking before about evidence-based, so we said, okay, it's a hunch, let's study the deep. So we ran another experiment, okay? And what we did, we sent text messages to people in order to encourage them to contact the MPs, okay? But we verified the contact of the text message that encouraged them. So in one of the treatments, we told people, hey, there's a new service, you should contact your MP. And in another treatment, we used their name, okay? So we said, hey Matt, hey John, hey Becca, you can contact now your MP. The only difference was adding the name. So as you can imagine, names work. So everyone in the treatment where we use name, everyone sent more messages, but what's fascinating is that with women, the treatment effect was doubled the men, okay? So for women, the personal invitation had a much larger effect than men. So for us this suggests that we did another thing which is also interesting. So the one with the names goes into what we call like subjective or subjective efficacy. You'll believe that you can actually make a change, okay? We did another experiment in which we, in the message we said, your MP wants to hear from you, okay? So the idea is to give them a, you know, can increase their confidence that on the other side there's someone that is interested. And again, when we told people that the MPs want to hear from them, huge up, huge uptake, about 50% increase in the messaging rate, okay? But again for women, significantly higher for men, okay? So a lot of lessons about the fact that by itself, even though marginalised populations value this form of communication a lot, by itself it doesn't guarantee the participation and there's a lot of things that we need to think of like the marketing and how personalised the invitation to participators. Okay, so I spoke a lot about the side of the users. Let's talk a bit about the government or the side of the recipient because we're also learning that the recipient identity is super crucial, okay? So two ways of thinking about how the message recipient is important. One is to think about level of government, one to think about even within a level of government whether we're talking about, for example, politicians or bureaucrats. Our studies, I'm going to go over this a bit fast, our studies show that people are much more likely to contact local governments in the national parliament because of what the government actually does. So it's much easier to contact about what a world that is broken, health, you know, the quality of teachers, health clinics that are always closed. It's much more difficult for citizens to message, to contact the MPs in parliament because in order to do that you need to know what's on the docket of parliament, what's the debate, what sites people are taking, and what are they going to vote on and people are not good at that. For that you actually need civil society, okay? So the only times I contact my representative in congress is when I get an email or some Facebook post of some group that says, hey, you know, parliament is going to vote, congress is going to vote on defunding plan parenthood, make your voice heard, okay? So somebody needs to mobilize me. This is not the case. When this is not the case in the countries that I study, we're seeing very low participation rates in national parliament, but it's also a lesson for us of the importance of bringing in civil society. The second one is a fascinating finding. We run the same platform, we implemented exactly the same platform in districts in Uganda. We're talking about the same platform, the same country, the same level of government, the same government responsibilities. In some places we gave the platform tablets, the access to the platform to bureaucrats, in some districts we gave it to councillors, to politicians, okay? And we wanted to see what's happening. And what we found is significantly higher uptake, significantly greater involvement when we gave it to the bureaucrats, okay? So when we gave it to the bureaucrats, people anticipate that the bureaucrats will respond. They anticipate that they have the knowledge, the know-how, the technical ability to address some of the problems, and they were sending messages in high rates. When we send it when the platform was given to local government, to politicians, people didn't think that they have the power, didn't think that they will have maybe the interest, and they weren't sending the messages. And the interesting thing is that we also do a lot of call-back surveys, so part of what we do in the research will call back people that are users. And in the ones with the local government we call back and we said, hey, we saw you sent one message, but then you never sent, and people would answer, the answer would get a lot is like, I sent once, never heard back, we'll never send again. Which leads me to the next insight that we've learned through the research, which is civic mobile, civic tech can create a platform of communication, but by itself it doesn't make a non-responsive government responsive. It's just a platform, okay? And now I can communicate, but that doesn't necessarily mean that on the other side there's someone that is responsive to the message, that will do anything with the message. And what we've learned that if politicians don't want this platform, they can derail the process by simply not responding. So you'll send once and you won't hear back, you'll send maybe twice, you won't hear back and that's it. You'll never send again, okay? So we're learning that by itself, creating a platform that allows citizens voice is not a panacea for the problem if you have a non-responsive government. What we're discovering that it's important to think not only about knowledge and information that is passed, but also about government incentives, okay? There has to be a political economy analysis behind what we're trying to do when we do mobile signal tech, okay? So we need to think about how to change incentives. And changing incentive is not that easy, okay? So here's one way of changing incentives that we came up in one of our studies. Not that easy, highly effective. And this is the idea that common knowledge is key. So do you remember before that we were talking about the government that doesn't know what's going on and the citizens know that it doesn't know and it knows that the citizens don't know? So let's just flip that, okay? Now, let's create a platform that enables us to push information to the government. So now the government knows. But we want also citizens to know that they have this information, okay? So the citizens know that the government know. But we want the government to know that the citizens know that it has this information, okay? So we want the citizens, so we want the government to have this information. We want citizens to know that it has this information. We want the government to know that citizens know that it now has this information, okay? So we want to create this common knowledge because if the government knows that everyone knows that it has this information, if it doesn't operate on that basis, you know, there might be implications. People, now there's more expectation. People might punish the people that have the information and are not operating on basis of that information, okay? So we want them to know that they know that we know, okay? That's kind of the logic of common knowledge. The truth is it's not very easy to do in a G2 world. It's easier to do in a world of internet where you can post a website where all the messages that come in, the response rate and everyone can see, everyone can log in. Much easier to do, much more difficult to do in a G2 world, okay? One of the things that we've discovered that was very effective is to hold community dialogues where we invite the government officials to the communities to discuss. Here's all the messages that came in in the last, like, three months. We do it on a quarterly basis. Here's the messages that came in the last few months. Here's what we did. This we solved, this we couldn't solve because this is the national government and not us. So there's a lot of dialogue with what the government can and can do and we've seen that every time we're holding these community dialogues, the rate just jumps after that because people feel that the government is much more accountable to them because they know that they have the information and the government is much more sensitive to the fact that now citizens have this information. The problem is, is that it's a bit costly. The plus side is that it brings the virtual world and the real world together, okay? If we think that by creating, you know, platforms, you know, we can somehow eliminate the real world because everything will operate on like, you know, some servers, we won't, okay? Especially for those studying of us working in developing countries. At some point, the virtual world and the real world need to kind of connect and that's when we get the best outcomes. Two other lessons that we learned before I moved to the things that we don't know yet. One is that the center anonymity confidentiality is key. When we now market a new platform, so we know we don't do it on the radio and we need to send people out to villages to hold community meetings, we've learned that. We have a painful lesson, but we've learned that. What we also learned is that when we do a demo, we actually need to show people that when you send a text message and it opens at the site of the government, we show them that it opens as a case, in a case tracking system, and it can't trace back to the number. They want to see, they don't believe us, they want to see, they always think that they can actually trace it back to the number. So showing them, like showing them that you can, that it really only opens as a case, in a case tracking system, is just absolutely fundamental for getting people the confidence to use the platform. And the reason this is so important is because people are not always aware of the fact that there's a social cost of complaining. People are afraid that if I will complain about the teacher that is abusive, she will not recommend that my child advances to secondary education to high school. There's a fear that if I will complain about the nurse that was abusive, they won't treat my child the next time around. So it's not just about the monetary cost of complaining, there's also a social cost of complaining, anonymity is key. But what we've learned is that if you really, like make everyone aware that this is an anonymous and impersonal system, there's also another side benefit, sorry, which is that centres will start using the platform in ways that are very, very different from the communication that they have using face-to-face interaction. So speak to politicians in a lot of developing countries and they say that they avoid meeting their constituents because every time they meet their constituents, the things that they ask for are personal favours. They'll ask for school fees for their kids, to pay the medical bills, you have your moment with your politician, that's what you ask for. But because this is impersonal communication, you're not gonna ask for medical bills when there's anonymity. So people are asking things like better schools, better healthcare, better roads, which is really interesting because by changing the mode of communication, we're also changing the content of communication, okay? In a very profound way, okay? So that's another lesson that we've learned from the work that we've been doing with mobile security. Okay, so these are some of the things that we've learned. I'm gonna go briefly on some of the things we don't know before I'm gonna wrap things up. And I hope I'll still have time also to get some questions. So here's some things we don't know. One thing we don't know is the nature of the collective action problems. Let's, sorry for being a bit technical here. So here's what I mean when I talk about the nature of the collective action problem. So when I hear that other people are using the system, what does it do to me? Does that increase the likelihood that I use the system or will that reduce the likelihood that I will use the system, okay? So we actually don't have a good sense of what's the case. So we can think of two possibilities. One possibility is a possibility of complementarities. If I know that people are using the system, it increases the likelihood that I will use the system because they can't ignore us, okay? If only one person complaining about the bridge of this book and they will ignore us, but if all of us will complain, they'll think that this is really serious and they'll take care of the bridge. Maybe if people think about it as complementarities, but people might think about it as substitute. People might say, oh, it only takes one person to tell them that the water well pump is broken. So why do I need to send a message? If I think that Mark will send a message, why should I? So then there's a collective action problem because it takes time, it takes money, it takes what I need to think. I need to be, I just want to go back home from a long day at work. So it might be that thinking that other people are sending reduces my likelihood of sending and we don't fully understand that because if we'll understand that better, we can manipulate people's knowledge of how it's used by other people and these are some of the experiment that I've been doing now is trying to manipulate information that people have about usage of other people to understand better the nature of the collective action problem. Another thing we don't fully understand is the importance of coordination in the context that we are dealing with, okay? So how much is it important that there's gonna be people in communities that say, hey, if there's a problem, come talk to me, we should all like coordinate and send messages versus just people, just the only reason why we have some villages that we have a lot of messages because you have super users that care a lot about some issues and sending tens of messages. So you remember that I said that we're doing this experiment in Northern Uganda where 100 villages are treated, 100 villages are controlled, we're getting what average 60 messages in the last six months but we have villages with 200 messages and we have villages with almost nothing, okay? So we have a lot of variation and understanding this variation to the extent to which this is about coordination or just villages are lucky to have super users is another avenue of research. One way of, we're gonna address it, we still don't have a good answer but one way of addressing it, it's kind of expensive but we're doing mapping of the social network of the villages, okay? We're doing complete mapping and the idea to see whether villages that are sending a lot of messages have more dense network than villages that are not sending that many messages. Another thing we don't fully understand is what does it mean to have high uptake or low uptake? We're doing these things and people say, hey, only 5%, that's really low. And sometimes we'll say, well, how many messages we need to get about the teacher that is abusive for people to get the message, okay? Do we need thousands of messages or it's enough that four or five people consistently complain about this teacher? If all the information we need is about the broken well, the water well pump is broken, like how many messages do we need? Maybe two is enough, okay? So we don't have a good grasp for, and I think this is related to a lot of the work that you're doing. What is the level of involvement that we need in order to say this is actually making a difference, okay? We don't have a good sense of that. We don't have a good way of mobilizing civil society. A lot of the work that has been done in this space until now is citizens' engagement directly with government. And this might be okay for some type of engagement, like water wells and health, but not for others, like parliament. For that, I think we need civil society, but we don't know yet how to do it well. And the problem is that the technology that works well in our world is not so transferable to the context of low-income countries because people don't have access to the internet and they don't have zip codes. So when I sign a petition about the funding plan parenthood, all I need to do is I put my zip code and automatically it's sent to my representative in Congress. But if you're in a world without a zip code, like how do you do it? Okay, if you're in a world in which we don't have a smartphone, it's really difficult to know what's your location, okay? In a world without zip codes, okay? So some challenges here. And also, as I said before, we understand that it's important to incentivise government, but we still don't have a good handle. So we think about common knowledge, but maybe there's other ways we still don't know. Okay, so I'm on my last slide, just thinking about moving forward. Given the things we know, given the things we don't know, here's some things about people that are working in the space, whether you are practitioners, whether you are government, whether you are, you know, tech people and programmers. Some things to think about. The first thing is this idea that I've mentioned several times in this talk, that the importance of bridging the G2 and G4 world, okay? Most of us are working in a world that we take for granted the internet, take for granted that we have Facebook and Twitter, but that's not the reality of many of the countries that I work in. And thinking about applications, thinking about platforms in which allow us to bridge the divide is of utmost importance, unless we all say, you know, let's just wait 10, 15 years and the problem will be solved by itself. But to the extent that we don't want to wait 10, 15 years, we need to think very carefully about bridging this divide. Another thing that in moving forward in this space is this is a space that relies heavily on a model of what we can say a crowdsourced model, okay? People, we assume people have phones and people will use it and if they will use it, that's great. If not, that's too bad, okay? But there's other models, okay? So we can contrast the model of crowdsourcing. We can also contrast to the model of seeding people. So I can go to villages. I can go to communities. I can give mobile phones to people and say, hey, I'm going to give you a phone, I'm going to give you some air time and I'm just going to call you once a month with a set of questions and if you answer, I'm going to send you a thousand shilling over that. Not enough experimentation with that or just saying if you see something, tell us. So there's an experiment that was done in the Congo where communities got phones and they were asked, so it was seeding. They gave it to specific people and said, every time you see rebel movement, send us a code and there were codes. One is rebels. Two, there was rape in the village. Three, there was killing in the village and it was all based on seed sourcing rather than crowd sourcing. So there was some evidence that actually the information that they were getting from that was more accurate than data that people have been working on like media reports and stuff like that. But we don't have a good sense of when it works, how it works and this is something we need to experiment more going into the future. I think we need the better analytics of the footprints of people do with their phones in order to understand better what people want and I think there's a lot of people here that and probably some of the presentation that we'll hear will talk about how we can use the footprints of people, what they post, what they tweet, what they call, when they call, who they call in order to understand better what are the interests and the incentives and preferences and priorities of people. So that's another exciting avenue. And the last point I'll say is all of the work that I've been describing now is an outcome of collaboration between civil society groups, government, donors, but together with researchers at every stage of the process we are involved and engaged and experimenting in order to learn about what works and what doesn't work. So research is really key for knowing is what we're doing impactful, is what we're doing make a difference in the world and one of the goals of me coming here and engaging with you is also trying to get people to realise more the benefits of collaborating with researchers in order to have an evident base field rather than wish will be. Okay, I'll stop you. We've got time for some questions as well and Gemma's going to pass a microphone round if you have questions as well. Hello. I'm Kersir, University of Amsterdam. Thanks a lot that are really interesting insights. What I was wondering, I see a bit like a gap between the new and the technology level and the non-tech level. And I was wondering if you have looked into more traditional and also in terms of trusted other forms of platform creation and community creation in low income communities as you call them. So looking into traditional forms in terms of community media, community radio, in environments where these are the platforms that have been there for a long time to create or intra, but also cross community connectivity, collaboration and mobilisation. I mean, that's a great question and that's not a space that I've been working on. I mean, the space that I've been working was mostly creating platforms that connect, not necessarily citizens, but connect citizens to government officials. But I think it's an exciting area of research. So one thing that I've been doing as we'll be doing is some discussions that we have with mobile phone providers on getting access to metadata. Super sensitive, there's a lot of privacy concerns so we need to kind of work them out, but we started analysing metadata in some countries, so one country is Senegal, that we have good metadata, there's good metadata in Yemen that has been available to researchers. And what we're trying to do here is also understand not only, so what we're trying to understand, intra and inter-village communication. A lot of this work is about around conflict, so how conflict affects inter-village communication. Super exciting, super interesting, takes a lot of data analytics, still not difficult to know how this will be used to think about civic tech, the way we kind of think about it here in terms of raising voice. But definitely there's analytics that we can do, especially after violence. So one of the things for example we learned in Senegal is that the way communities have been contacting each other after violence, there's a real disruption. So we see a lot more communication between areas that share the same ethnicity than areas that don't share ethnicity after sparks of violence, very interesting. Hi, it's Patrick Levy here from Newcastle. Your last point creating collaborations between researchers and the various stakeholders and organisations involved. I'm very interested in that, so in the sense that there's actually quite a sort of landscape of different types of researchers as well. So often as a political scientist for example, a political scientist might be happy to just sort of observe what goes on, theorise about it. But obviously if you're thinking about sort of computer scientists, designers, they're a bit more interventionist in their outlook and I'm wondering about the balance here. Where should innovation come from? Because obviously what researchers want are not necessarily what the society wants or the participants want. I wonder if you sort of experience those sort of tensions Yes, so it depends, a lot of them. I think so the key, so obviously the researchers, one of the big benefits is they come for free, like the only thing, so I don't work as a consultant, I only work for Bona and a lot of the people that are in my space, that's how we work. Our payment is data. So we give our expertise for free, we want access to data, but the things that we care about because we want to publish papers are not necessarily the things that the problems of the communities that we engage in. So the key for us for successful collaboration is twofold. One is to try and have teams of researchers rather than our researcher where people bring different insights. But the second thing is involving us in very early stages of the discussions where a lot of this is about identifying the problem, trying to understand what is the thing that your partner wants to get from it. I mean the discussions with the Ugandan Parliament were taking over like two years, and this was high level with discussions with the speaker of the house, with the political parties. We didn't just come and say, hey let's implement something out of thin air. So a lot of it is about making sure that each side understands what are the limitations and interests of the other side and trying to be as sensitive as possible. The key thing is coming early. So when somebody comes to me and says, hey we've done this and this and this and the results usually it's too late. To make some, to have like re-learning. Re-learning comes when you can embed within the design from the get go experimentations that will allow you to address some questions of interest. For example, randomising the cost. For example, randomising the marketing strategy. For example, randomising aspects of the interface. If we come at very late stage, our ability to influence the learning is much more limited than if we come at the early stage. But the short answer is meet at the very early stages where everyone is very transparent about what they're trying to get from the get go. We've got time for one more question. Thanks for your presentation. I had a question. You hadn't mentioned the political context and especially in Uganda and maybe East Africa very much. I've been there and from my understanding people are quite afraid of the government in many ways. So how do you sort of adapt the ICT or how do you approach governments in those very different political contexts from Europe and the West to say? So that's a great question. At least in most of the work that we're doing at the local level that is focused on service delivery there's less tensions and the stakes are lower than the presidential election. So we've kind of veered away from the real high stakes because there's some sensitivities there and in focusing on things that also the government I mean what we encountered was that there's a lot of good-willing people in the level of the local governments that want to improve the services but they're really constrained about the knowledge. So there's real constraints so they were very supportive of our effort. So on one hand one way of avoiding it is understanding what are the spaces that you can operate and what are the spaces that the office of the president was very clear on what we can operate and what we can operate but the second thing is going back to some of the points that I raised before about the importance of anonymity and confidentiality. Not only the data when it comes in it opens as a case and it can't be traced but also the back end doesn't even sit in the country. The back end of the data is not sitting in Uganda is sitting in the United States. So that's very important and everything is like decoded and there's a lot of we're very aware of privacy issues. Okay run out of time I'm afraid sorry. Thank you so much. Guy again. Grab him in the coffee break.