 Well, hello everyone and good afternoon or good evening or good morning depending on where you're joining us from today. Welcome to Engineering for Change or E4C for short. Today we're very pleased to bring you a special segment in E4C's 2016 Webinar Series focusing on mobile data collection. My name is Yana Aranda and I'm the Director of Program at E4C and I'll be moderating today's webinar. I'd like to take a moment now to tell you about the webinar that you are joining today and as an un-mobile data collection series. The widespread availability of mobile communications offers international development researchers, practitioners and students new tools and techniques for collecting field data and determining successive projects. So we've partnered with the Development Impact Lab at UC Berkeley or DIL for a series of six webinars to introduce a sample of 30 software tools and demonstrate how to implement each tool in practice. For a recorded introduction to the series, we invite you to visit the E4C homepage. Today's webinar is the fourth in the series featuring VotoMobile, introduced by Melissa Persaud who is the Regional Director of Programs for the organization. Our next webinar will be with premise data on April 7th at 12 p.m. Eastern Standard Time. If you would like to make a recognition for a specific platform, future topics and speakers, we invite you to contact the series team via the email addresses visible on this slide. Now before we get started, I'd like to tell you a bit about E4C and who we are. E4C is a knowledge exchange platform in the global community of over one million engineers, designers, development practitioners and social scientists, leveraging technology to solve quality of life challenges faced by underserved communities worldwide. We invite you to join E4C by becoming a member. E4C members enjoy access to relevant and current news, professional development resources including this webinar series and jobs and fellowships, and a growing database of hundreds of poverty alleviating products in our Solutions Library. E4C delivers a unique user experience based on user site behavior and engagement. Essentially, the more you interact with our site, the better we are able to serve you resources that meet your needs and interests. We invite you to join our passionate global community and contribute to making people's lives better across the world. Please check out our website to learn more and sign up. We're excited to collaborate with DIL on this and future webinars. DIL is an international consortium of universities, research institutes, NGOs and industry partners addressing global poverty through advances in science and engineering. DIL has headquartered at the University of California, Berkeley and was launched in 2012 with support from the US Agency for International Development through the US Global Development Lab. This leverages the innovative capacity of world-class universities to design development solutions which couple new technologies with novel economic and behavioral interventions. DIL calls this approach development engineering. Now the webinar you're participating in today is part of E4C's professional development offerings. Our webinar series is a real-time and on-demand resource showcasing the best practices and thinking of development practitioners. Information on upcoming installments in the series as well as archive videos of past presentations can be found on the E4C webinars page. The address is listed here also on our YouTube channel. If you're following us on Twitter today, I'd also like to invite you to join the conversation with our dedicated hashtag, hashtag E4C webinars. Now a few housekeeping items before we get started. Let's see where everyone is from today. In the chat window which is located at the middle part of your screen over here, please type in your location and I'll get us started here. All right, here I am from New York, New York. Yes, Washington, D.C., thank you. Naturally, we have some folks from California. In the chat window, you should be sharing any comments you have, anything you want to share with the rest of the participants. If the chat window is not open on your screen, you can access it by clicking the icon at the top right corner. Any technical questions or administrative problems should go in the chat window as well. For asking questions to the presenter, please use the Q&A window which is located immediately below the chat. If you don't see this, you can click the icon again on the top right-hand side. If you are listening to the audio broadcast and you encounter any troubles, try hitting stop and then start. You may also want to try out the opening of X in a different browser. Following the webinar to request a certificate of completion for one professional development hour, please follow the instructions on the top of the E4C professional development page. So with that, I'd like to take a moment now to introduce Melissa Versace, who is the Regional Director of Programs from VotoMobile, Ghana and U.S.-based mobile engagement social enterprise. Melissa builds and maintains partnerships with impact-oriented organizations in order to improve their choices to more voices around the globe. Personally, Melissa has a passion for global design, for program design apologies and implementation, mobile for development and financial inclusion. She has a master's degree in development practice from the School of International Public Affairs at Columbia University here in New York City, and a BA from Lafayette College. So with that, I turn it over to Melissa to tell us a little bit more about VotoMobile. Thank you. Thank you for that warm introduction. Once again, my name is Melissa Versace and I'm Regional Director of Programs with VotoMobile, and I am based in Washington, D.C. I'll spend today talking a bit about Voto, what we do, how we do it, dive deeper into one case and then hopefully leave plenty of room for questions and hopefully give you enough fodder for a fruitful discussion. And I thank you for coming on today to the webinar. I know webinars can be challenging in some of the locations that we work in. So to begin, VotoMobile is a social enterprise that really focuses on two-way communication. So not only are we communicating out to individuals, pushing information out, but also collecting information back. And for me, that's the most exciting part, that feedback, communication, as we call it today, the hot topic in most development. So as an organization, we're working with development practitioners, we're working with companies, individuals, governments to connect the three billion disconnected people out there in the world who are disconnected from services that everyone has a right to be connected to. So what's the challenge? Why aren't people connected? We have challenges of literacy and language, infrastructure and distance. So you think about how we're connecting to the rural populations, women traditionally underheard and marginalized groups. They're facing unique challenges in communicating with those who are trying to communicate with them or provide services to them. And those challenges can also come with a cost, both financially and time. So we were founded out of the need for that communication and then pairing that need with this new technology, the mobile phone that is almost ubiquitous around the world has provided us with a new opportunity to reach the underheard. As we know, mobile penetration rates are on the rise. Almost everyone has a phone. And when our co-founders were working in Ghana and trying to connect people to services, they really saw this as an opportunity. And it still is today. And when I talk about the mobile phone, I'm talking about the feature phone, just accepting and sending text messages and voice messages, voice calls. It's as straightforward as that. Of course, you can get more complex, but that's the baseline of what we're talking about today. Of course, the mobile phone is not new, and we've probably been talking about mobiles and development for about 10 years. And there are still pieces that aren't working. So the first generation of mobile for development really focused on text messaging or SMS. And that had a lot of power. It was cost effective. It reached a lot of people, but it's text-based. And so when you think about the traditionally underheard and marginalized literacy is an incredible barrier. And SMS still did not connect those individuals to the services that we're talking about today. Additionally, the software was being developed custom in-house for a specific project for a specific need. And that done at an individual level is costly, it's time consuming, and generally inefficient. Also, one thing that's sometimes overlooked is that when you're using a mobile phone, there are infrastructure investments and connections and negotiations with mobile network operators and telcos. And that piece also is time consuming and was a barrier for a lot of projects. It delayed a lot of projects. When everyone's trying to take that initiative on on their own, it's just another point of inefficiency. And then when you're doing something for the first time on your own without any expertise, it's hard to get right the first time. Avoto, we are the mobile engagement experts. We are by no means content experts. And so when we work together, we're leveraging both of our expertise to enhance projects. And when you're trying to be an expert at everything, I think we know the consequences of that all too well. So what we're doing is transcending literacy barriers by reaching people in their own language without written text. So moving away from text messaging, while we offer text-based solutions, we really focus on the power of voice. And when I'm talking about voice, the technology is called IVR, Interactive Voice Response. But really, it's a voice call. It's a recorded audio recording being sent out as a phone call. And then we're, and why are we doing that? Obviously, literacy is no longer an issue. You can record content in as many languages and send out content tailored to that person's preferences, which adds a whole new layer of interaction and value for the participant. Then you also have our Voto platform. So at votomobile.org, you can create an account today. I encourage everyone to do so. I can give you some extra credit to play around with. And that simple, reusable tool at votomobile.org is our core product. It's where every mobile engagement project we are working on is built. And that means we're doing health projects, agriculture, research, citizens engagement, and democracy and governance. We're working with our partners on one tool, one platform, to serve all these sectors. We're establishing these connections with mobile network operators ourselves, taking that labor off of your plate, off of anyone's plate, taking that on and making sure that those connections are happening, they're reliable, and they're affordable for our partners. And then that last piece, really taking all of our experience thus far and being thoughtful about our design and mobile engagement. We're helping all of our partners and researchers adapt a typical paper-based intervention and putting it on mobile. You know, the order of questions may not be the same. The types of questions may have to be adapted a bit. And that's where we're really adding value as a voto team and in the partnership between us and our partners. And when we think about the lessons learned and all of the work that we're doing, it really falls into two buckets across all sectors. Evidence gathering and behavior change. And when you think about that, they can be connected, they can be separate, but those are the two categories of work. Evidence gathering could be as simple as a survey pushed out once to a group of people to understand current behavior, current attitudes, all the way to ongoing reporting throughout a project. You could imagine a community health worker, client satisfaction survey going out once a month to everyone who is in the community health workers network to increase accountability, understand the quality of service and, you know, identify unmet need. And then in the behavior change component, when we think about behavior change, there's one piece, right? There's the communication pieces of the outgoing information sharing piece, which you can easily do with voice messaging and text messaging across the board, right? Sending out a call is very easy. The piece that we find really interesting is pairing that with the follow-up question. So I'm really measuring that change in behavior. And that's a piece that you can still do on mobile, and we're constantly exploring new ways to collect self-reported data, validate that information, and move forward. And, you know, when we're doing this work, we're doing it through voice because of this, because you look at the participation rates comparing text message engagement to voice message engagement and participation, excuse me, is quite, is just so much higher. We're getting more women voices. We're getting more rural voices, overall participation. People are just sharing more information and they're listening more to the content. With 35% participation, we're much more satisfied with the data collected than with, you know, a 3% or 4% response rate on an SMS-based survey or a text message-based survey. And the final piece that I'll add just to push on the power of voice a little bit more is the types of data we can collect. So when you think about a text message, it goes out, it's sent. That's about all you know. When you have voice messaging, there's this whole layer of what we call passive data that you can observe on the VOTO platform to see how people are engaging with your survey or your project. You could see how many times it takes to get someone to answer the phone. Even though they've scheduled, they've picked their selected time to be reached. You can measure content consumption. So how many seconds do they stay on the phone? In the case of a survey, where did they hang up? How many questions did they get through? And then you can look at the time of day. Where, when is it best to reach that person even though they've selected their preferred time of day? Or in cases where they haven't selected their preferred time of day, we can make adjustments to projects to call them through their kind of past activity. And then you pair that with the active data. Obviously, this is the answers to the questions you're asking, right? What people are self-reporting in terms of behavior change? If they're talking about attitudinal shifts, you can do ongoing panel surveys once a month to understand how people are reacting to radio programming that you're pushing out through traditional development projects. You can evaluate citizens' priorities. You could use that as a panel as well, or you could just do it as a one-time survey somewhere and somewhere in between. And then obviously, informing future programming. You can ask questions about what works and what doesn't work for the participant. And so in pairing, really, in pairing these two with the power of voice, you end up with just much more interesting data that can result in even better projects. So I'll stop, I'm gonna shift gears now that I've done a bit of background and dive into a little case study of a project we did that we're very proud of, and this was a VOTO-led project. Basically, what we did was take your mobile Alliance for Maternal Action content. So Mama created a curriculum for pregnant women and we translated it into about five, four or five different local languages in Northern Ghana, enrolled about 8,000 women into the project and they received timed curriculum based on their expected due date. So you would imagine, you know, the call went out every Wednesday at noon, but Mother A was at week 27 of her pregnancy, so she got week 27 message and Mother B was at, she had given birth already, so she was at her after birth messaging, but all receiving it at the same time during that same week. And so the really, this is where we're looking at, I'm gonna show you two examples. This first slide is about the passive data and then the next slide is looking at the active data and then we're gonna put it all together. When you look at the passive data, this is what we call our engagement rates, which is the average percent of the messages listened to by week. So on average, we had about 74% consumption of the Mama content, so in general. And then every week we could look and see what people were listening to and how much they were consuming and start asking questions. So here you can see like there are dips and we were wondering why was this less interesting? So it triggered us as the implementers to react very quickly and see, hey, what's the problem here? Let's do a deeper dive. Our reaction was to add in one question at the end of each message asking, it was a rotation, so it was one of three options. Was this message useful to you? Did you understand this message and would you refer this project to a friend? Those three questions, one of those three questions now were tacked on to every message that went out. And the results were really interesting. So you have this graph here where we're showing how people rated the usefulness of each message and you can see with it, you know, which weeks performed really well and which did not. And imagine this happening on an ongoing basis. So in real time I can look at this week's people's reactions to their message this week and say, hey, everyone who got week 40 of pregnancy this week didn't like it. It was not useful to them. How can I adapt? And that pairing, so the trigger was the passive data. Then we started collecting some active data to understand more. And then together, those two pieces of information helped us create a better program that really yielded some interesting results. So in Northern Ghana, or in Ghana overall, the expect the rates of attended birth is about 57%. The women who participated in our project self-reported, 86% of them had given birth at a clinic. And that's staggering. It's, I mean, we were so impressed and also the satisfaction in general with the project and the program, they're very, everyone's very happy. And we call it the net, well, we don't call it. Everyone calls it the net promoter score. So was this, would you refer this to a friend? 93% of them said yes. And so by pairing the active and passive data, we can see behavior change. Granted, it was not set up as an experiment, but it is the beginnings of a really interesting work that we are hoping to expand into more robust research and working with researchers, such as yourself to do so. So I will say that we are now in this world of active and passive data across all sectors, learning so much and sharing those learnings across sectors, removing those, breaking down those silos between subject matter and saying, hey, this is how to reach the right person at the right time. I will, that's the end of the case. I will just briefly show you a snapshot of our platform. This is where any intervention is designed. We call it trees, it's basically a flow. You can design your conditional logic and see how you can kind of just flow through a survey. You can see there are three types of questions, multiple choice, numeric and open-ended questions. These three questions can then be kind of stretched to any imaginable use. You could use the NPS, Net Promoter Score, how satisfied are you on a scale of one to five and have people enter one, two, three, four, five. You could have people enter their age, instead of having to put people in two brackets of ages, so ranges, you could just ask them their age to enter that. And then open-ended questions allows you to have this qualitative data experience over the mobile phone. So now you've had participants complete a survey and you want to have basically a comment section. Is there anything you'd like to add? And people can just speak into the phone, their voice is recorded, and you can listen to it on the platform. And then that audio file can be translated, transcribed, used in more qualitative analysis. So I will stop here because I've been talking for quite a bit. But in conclusion, there's the pairing of active and passive data using voice messaging to find some very interesting triggers for adapting projects and then influencing some significant results. And all of that can be done at votomobile.org in partnership with us and our implementation team. So I'll pause here, I have plenty of cases I could talk about and I will leave my contact information here as we go through Q&A. But feel free to email me as you think through any challenges or research questions you may have in mind. Thank you. Thank you, Melissa. This has been a very contextual overview of votomobile and what you've been able to achieve. So we already have a question that's popped up. So I'm gonna address that one first and encourage anybody else who has questions out there to enter them in the Q&A window please. So relative to how you engage with survey takers, the question here is if you ask them to respond to the SMS, don't you have the same feedback data as if you use IVR and those who are not sure what IVR is, I believe it's interactive voice response? Do you get the same? So well, I will say with SMS, there's a character limit to begin with, right? So you're at 160 characters and if you're typing in a different language, that number actually can be smaller because of accents and characters taking up just more character space. So there's that piece. It does still require some level of literacy and honestly more understanding of how to type on a phone. Using IVR, you're able to... It's quite simple, right? Press one, press two, enter your age. It's a few pushes rather than having to write out your thoughts and your responses via phone. So those are the main reasons. And then the last point is cost. Text messages can still be costly for the respondent which is a barrier. With voice messaging, we can use a missed call feature or flashing or beeping depending on your context, whatever your... Whatever the locals are calling it where you call into a phone number, hang up and the platform then calls them back and their response is free to them. Or you call them directly and they as receiving the call, it's free to them. And it happens all in one instance. So I pick up the phone, I complete the survey, I hang up. If I'm texting and there's service cuts out or there's a delay in the message back to me, I get distracted. Really, the likelihood of me completing a survey as a participant decreases significantly. Is there just a kind of building on that question a little bit? And I'm not suggesting that you have this available immediately at hand, but could you direct some of our listeners to resources where there's data behind some of these kind of anecdotes or these specific findings that you've had in the field? Yeah, so a likelihood of response or... Yeah, a comparison is hard to find. We're working on that. So we, in some of our national survey work, we've worked with Center for Global Development and they've published a working paper on national representative sampling using mobile phone, mobile surveys. And that's powered by the VOTO platform. So I would encourage people to look at that. It does show some costing information for voice surveys. But in terms of comparing, I will do some more research. It's something that we think about every day and I don't know anything offhand, but I've come across them, I will definitely be sharing it with you. That's fantastic. And for anybody who's listening who is doing research work in the stereo, having trust, maybe this is something that you can also investigate and feel free to share with us. We are going to be publishing an article after the series is complete featuring the platforms that we've highlighted as well as others and would be happy to include those resources as part of that overview. So another question has come in and this one is quite specific. So relative to this organization, the question is their biggest issue with data gathering is trust. So many people have worked in Cabera and in Nairobi that participants simply tell the survey person that they think what the interviewer wants to hear. What have you found with trust issues in IVR? So again, this is anecdotal to date. It's definitely, we're scratching at the surface and really trying to be more rigorous when we come across these kind of inklings and insights. What we understand is that people are more likely to provide information about taboo subjects using IVR than they are to give that information face-to-face or to a person calling them, so a human on the phone rather than the pre-recorded kind of menu that they're navigating. And as we are working more and more insensitive area, so we have a few projects that are starting up in the gender-based violence space, we'll be able to glean more insights in terms of trust. And we kind of know it inherently. People are just sharing more information over the voice messaging, but the rigorous analysis hasn't happened yet. Next step. Yes. So another question more from the practitioner perspective and those who are looking to use VotoMobile for their work, can you seek at all to the costing structure for the users? Yes, so there is basically on the platform today you could send a call anywhere in the world and the platform will charge you a marked up airtime rate. So that's at the basic level. As I mentioned, there are a lot of learnings on kind of order of questions and how to reach out and engage with individuals through their mobile phones. We found that our partners are just much more successful if they engage with us even for a short term to do the co-design and then kind of run off with it on their own. So in that case, it ranges between $1,000 and $5,000 a month for the co-design, co-creation, full platform access, ability to have two-way communication, all of those things. And the range is so big because it really comes down to volume. Right, right. Thank you, that's very helpful. We have another question relative to other platforms. So how does VotoMobile platform compare technically with that of a group like Medic Mobile? Is it Open Data Source or ODK? Yeah, so Medic Mobile is a great platform. We are different in the sense that we're more on the messaging side of things, but we can connect, we have connected with existing platforms that are doing tracking information or doing patient management so that they can roll up their messaging to one source. So you could design a messaging campaign, an example would be like an adherence campaign. So you're an adherence initiative, excuse me. So you need to take your blood pressure medicine every day and then you need to get your blood pressure taken once a month at the clinic. And so all of these behaviors can be encouraged through messaging. And then when you do not take action, let's say you do not show up to the clinic, it could trigger a call to say, hey, come to the clinic, it's your time, remember? And kind of keep reminding and getting people to take action. Then all of that information could be rolled up into the other tech tool, so something like Medic Mobile or something that's doing patient management to like calm care. And ODK is interesting, so that's like a tab, it's the way you would design a tablet or smartphone survey. And we've worked with ODK to basically design a survey that enumerators use when they go out and one of those pieces, one of the pieces of data they collect is a phone number. And then the phone number is connected to Voto and all of that, the data collected face-to-face using ODK and the data collected through Voto Mobile are synced up and you have a full participant profile. So we are kind of a piece of the puzzle, complementary to these types of things but are quite different. Got you, got you, and a few more questions coming in here. Do users of mobile phones know how to use touch-tone, response, or slash voice surveys? Yeah, it's a great question. They're, I mean, of course it's context-specific but it's quite intuitive. We have a lot of people who don't use touch-tone but it's context-specific but it's quite intuitive. We have been doing some work with the World Bank on trying to figure out on some sensitization and training on how to navigate the survey, particularly in, this is really easy and useful to do when you have a high-touch situation. So there's someone, you have either enumerators who have gone out at one point or you have a community health worker who's engaging with individuals and they can just kind of do a quick five minute, this is how you do it, this is how you press one, two. When it asks you to do this, you do this. We've also designed some visualizations, like little flyers to give out so people can remember this is how it works or like this is how the missed call works or some of those little nuances but most of the interventions we've designed and the research that we've done has been based on people's existing understanding of the phone. As I mentioned a little bit with the SMS versus IVR discussion, you don't have to push that many button so people get it very quickly. Fantastic. So we have one, I think last question and I am going to say that I am no ITT expert so this may make sense to you but I have to say it's a little bit over my head but what's the difference between a survey and a tree? Sounds rather philosophical to me but you might have a more academic answer to that. Yeah, no, so we call it a tree because you could take a survey could be seen as like a really technical thing, right? You, it's a type of, it's a thing you do, it's a survey, you design the survey instrument. So we call it a tree, you could call it a flow because it's basically anything you can build that needs some flow or a conditional logic. In some cases it's a quiz, right? It's like, hey, you participated in an in-person training last week, what do you remember? And you like take a quiz. It could be labeled kind of something else but it's just a name to not box it in. Thank you for that. Yeah. Fantastic. Well, I think with that we have come through most of our questions and this was a quite rapid overview but we do have an opportunity for our folks to email us as a follow-up and also reach out to Melissa. She shared her information. They'll be recording of this webinar available. With that I'd like to thank everybody for attending. Certainly for those of you who are speaking to get your professional development hours, the code is available on the slide. Feel free to email us with additional questions. We will follow up with an overview of the entire webinar series including links to recordings and hopefully some additional resources if we can find them. And I'd like to invite everyone to become an E4C member to get information and invitations for our upcoming webinars. And last but not least, thank you, Melissa, for your time today and joining us on this series. Thank you to everyone. I hope you have a great afternoon, evening or morning wherever you may be. Take care.