 Yeah. Team effort. So, Brendan, I think when you're ready, so mostly so. Hi everybody, thank you for coming today. My name is Toby Negrin and I'm the Head of Reading here at the Wikimedia Foundation. And I'm going to tell you both in San Francisco and on our video conference about a project that we've been working on called The New Readers Project. So when we started the reading team a year and a half ago, we took the mission as our guide in the kinds of things that we wanted to do and I'm just going to read it to you. The mission of the Wikimedia Foundation is to empower and engage people around the world to collect and develop educational content under a free license or in the public domain and to disseminate it effectively and globally. And we really took this last part of the mission as the touchstone for the reading team and for what we wanted to do. So, when we thought about, you know, the globe, the puzzle globe as, you know, sort of the, you know, as the globe, we realized that we really were only reaching about a sixth of the people on the planet, right? People in North America, people in Western Europe, people in Japan and we felt that there was a real opportunity to take some amazing content that our community has created and get it into the hands and minds of more people. So we envision this New Readers Project as a way to reach the rest of the planet. And I want to give some thanks and gratitude to the New Readers team because it really started this project from a position of asking questions and doing research and engaging in meeting our users or our potential readers where they are. And I don't think this is, I think this is something that we need to do more of at the Foundation and it's something that I'm really excited to see come to fruition. And this presentation is going to be about that research and some of the findings to the methodology. So I think that's, I think it's really great and thumbs up to the team. And so these are basically the two questions, two like high level questions that we were asking. So going from the mission to the globe, one of the things that we also realized fairly quickly was that the planet isn't a one size fits all kind of thing. And if we were going to be effective we needed to be a little bit more sophisticated in the people that we were trying to reach. So we did some qualitative and quantitative data crunching and you can check it out in the link. But we basically arrived at three countries, Mexico, Nigeria and India as places that we wanted to investigate. This is about a billion and a half people all told. You'll see from the research that they are both very similar and very different and we think this is really an effective way to get Wikipedia into the hands of more people essentially one country at a time. So thanks again for coming and here's Ann to take you through the deck. Hello. So yeah, I'm Ann Gomez. I'm the product manager with the reading team. And yeah, so just as Toby mentioned we, when we started to, when we decided to go to these countries we really knew that we needed to sort of reset a lot of our assumptions and expectations and understand that people who aren't among our existing editing community like, or work at the foundation are probably going to have a pretty different understanding of the internet and of Wikipedia and all that. So we went to these people to actually learn about how they learn, about how they approach the internet, about how they think about Wikipedia. Before we get too deep into this, some quick logistics. So we're going to be taking questions at the end of the presentation. There's tons of time allotted for that so note them down. Joe is going to be looking at questions on blue jeans and an IRC in the Wikimedia office channel. So if you have those there you're welcome to put them there. We'll cue them at the end or we will have a microphone for those of you in the office. Yeah, so that's that. Now we're going to get into the content. So when we started this research we really decided to investigate along three perspectives. We knew that our existing communities were really important. We knew we wanted to get a broad look at what people across the country were thinking and how they thought about things. And we also wanted to get really deep as much as we could in certain areas through deep design research. So I'm going to pass this on to Zach McCune from Communications to talk about how we approach the community. Thank you. So yes I'm Zach McCune. I work on global audience development on the communications team. And bless you. And yeah I want to tell you a bit more about how we worked with community. So as Ann said we started this project focused on new readers potential folks that we could reach around the world. But we understood that our communities in Nigeria and India and Mexico could tell us much more about what they knew and more importantly what they didn't know and wanted to know. So starting in January we began liaising with folks across these community groups emailing with the lists following up with our friends in CE who enabled us to kind of reach key voices there. And at that moment I want to just thank Asav Bartoff, Katie Love, Casey Harreld, Maria Cruz and Joe Sutherland as well as frankly everyone on the community team here at the foundation for helping us to do this effectively. I also want to say that you can note here in the notes that we basically proceeded with a cadence of going very broad and then trying to set up time to actually speak in person or on the phone. So wonderfully when we were able to get to Lagos in Nigeria we met with the user group there. Here they are. They were fantastic to meet and in fact we had so much fun with them that I wanted to wear my Nigerian Super Eagles soccer scarf today to show pride. They did really well at the Olympic Games finished with the bronze medal. It was a great result. When we say we consult it with the community this is the kind of things we tried to focus in on. We wanted to know what topics were wide open. What topics were very urgent. We also wanted to establish language priorities particularly in countries like India where there are dozens of massive language groups and essentially 20 plus languages that are recognized by the government in the way that they publish across India. We realized that we couldn't focus on every language group so we needed to create a hierarchy of language and Dan Foy on Global Reach will tell you a bit more about how we went to work on that for phone surveys in just a moment. We also realized that as we think about readers and not editors we wanted to know what our editing community thought they knew and really were desperate to find out. So essentially we kind of went to the people that are closest to making Wikipedia to try to ask them what they wanted to learn about the people who might use it in their regions. And that sets up Dan Foy here from Global Reach you can tell you a lot more. Great, thanks. Hi, I'm Dan Foy. I work on the Global Reach team talking about the phone survey portion and the phone surveys were done by the Global Reach team in coordination with the reading team for new readers. So we have what the phone surveys are all about is reaching a large number of people and making sure that we understand how widespread certain things really are. And so a lot of this is about why use a phone in the first place. And the reason is because there's a lot of people that are not in our communities that are potential readers and we have no other really way to reach them. So the phone surveys are really about quantifying the awareness, usage patterns how they use the internet and their general demographics and around 20 questions per respondent. So for these countries surveyed here, Mexico, 2500 responses we presented it in two different languages. Nigeria 2500 four languages and India much larger 6,012 languages. So in addition to covering the different countries we want to make sure to make sure that everyone in the country was fairly represented. And so what we did is when we can target a survey by area, by changing where we were calling, we would say for high population states they would get a fortunately higher share but it also guaranteed that the lower population states were also heard because the risk would be we would just be sampling just a bunch of people in one or two large cities in the country and they really wouldn't be really representative. Okay. So India was particularly challenging because they're so large and so diverse and we had 12 languages that we chose to do this and see. It's because there's when you get a call and you may not understand the language and asking, rattling off a list of 12 languages is just going to be overwhelming. What we did is each region we picked the top four languages for that region and then simplified it that way. And you can see the list of languages that we used in India. So I'd like to pass this on to Abby. Thanks Dan. I'm Abby Ripstra and I'm the lead design researcher here. So I'm going to talk a little bit about design research. So we do design research to understand what people need to understand people's motivations and goals and constraints in their context so that we can help them access and potentially contribute to free knowledge. Some of the ways that we do that there's different methodologies, the methodologies we used in New Readers Research. We're some ethnographic interviews where we went and interviewed people in their contexts and where they work and live. We also asked people to demonstrate how they use their technology when they're looking for information or what kind of apps they use and all kinds of things about their context of technology. We also did some key informant interviews talking to experts about the use of technology and education and how people get information. And we also used the phone surveys as Dan spoke about. So we did contextual inquiries in each of these three countries and the first place we want was Mexico and it was the first research we did, the first contextual inquiry. And so we did a small team of six of us from the foundation and there were, we did 15 interviews and we were in Mexico City, Puebla and Apizaco and we were there for two weeks. So this was the first time we did this and it was a prototype and so then the next couple research projects we partnered with an agency called Reboot and they're experts in contextual inquiry. They do them all around the world and so we were able to scale our research and so we had a little bit larger teams and a different, a little bit, we upped our game a little bit with them. So we, in Nigeria we were able to do 70 interviews and we went to Lagos at Bambini City and there was a team of 11 consisting of Reboot team, three from the foundation and four local researchers. We were there for two weeks and then in India we also did 60 interviews with a little bit different team but also three local researchers, two staff foundation and two people from Reboot. We were in Delhi, Jaipur and Chennai so we were there for two weeks also. So I'm gonna pass it over to Ann to start talking about some of the findings. Alright so this is the real meat of the presentation guys and all this content is on Wiki too, we'll make sure to point you that link afterwards. We know it's a lot so if you have questions just note them down, we'll take time at the end. One thing I did want to call out also was that the ethnographic, the design research we went specifically also to more urban and peri-urban environments and just to kind of like string everything together then we had the phone surveys to help sort of grab the respondents from more rural areas but just so you guys have an understanding of where this information is coming from. So we compiled all this research from all these different sources into themes. It's a lot of content like I said there's seven themes. We're gonna move kind of quickly, like I said take down your questions. The first one is information seeking. Yeah so the first finding we have here is that people seek news and actionable information first and context second. So what that means is that among the people we spoke to we learned that generally browsing and information seeking are something that are triggered by some sort of something happening around them whether that's school assignment or work assignment, current event, something that's come up in their day to day lives and that finding deep informations which our projects excel at sometimes require deeper processing than people are necessarily looking for at that time. In our three countries we learned that people often use diverse sources to find news and information that they trust. So here we actually saw some kind of differences in behavior between Nigeria and India and in Mexico. In Nigeria and India people were often relying on reputable non-local sources for information about their state or their country or the world like BBC we heard, Al Jazeera, New York Times but for local news they relied on community sources either peer to peer or local publications. Also in Mexico we learned that many people we spoke to talked about triangulating information so there was a general sense of like well I'm not sure if I believe that so I'm going to try to find it in other sources or ask my friends and circle in on what I think the real truth is here. So people seem to assess the credibility of well known sources like we talked about BBC, Al Jazeera, those guys only when they were finding information for something that would be externally assessed. So like a school assignment or a work assignment. People were much more interested in being critical than maybe unsurprisingly. But that said people don't really need to trust an information source to find it useful. So if they find information that doesn't suit their needs they might canvas different channels blend together a picture of what the truth of the situation is or they might just discard it and start over. We saw behaviors around people opening like five search results and then kind of picking the ones that fit them best. Successful information systems meet users where they are today while also evolving with their changing information habits. For example in Nigeria the lottery system is really popular and people the lottery system there has evolved so that you can check the numbers or the results of it basically any way you want to like by your phone, by Facebook, at kiosks like they're in the newspaper all over the place. We also found that in India people are still using printed newspapers like that's actually an industry that's growing in India which we were really surprised to learn. No surprises here visual content and design helped attract users. Exciting GIF. So for example here in Mexico we heard over and over again when we asked people where do you like to learn? How do you use the internet to learn? People would say YouTube, YouTube, YouTube especially around tutorials how to is making recipes and people really like content that they can see. And with that I'm going to pass this on back to Dan Foy to talk about what we learned about how people access the internet. Thanks. Alright, so some more findings. New readers found constant internet, individual internet access is not normal for everyone and this is a lot because there's a lot of connectivity issues. There are areas where there's simply no wireless available. There's cost issues keeps people off the internet or keeps a very limited amount. Two addresses people share devices between families they can also between sometimes friends see that quite often and then yeah that's so there's plenty of lack of connectivity and intermittent connectivity that exists it's very prevalent. And the next thing is mobile dominates for getting online certainly there's no since the infrastructure is less in these countries than wireless works well and the mobile phone costs are a lot less than computers. Android is a platform of choice probably mainly economic but it's very prevalent. So for the findings we ask if they use the internet because we just call random people we don't know. In India and Nigeria they're pretty close at almost two thirds and then in Mexico it's higher at 80% people said they use the internet. Now we don't know how often this is not something that dives into is this a shared connection or do you subscribe or anything but just if they use the internet. And then we ask what they use the internet for the most and really what happened what we see here is social media is probably the most popular but very closely followed by just looking up information. So it falls well within our area and then entertainment and news were definitely secondary to those two things. Now the next thing is can you use the internet with your mobile phone and this is just the way we worded it to avoid issues around what definition of a smartphone is we just wanted to know essentially if they had a smartphone but just to make sure that their phone could access internet or could not. And so we see is we still have a lot of feature phones out there dumb phones because we did call them we call them on mobile phones. So the area you see here where they said no means they have a dumb phone. And dumb phones for years have been predicted to be to go the way of the dinosaur around the world and this really hasn't happened and it hasn't happened for a couple of reasons. One is just the cost of the little Nokia candy bar phones are really cheap. And the second thing is battery life. If you have intermittent if you have intermittent access to electricity then a smartphone needs to be charged a lot and so when you those old Nokia phones they could go for weeks on a charge and that's not really being improved yet on smartphones so really don't see that changing drastically here. Okay so in Nigeria is probably the highest internet cost and because of that consumers do shop around a lot. They use switch SIM cards around in order to get the best deal. India is somewhat better in terms of cost but it's still very high. In Mexico we do see some Wi-Fi usage to offset things but that's more about Mexico than the other countries. The other countries tend to be limited access to Wi-Fi. There's not broadband at home mobile is it and if there is a Wi-Fi spot it tends to be a shared 3G which is really bad and this represents the pricing and some of the reductions that would be needed in order to make 500 megabytes cost less than 5% of their income. So these highlight the New Readers countries here and I'd also like to point out that unlike most of the US and western Europe that people here are buying are not on a monthly plan. They buy data essentially by the tens or hundreds of megabytes and so they're always on the meter. It really discourages browsing, discourages exploration because of cost. They just have to choose. Everything costs some money. Word of mouth is really important to use for discovery. I'd like to pass on to Abby. We learned some things about how people are using the internet. People's understanding of the internet and how it works can be confused. There's conflation between browsers and search engines and there's conflation between what is Facebook versus Google. Is Facebook the internet? So a lot of not clear understanding of the technology and what it makes to make the internet go and what's an application and what's a browser and things like that. People are learning from each other how to use the internet and how to use technology. Children are digital natives are teaching digital immigrants how to use technology. For example, in India we heard several stories of kids giving their mom and dads a phone for when they go away to college so they can all keep in touch and installing WhatsApp on the phone to make that connection strong. So people learn about how to use technology from their family, from their people at work, from people they meet to share that information. It's about using the internet. This one is a really interesting finding in all three locations and it's also different in each place. English is the language of the internet as well as computing as well as the hardware keyboards. That creates a challenge. People prefer to speak in their local language, their language of comfort, but then it doesn't necessarily translate into how people are typing and writing and searching online. Sometimes it's because the content isn't there in English. That causes sometimes depending on someone's literacy in English versus their language of comfort there may be workarounds. For example using Google Translate or a translation app to still get the information they're looking for even though it's there in English. So they'll use a workaround to get that information. And also having to, for example we saw people having to type out Hindi in Roman characters. So there's a lot of very complex things that people do because when people are motivated to get information they're going to get it, but if there's a barrier they're going to be a workaround, they're going to figure it out. So we were very lucky to get to observe some of these workouts which will help inform us how to support people getting information. It's a really interesting challenge. Also as we can kind of, as we, as it seems obvious from some of the things we've been hearing just now that people are precious about data usage and things like UC browser and Nigeria was more, was big and Opera Mini in India are browsers that are designed for low bound with and to help people get the speed, get it a little faster, get their information faster and also it reduces cost. So in Nigeria it was, the big deal was UC, which I get I'm confused, but UC browser helps it be less expensive and it's faster. In India it was more, Opera helps me get the data faster and also it's less expensive. Okay, so that's another kind of workaround to the infrastructure that's not meeting people's needs and UC browser and they found a niche to function in. So also this one, you know, mobile apps everyone uses them, they've exploded in popularity, you can see Facebook and WhatsApp on almost everyone's device, whether it's a phone or a tablet and yeah, instant messaging, social media they're way up there in what people do on the internet and social media isn't only used to coordinate and connect and check in on your friends and say hello and communicate, it's also used for content sharing. For example in Jaypur we interviewed a senior attorney and she was describing how she uses WhatsApp to discuss legal content and things with her junior attorneys as well as get consult from her senior attorneys and they're passing content and documents. It's not just hey how's it going anymore, it's like a very serious method of passing documents and content and information. So that was something that was surprising from my perspective. I use WhatsApp sometimes, but I use other ways to share content so this was pretty insightful. Also in the context of education there's conflicting views about how to use the internet for formal education. In Nigeria particularly we noticed that students are not necessarily inspired to use the traditional books and content of education and so they support each other in work arounds of sharing content and getting information online. It helps save with cost and also it makes it more interesting I think. So there's a lot of work arounds and when it comes to Wikipedia teachers and professors say to their students in all of these countries don't use Wikipedia. For one there's trust issues sometimes in the content. There's also not being able to use Wikipedia articles as a reference though there are other references down at the bottom. So there's and also teachers in the lower grades of school are very concerned about copy paste plagiarism so there's some concerns and at the same time students do use all the students said that they used Wikipedia for learning definitions and understanding a topic at the high level and then there's also looking across various sources to validate that content but students are using Wikipedia but the educators are not so sure quite yet. So there's a tension there and I think it's a really interesting tension to explore. So now to Zach getting information online. Thank you Abby. Thank you Abby. Is this on still? Great. Okay let's talk a little bit about getting information online. So so far we've looked at the way people seek information, the culture around accessing the internet and some of the ways that folks understand the internet and now we're going to look at kind of bringing those two threads together into how folks get information online. People just online search with Google in particular to get them what they need. This was something that we observed across regions and it's something that you can imagine is baked into that preference for mobile devices. With the ubiquity of mobile as something we observed, folks were opening up their mobile device and either using the search toolbar built into Android or built into their OS and putting their browser or they were going to that browser and entering a query there. Folks also occasionally would use an app to start this and as you saw with Abby's slides a few moments ago there are Google search apps where people begin these things. One of the interesting things about trusting online search was this idea that it's a beloved part of their life. So these are some of the things that people phrased Google to us. My big boss Google, uncle Google, Google is the shortcut, Google is the solution to the world, Google Maharaja, also we had Google Eva which is Oracle in Yoruba as well as a host of other. St. Google in Mexico so again there's this personification and even deification of Google as something that will get the answer regardless of what the source is. I'll also note that this sets up an interesting challenge we'll talk about a little later which is the idea that if Google delivers you to the information you need Google kind of becomes the information you need so there's this conflation between the beginning of your query and your destination. In terms of the search habits we saw they were relatively basic and they used a lot of trial and error. So students when we spoke with them would talk to us about kind of like starting to query for information for a school assignment. If they had to write about a part of history they would initially start with a very broad topic, French Revolution and then might start to zoom into specific attributes, causes of the French Revolution, end of the French Revolution, participants in the French Revolution and they would often open five or six of the results right away from the results page. So there was this multi-tab, multi-channel habit where folks would receive the results and quickly open up multiple tabs. And then in terms of this idea of looking for quality indicators this actually means that they were looking for a form of pattern recognition. So they were either looking for a brand that they recognized for example MIT. This is something I know stands for quality and I can trust this as a source. Or they were looking for something that reflected what they heard from teachers. In Nigeria during our qualitative research I actually observed a student who was looking for a specific phrase around a history topic that her teacher had said. So she was looking for the idea that the Russian Revolution had been caused by investment in World War I. So she was looking at the different results for the query which was Russian Revolution but was looking for World War I to kind of have this like ah yes I'm in the right place. This is a kind of fascinating tension we saw throughout. In an era of search led, task oriented browsing there is little loyalty to specific web properties unless they relate to personal passions. So let's start with the personal passions. Right here we have a photo of some supporters of El Tri of the Mexican national team. And what we found was that within certain subject areas such as football or pop culture people had specific URLs that they knew they were going to. Goal.com, ESPN, live scores or well known pop culture places. But after that loyalty went out the window. And the paradigm that URLs are a vanity marker in the web. Something that you recall in your mind or bookmark to return to was something we did not observe much of. Rather folks started their search for information about a topic at the search bar level. As Dan talked about internet access can be very intermittent. And this means that we are finding folks who have access to the internet planning for moments where they don't have access to the internet. And that means increasingly developing habits where they get information online but then use it or share it offline. An example that Abby likes to remind me of for this is that in Nigeria power comes on and off. There are frequent rolling blackouts. And with those blackouts comes loss of connectivity. There's also basically downtime for the internet that don't relate to the blackouts per se but still constrain users who would otherwise use it online. This means that in Nigeria when an internet user finds they have good connection they actually often make a series of actions very quickly. They will download things, movies, files that they expect to need in the next couple hours or even the next few days. So this is an area that we're particularly interested in because it's a way for us to potentially be predictive and helpful in the technology constraint for internet users. Now we have arrived at the moment of talking about Wikipedia. So the next two sections are Wikipedia awareness and Wikipedia usage. And we're really excited to present these parts. I do want to remind or rather explain that during our interview processes we did not assume that people knew about Wikipedia. That meant that our research script began with asking about how they find information, whether or not they use the internet, how they use the internet. So we did not start with this idea of we are from Wikipedia and let us explain what Wikipedia is. It enabled us to keep the experience very open-ended and allowed us to learn more about some of the challenges and limitations in understanding Wikipedia. Overall as a brand Wikipedia is not widely recognized or understood. Some people are Wikipedia readers without realizing it. So this is a major finding for us within India, Mexico and Nigeria. Some of the statements that we would hear about Wikipedia from definitions were Wikipedia is something you can get over the phone. I am searching in Wikipedia when a user was searching in Google. Wikipedia is run by a non-profit and donations. That was said by a professor in Chennai. We were thrilled to hear that. The phone survey findings went deeper into this question. So let's consider these for a moment not as a group but as individual countries. In India 25% of the 6000 phone survey participants said they had heard of Wikipedia. Now again the phrasing is heard. That means recognition not use or familiarity or advocacy. When we go to Nigeria that number drops to 23%. And in Mexico it's 45%. So in all three of these countries less than half of participants said they had heard of Wikipedia. Just could recognize it. Now this is a fundamental brand awareness challenge for us. And folks may ask why does brand awareness really matter for us if search engines are delivering our content and people are finding their way to our free knowledge? I think the answer is that without brand awareness you don't know where you're getting the information from. You don't know the values that we hold dear as a movement. You're not an advocate to tell other people about Wikipedia and about the way that our knowledge materials are produced. The costs or lack of costs associated with them. And you also don't have an opportunity to make this into a part of your habits. So should a search engine redirect you, you will not notice no longer traveling to Wikipedia. So we believe that this is a fundamental space of action and as somebody on the communications team you can imagine that I'm particularly interested in this. We also found that people confuse Wikipedia with a search engine or a social media platform. And this can create unrealistic expectations. Here's a few of the things people said. Wikipedia is a poor cousin of Google. It is the lesser model. Google and Wikipedia are similar. Google is more distributed. Wikipedia is more analytical and comprehensive. Wikipedia is a social network. You'd use it if a friend in the US was on it and you wanted to connect with them. That last person was in Lagos and they were wondering how many social networks that we could connect with them on and they were like, Wikipedia, that's a good one for us to be friends on. Also in the phone surveys we went deeper on to this topic as well. So among people who said they had heard of Wikipedia we asked how. And you can see here that across the regions the internet continues to be the leading way for people to find out about Wikipedia. But as Dan Foyd pointed out, and I really love this Dan, if we basically combine school and friends and family and think about this as people, then we can see that people are equal to the internet as a site of reference and a site of referral. And that really emphasizes the importance of advocacy. So again we go back to that idea of low brand awareness and we look for people telling you you should go check it out on Wikipedia. Because school and friends and family are places where people who do know about Wikipedia are being advocates for our content and our movement. And with that I'd like to turn it over to Ann Gomez to talk about usage. This is our last section for those of you keeping track, so almost to your questions. So Dan talked a lot about affordability, that's come up a number of times as you guys have heard about people having affordability issues around using the internet in general. And we found this to be true as something that people cited as a reason for not using Wikipedia as well. One thing I want to call out is that looking at this graphic, this is specifically among phone survey respondents who said that they rarely or never use Wikipedia, but did indicate an interest in wanting to use it. So these are people who want to use us but aren't for some reason and why they say that they're not. So expensive data was definitely a big barrier, there are others here. One thing that I find particularly interesting is that lack of trust in Wikipedia and in the content was really not cited very much. With the exception of in Mexico where it's still only 26%. So it's an issue but it's perhaps less of an issue than I think we end up talking about a lot. So much like how people use the internet, most Wikipedia readers are generally task oriented, they're not exploration oriented so at least among the people that we talk to this kind of works against the whole rabbit hole usage pattern. The people were generally talking about coming to Wikipedia for an assignment, for school or work because something's come up and that it's not really about coming to our content as a destination on its own, maybe in some part because of what Zach just talked about as well that people don't have awareness of what they might be able to get. And in the phone surveys we were able to sort of pull this together as well and see that the majority of respondents said that they were coming for school or work that there is still a large number of people coming for entertainment or other reasons as well. So I just want to bring up again that Zach mentioned this but we asked people about Wikipedia and so it didn't come into the conversation at the end of the interviews, it didn't come up until much later. When we did we generally came around and was like well how do you think the content is written and almost universally people didn't know. And when we would ask them if they'd ever edit or if they knew they could edit almost universally again people were shocked like you could visibly see this visceral reaction of like I can't believe I've ever trusted anything on that website. Despite this there was no observed relationship between trust in and reading of Wikipedia. Like we said earlier people tend to find if the content is useful that it's good enough whether or not they trust it really is a source. And just to make you guys feel better at the end of the interviews we did go through and talk to people and explain a little bit more about moderation and bots and the wonderful work of our community of volunteers and the studies that have been done that say that the content is more accurate than other sources. So we didn't make anyone into like an enemy of the movement as much as possible of course. Yeah so that's all of our findings. We have some information about kind of what we're doing next as a team, the new readers team which I don't know if we mentioned reading, communication, design research, partnerships and members from community engagement as well. We've got some stuff that we're doing but we wanted to make sure to reserve as much time as possible for questions about the actual research because it's really important to us that you guys be able to take this into your work as well both as volunteers and as staff. So we're going to pause now for questions and ideas and if it comes up we've got some slides about next steps but we'll also make sure to reel it back in for the last few minutes at the end here to talk about that too. Alright. Push the button. Hello. Oh there you go. Fantastic. We have a couple of questions on IRC. One from James who's asking do we have a phone survey in the US or European countries we can use as a baseline? Not yet but I think that's a great idea. And one from Mel will we do the surveys again so we can make comparisons over time? I believe it's the phone surveys. I think that's something that we're we would like to do that's still being decided. The question from Mel it was whether or not we're doing the surveys again later on to track them over time. Hello. Can you hear me? Yeah so we are considering doing that running additional phone surveys. We did run Brazil and Egypt as an addition to the initial ones that we're planning on running and another idea would be for the countries where we are doing more product development or even launching new partnerships we would like to do phone surveys before we do something in that country and then another one after we have done to track progress if the initiatives or things that we are launching there or partnering there are working or not. So yeah that's in the blends. Hey y'all I'm so impressed. I really am. The trips to these places knocked me out. That comes back with stories I see what you guys bring back. I think it brings an understanding that we couldn't get otherwise. I wanted to ask the theme of a utilitarian use of Wikipedia seems to come up a lot. Homework, people bullying right through stuff that maybe they can't trust like don't care going to English Wikipedia if the article isn't there. If that practicality is a theme that you guys saw have you already started thinking ahead about ways that we can try to address that and do you think this is representative of other places where we need to grow and take into account that we are not a sort of browsing library for the world that the world wants some stuff right now and I just wondered if that is one of your focuses. So yeah we'll go into our next steps. We'll come back to personas in a minute here but yeah we did hear that a lot and the short answer is that that's not something that this team is actively considering pursuing right now. We've narrowed in on wanting to work from four of these findings. So the two on the right here and also the one about addressing affordability and we're going to be pursuing the one that Zach talked about about people really moving between online and offline and also around awareness because we know that's a huge barrier. I would love to see us address that or see some of you guys or our editing communities try to address that and think about how to do that but in terms of prioritization that's not where we're focusing our efforts right now. Hopefully someday. Thanks for the excellent presentation. I'm really excited about the results and particularly there's a one part that really resonates with me and that's the set of findings around the brand awareness. I really like how Zach articulated the fact that brand awareness is just more than just it's really about understanding evidence of information and making distinction between an aggregator or a search engine and actual source and understanding the quality processes that are behind creation of that information. So I like to hear it all so you have both things like product strategies and communication strategies around brand awareness at a super important angle. Specifically as we try and find just more readers some more stand up to use. Yeah Dario we're thank you. Brand awareness and just awareness of what Wikipedia is and how to use it I think is something that's very important for a new or potential reader right if we have a chance to explain ourselves how do we do that how do we do it short how do we do it in repetition what do we want people to understand about our movement our content and our projects so the short answer is that we are going to begin doing some message workshops to focus on simple messages to define Wikipedia and we want to do those both within the foundation and within these communities India and Nigeria particularly because we think that the folks in Mexico the strength of that community mean that we know they're already working on things there so we're going to begin some messaging workshops and then we want to begin testing a few different channel approaches to raising awareness in those communities it's something that we want to be concepting actually in the next three months so for everyone here today please feel free to email me or shout at me on my talk page if you want to be involved in those messaging workshops I think that our opportunity here is pretty massive right like when folks arrive at the Wikipedia pages on mobile we've actually noticed that there is almost no branding there's no word Wikipedia right now there's not even a W you simply have I think it's the Minerva skin which is very minimal and does not say where this content is so again if you're a user and you're searching on Google which is like well branded and then you arrive at a piece of content that has very streamlined text as our displays do you might not realize where you are or what the values are so there's an opportunity to message there you are on Wikipedia this is what Wikipedia is and why we do the things we do then there's very much an opportunity with press and with social media Jeff and Aubrey on that team to go ahead and expand there as Jeff Elder would tell you we have is it over a million on Facebook from India okay that's amazing I'm going to say it again so we have 1.5 million of our Facebook fans are in India 200k are from Mexico and 25k are from Nigeria and so those are other ways we want to begin to build advocates who understand the movement who already we know care about the movement and can go further and we hope to experiment even more with that so more to come yes Adele real real quick we are we're adding we're adding them branding to the mobile experience this quarter so we'll shoot you the mocks and to make it even more systemic partnerships is a huge part on that too because then when we redesign and have the branding conversations we can true partnerships take that to local partnerships and local partners that will build more advocacy and word of mouth and all that we need so we have being like with the four years of Wikipedia zero we have been coming back and saying people in the country's we are operating don't know Wikipedia and often times it was hard to have that message coming through so that's why we're so excited to have all this data and and this project working now and have all this attention awareness is real and I'm really glad that we're all together trying to tackle that hello all right we have another one from our C from Tillman regarding the results about the importance of visual content and the popularity of YouTube can we conclude that Wikipedia usage is likely to be held back by our lack of video content talk on this though we did notice in the field we did notice in the field the finding about visual content is very attractive and people seek it I'm not sure I think that would be making that if we because we have lack of video that I'm struggling here a little bit but I think we could do Toby okay I think we could investigate it further I think it's like pretty much across the board that people are seeking this kind of information and so as a researcher I hesitate to say yes but I think it's a good hypothesis that we should have could investigate yeah well yeah so visual content can communicate concepts and ideas a lot more quickly and there's less barriers to getting that information for example one person in Mexico he's a mechanic or he's an electrical engineer and he speaks a very little bit of English but he went to Wikipedia to find a diagram about some part of a I'm not an engineer a resistor or something that he needed to see diagrammatically and that information was rapidly delivered to him and he saved that and referred back to it as a way to communicate that to learn it himself but then also to share it with the concept of someone so I think can I have a question I have two questions what hierarchy of languages meant so maybe is that how languages were chosen to do their research you're down do you want to answer that the languages were chosen by popularity a number of people who spoke them across the country and also to make sure that we got the regional coverage that we needed so on the meta page there's a breakdown of how many people chose which languages for the India survey also how were the cities chosen to perform the interviews I was curious to know for example why Bangalore was not chosen in India so to repeat the question cities the cities were selected based on language and density and the overlap with the type of research participant we were looking for so in Nigeria we knew that we were looking for someone who had access to the internet of some kind we actually would remove people who had no access to the internet from our research and that meant focusing on more wired cities and then we knew we wanted to have I think it was about 40% students is that right again we had to go to urban areas that had high density of students so in and the literacy as well so in Nigeria that ended up being Lagos which is an incredibly large city estimated between 18 to 21 million people living there and then semi-urban what we called Periurban which would be Epe and Benin City in India picking regions was very challenging and it's one of the reasons why we wanted to work with our community to try to prioritize languages particularly because we had the constraint of the same time window two weeks to do the research preventing us from being in multiple cities across India so I believe that we selected the cities of Delhi, Jaipur and Chennai based on language because we wanted to focus on Hindi and Tamil so we would define people who were local researchers who had the skills of researchers who are researchers and could speak the languages we were looking to do the research in so that also sometimes informed this is another time where I would like to say we should do more research and we should go back to India and we should learn more about India because it's big and amazing and there's lots to know thank you and also thank you so much for sharing this it was super inspiring I have another question from IRC from Mike1901 who asks has the team done any research into the possibility of automated tools to guide people through the interface of Wikipedia's where the Wikipedia concerned may take a second or third language maybe a second or third language to that individual but he wants to view or contribute pages of interest in the destination Wikipedia I think it's a content translation related question this research was for learning about what people do now learning about their barriers and their constraints motivations and goals so we were really keeping solutions when we started to think about solutions we would put them in a parking lot and now we're kind of thinking about solutions so that sounds like a very interesting idea and it sounds more complicated than I can understand at the very moment but I think it would be great to hear more about and he can connect with us on our talk page on Meta and Anna has something I found the behaviors around moving between languages really fascinating and one of the things that we learned particularly in Mexico in Mexico as you guys saw Wikipedia awareness was higher we ended up speaking to a lot of students too so they were in this very academic space and people were often finding content in English even when they weren't really English proficient and doing this really fascinating spot translation behavior so they'd find an article in Spanish or maybe there wouldn't be one so they'd go to the English one and if they had rudimentary English skills read through it and piece it together using Google Translate or different translation apps to like a printed dictionary to look up words in English and what they meant in Spanish I think there's a lot of potential sort of like product opportunities things that we could do there to help people sort of like deal with those workarounds but as I mentioned before that's just not where we've decided to prioritize our efforts right now I would love to see somebody from our communities or from our staff like take that and run with it I would be so happy to like spend my time sort of like helping with that too but it's not going to be a focus of the next few months of this particular effort for us that doesn't mean we won't do it at some point though because there's a lot of findings and a lot of things we just have to prioritize to address some things and see how we do and then we can make a decision to maybe address something like that in the next round talking about web maker I'm going to struggle to find the context here but Mozilla did some research recently with some people on the ground around the world about helping people understand how to make that the internet and like pages on the internet or something that are modifiable it's a was about how they arrived at web maker and I'm not really sure what point you're after here sorry Ann the point was just that there is some research out there about contributions to online knowledge in some of the countries that we visited and I don't know if it explicitly addresses translation which as Ann pointed out is super interesting but if you'd like to learn more about it it's certainly out there and we've talked to those people a couple of times so there's a few questions we have someone raising their hand I'm going to read a few questions that are on the YouTube channel first and then we can go to blue jeans and if there's anything in the room feel free to queue up at the yellow microphone so first we have Suyash and Sony on YouTube and Sony asks a follow up question to visual content he writes clearly visual content is more attractive than walls of text given that a good number of people consider Wikipedia's current layout of displaying information to be archaic for instance see the popularity of apps like wiki wand when and if we can expect to see an overhaul of how content is displayed on Wikipedia in my personal opinion Sony continues giving more focus to info boxes and pictures might be much better than current ways of displaying content both on mobile and desktop this is a truly interesting question and you might not be surprised to find out on the reading team we've thought about it quite a lot and I can't speak to like when the big redesign is coming because the big redesign is not coming and I think one of the things that we are navigating in the reading team is the fact that it's not our content and the reading team really doesn't get to say here's what it's going to look like and it's much more of a process of working with the community and working with the editors who are building the content and leveraging research like what the new readers team did to come up with a more relevant and more modern perhaps actionable way of laying out the content so I think it's a great question it's certainly something we're thinking about it's not the kind of problem that a big bang type of solution particularly coming from the WMF is appropriate also we are working to be as collaborative as possible with people so if you want to participate in the solutions that we're building talk with us get on the talk page share your ideas your thoughts we want to have feedback on things as we move forward we're making more space even for that kind of interaction it's not just we're not siloing ourselves we're expanding more see I think is that a Vahid on Lujins do you want to chime in? I wanted to comment on your slide on education and the use of Wikipedia in education it matches very closely whatever experience here in Ecuador South America so I would say that it's probably a general idea in emerging countries that yeah there is some sort of tension between what students thinks of Wikipedia and what educators think of Wikipedia so it was great to see that your research matches closely what I found on the field here my question though was actually about the use of apps of phones in use it I got the idea from your slide that's a minority of phones in usage now where dumb phones and most of the phones in use where smart phones did I get that right? Yes I think there's a slightly more phones are capable of using the internet but that said we saw some pretty junky smart phones right I mean phones that were crack screens are held together with rubber bands and it does vary quite a bit between countries I think like some place like the surveys we conducted outside the study like Egypt we saw maybe in the 40% range or definitely not the majority in some places are still feature phones so it's not universal but these three countries have over half smart phones and we did when we asked the question it was to mobile phone users Yeah and you alluded to the kind of cheap smart phones so there's the phones that are made really inexpensively and the parts are soldered together and they're not as repairable as the ones that are a little bit more expensive in their modular design so that things can be switched out by the person that's going to fix it so if someone like the cost of maybe a feature phone or a dumb phone could be compared to the cost of a cheap smart phone the length of the life of that dumb phone is probably longer it's got a bigger battery length life so what we, I remember seeing in Nigeria was that there were a lot of really cheap smart phones that were just broken and they also might have a feature phone because that one's reliable and that one's going to get them on the phone or do a text message but they maybe can't get online right now because their cheap smart phone is broken so there's a lot of play back and forth about which phone, there's a lot of reasoning about it. Vihid does that answer your question? That's fine. Thank you very much. Thank you. Another comment actually slash question from the YouTube live is from Suyash which is, let me see if I can just get to the right part of it here but are there any special programs we can do in Hindi speaking states for students so that Wikipedia can become familiar for those students that they may start reading and contributing so I'm both I guess going to ask the question and then answer it. So outreach programs are something that I think we want to look at for awareness and for participation but it's something that we would be partnering with the community as well as community engagement so local community engagement communications I think would be a great solution and something we definitely intend to explore. Do you want to say something about that? I was thinking when I saw this data one idea that I had was how can we make a workshop to connect this data to possible activities that communities can do locally to help understand how to use the data and work with affiliates for that but we can talk more about that later. So in India we spoke with a few educational entrepreneurs noticing how students are not as inspired by traditional educational content they're building content as a business or as a non-profit so there were a couple non educator entrepreneurs who were building content creating content and that seems like a not in partnerships team but I think it seems like a natural partnership to connect that content with those entrepreneurs. So we spoke with a few in India I'm sure there are more. Yep that's right Jorge probably can say more about this but Global Reach does great partnerships in this space. Yeah I simply just want to add up that the Global Reach team before and now with this findings are actually finding actionable opportunities where we can do more and better outreach. What was a simple example of this is how we're trying to work in the digital India project and how we can incorporate Wikipedia as relevant content for those people that are being connected through it that's just one of the many things that we're working on so definitely we're taking all of this research and all of this findings to find partnership opportunities where we can bring Wikipedia to more people. Okay I think this is a good time for us to show you our final thoughts including personas and next steps. Okay so personas are designed to really they're a method for people who are building something for someone else maybe not in your context to keep that person in mind and a persona is an archetype so an archetype formed from the same needs, behaviors, goals, context, constraints and so we from talking with 70 people we noticed some patterns in Nigeria we formed four personas which you can find on the new readers page on meta to look at more detail. We didn't have time to go into all the detail but so we have and we have three personas from India and we're also working on one from Mexico we also have a series of pragmatic personas about editors and the community the people who contribute content and so they're never done we're always learning more we're always evolving them but it is definitely a way for us if we're sitting here in San Francisco building things and around the world to keep our focus on who we're building solutions for and who we need to keep in consideration. So I'll any questions about that but I'll pass on to Anne for next steps. Yeah so we talked about this a little bit already and Zach mentioned this as well and Abby mentioned this like we're really interested in being as collaborative as possible on this there are five teams actively involved in this research and kind of move forward on what we can do with it but we'd love to partner with Maria to help with advocacy or anyone else who's got these ideas as community members like we want to hear from you and work with you. In terms of the sort of priorities that we've outlined as I mentioned before we're working from these four findings for now and that's not to say that the others aren't important it's just life is about prioritization we can only do so much at once and we want to really make sure to address these things really intentionally and iteratively to make sure that we're doing a good job and we're having impact the way that we want to and just to review that again we're going to be looking into how people are increasingly getting information online and consuming or sharing it offline awareness that as a brand Wikipedia is not widely recognized or understood and then affordability and we've got a number of findings that relate back to that as well as some sort of like desk research as well so that's where we're going to be but again if you want to work on other stuff you have questions let us know we're eager to collaborate. In terms of our sort of immediate next steps we'll feed back here the sort of from the product perspective which I can speak to the most because that's my job we're in the process right now of prototyping based on concepts around these specific findings we're focusing first on the online offline finding that's number 20 so we're going to be building some prototypes and testing those in the field both hopefully with editors and community members and also with readers and not yet readers on the ground in the countries where we learned these things from and that's going to be kind of over that through the rest of the calendar year in fact do you want to speak about? So this goes back to what Daria was asking around how can we help raise awareness for Wikipedia how can we help better explain it over the next several weeks we hope to have some message workshops that will be focused on better doing that simplifying our explanations finding where we can go to advocate for our movement and we'd like to make this again very collaborative I had the good fortune of recently being at Wiki conference in India where this came up a lot so Ravi had mentioned a distinct need to potentially work on awareness materials we are interested in how we can produce videos that explain Wikipedia and post those digital and social channels for distribution there's a lot of ways that we want to tackle this so I'm going to be starting a series of workshops and again you can email me zmcune at wikimedia.org or find me on my talk page Zmcune parentheses WMF to get added to those and don't worry I'll also probably harass you on public mailing lists to join so if you don't want to opt in I'll also chase and that's really everything we have and do you have some final thoughts? I think we've talked about this but this has been an effort from between five teams and just a ton of people have put into this so thank you all of you who have been involved you know who you are thanks especially also to Blanca for spending a bunch of time on these slides helping us make them pretty it's been really really amazing working across the organization and we're hopeful that we can go forward and have real impact by continuing to work in this way and Dario has a thing I just wanted to make a final note well thanks for all the teams involved a final note about that and the data and the research materials we're working this mostly for researchers and community members we're working to make all the raw data and transcripts obviously after taking care of personal information work to make them available to our communities and researchers under open licenses consistently with our open access policy and so I think that beyond what you hear here there's also way more opportunities for engaging with these results and delving into the materials themselves and if you want to know where to talk about this project please do so here we're on meta we're active this is the best place to be involved in the discussion on the talk page this is also where you can find all the things we've prepared linked including this presentation which Joe Sutherland has just uploaded and this video which you may be watching sometime in the future and we'll be linked there as well thanks so much