 All right, welcome, 2014 December Metrics meeting. Happy holidays, it's good to have you all here. We have a lot of new hires, so I would like to welcome Jacob and Juliet and Jennifer and James and Grace and Stas and Nerzar and Pratik, so please make a point of finding them. We also have new contractors, interns and volunteers, so we'd also like to welcome Sandra, Jerry, Megan, Maria, Samuel, Sabrina, Valerie, Wes, Niharika, Tracy, Amanda, and Sameer. Thank you for joining us. Moving on, we have a bunch of anniversaries. I particularly want to call out Megan for five years, and it's good to celebrate the time that folks have had with us. I also have one more little tiny thing to sneak in. Our general counsel, Jeff Brigham, just got awarded most innovative general counsel by the Financial Times, so he's not here in person and embarrassed him, but I think that's a fabulous thing, and that's after our own Garfield Bird got various year above the year, so we're doing pretty well in some of those categories. All right, moving on. We're doing a new thing where we're announcing some of the milestones that we've achieved on a monthly basis. I'm gonna turn this one over to Eric. Good morning, so, Pages should be a lot faster now, and you should notice that even if you haven't turned on HHVM yet because it's now turned on for everyone. We also rolled out a completely new search engine for all Wikimedia Wikis that should make it a lot easier for our developers on mobile and desktop to actually build search into various types of applications, use it in new ways, and we also have lots of new user-facing capabilities. We have committed to a whole bunch of changes for media viewers, including making the captions visible whenever you look at an image, making it easier to zoom, making it easier to turn off if you don't like it, all of which has gone out. Once we pushed out the zoom feature, the folks on the reader side of the anonymous users actually turned it off a lot less, like we saw massive decrease and the up-down rate for readers, and we have done much more usability research on it, and also validated that there are no usability issues with the current product at this point in time, so people know how to use it. They understand what it does, how to turn it off if they don't like it. So we've got that done. In November, we also pushed out table editing for Visual Editor, which was one of the last reasons you might have to use WikiText when editing a Wikipedia article. When looking at a complex page with a complex table where you wanna add or modify rows or columns, you can all do that in Visual Editor now. And as you know, we moved our bug tracking system over to a new tool called Fabrikator, which was a huge migration with 73,000 tickets, which went painlessly, so thanks to everyone involved in that massive undertaking. And I'm turning it over to Anastasia to talk a little bit about what happened in grant-making last month. Hi, Derek. Hi, so grant-making and sixth floor. We launched last month the first ever Global South Survey. It's also the first ever large-scale survey we've done at readers. In the history of the Foundation, we had 47,000 completed responses from 11 countries and 16 languages. We have a teaser for you later today, but the full-scale analysis will come soon. We had a very important meeting from the Fund Dissemination Committee, which gave recommendations for 11 Wikimedia organizations around the world and funding towards their annual plans. The strong message from the FTC, which is a nine-member community team from around the Wikimedia world, is that large Wikimedia organizations should be showing better impact. In their recommendations, they reduced the funding by 14% from last year. We did the first ever Wikimedia survey to make sure that as we think about events worldwide that we run, we are designing them well and effectively. We had a 52% response rate and 85% said they would do a new project with someone they met at Wikimedia. And finally, in terms of partnerships, this is an important one for all of us. The Wikipedia Library has Elsevier joining. It's a small pilot and a big symbol for those of you who know and use Elsevier in your editing work. It's one of the largest academic publishers, especially in medical and scientific literature. Thanks to everyone who made this happen. So you may remember that last month we tried something new, which was to actually have a theme for each of these meetings. And the theme for this month is going to be all about readership. And we have a series of presentations just on that topic, which I'm going to pull up now. I'll just click on this guy here. All right. Now. Oh. You know, Chrome actually just is all the time now. Even when it has no problem. Interesting. Yes. I wanted to start just talking a little bit about who our readers are before we go and get into the middle of the presentation. And you may have seen some of the comments that always come in from the fundraiser. And of course, now that we have a fundraiser, we get a lot of those coming in right now. This is some of the themes that we've also seen in, ah! This is incredibly frustrating. Let's see. Yeah. These are some of the themes that I'm encountering if I'm looking at comments that are coming in from our readers. Like one common theme that I'm seeing in a lot of these reader submissions is that folks are finding Wikipedia to be a tool that breaks isolation, that connects them to the world at large and that helps them discover and understand the world. And this is going to just keep going, isn't it? Yeah, we should get someone to stand here and press wait. Okay, so the other common theme that we're seeing in a lot of these comments is that people really love this idea that they can explore like any topic. And when they go to Wikipedia, they can click around. They can discover very, very complex relationships between the subjects that they're exploring. And then of course we're seeing comments like this, slam my arrogant stockbroker brother-in-law when we were arguing over which two countries had the larger GDP. So that's also a theme. Readers love to use Wikipedia to win barbats or to impress their friends. But then every now and then we also hear an extraordinary story from our readers. And one of those I want to show you and to push the limit on technology, we're going to actually try to play a video in a metrics meeting. This looks increasingly fun. So continue, accept. Yes, you can have all you want, Google. A teenager who has been using Wikipedia to develop cancer prevention and detection methods. And these are the kinds of stories that you find once in a while out of like thousands of stories. And when you find them, you really want to tell them and document them. And I think Jack actually also spoke at Wikimedia and told his story there. So this is pretty amazing stuff. So what do we actually know about our readers as a group? Like what do we know about the whole of Wikipedia's readership? And to the extent that we know things that reflect changes in the world around us, what are we going to do about it? So that's the core of this presentation. So what we're going to go through is an update on the traffic trends. We're going to talk a little bit about how we can measure the reader experience on an ongoing basis using design research. We'll talk a little bit about what we're doing on the product side. And Carol and Suya are going to talk about how we reach the next billion users. So with that, I'm going to hand it over to Toby for an update on our traffic trends. Thank you, Eric. You can use this in theory. Theory. Awesome. So hi, everybody. Good morning. So I'm going to start out with some numbers. 248 billion articles that were served in the last year from October to October. That's a pretty significant number. It comes out to 34 articles per human on Earth. It's obviously, it's a little bit of a vanity metric, right? Not every person is reading 34 articles. But we're reaching people in 235 countries slash regions. I did look this up. This comes from our geolocation software. There are only 206 countries. So there are some regions in here too. Slightly, maybe slightly nervous, but. Read the documentation. So these numbers are amazing. And I think they really speak to the significance of our work in the foundation and above all the work of the community. Like this is awesome. And I'm really excited to be part of it. So hold these. But like Eric said, the world is changing and I'm going to talk to you about some of the changes in the world and some of the changes in which people are using, are reading our content. And these changes are changes and changes sometimes scary, but there are some amazing opportunities here that I'm going to highlight and ways for us to push the mission. I'm going to try to also put these in the general context of the internet, like what is actually happening. And I'm going to talk really quickly about how another company has dealt with these challenges. So on to the data. So high level, mobile's growing, desktop traffic is shrinking. Our page views are flat globally. It's down about a percent last year. Global North traffic is definitely flat and that's about two thirds of our traffic. But what's really exciting to me is that global South traffic is increasing and that increase is being driven by mobile. And in the US, page views are definitely declining. And what we're seeing is as users transition to mobile, they're not offsetting, that the increase in mobile is not offsetting the decline that we see on the desktop. And I should say that if you have questions about this, I think we're going to push those to the end of the presentation because there's a lot of good stuff falling. So globally, it's just another lens on the data. The thing to really look at is like, wow, 64% increase in people coming to our mobile site. Like that's something that's really exciting. Quick word on humans and crawlers. I'll touch on this a little bit later, but yeah, we have a lot of automated traffic coming to Wikipedia and this is something that we also need to think about. And for those of you who are more visually attuned, here's the graph. See the desktop site, mobile site. So I think one of the interesting things that we like to do with data is segment it and start slicing and dicing it to get a little bit more insight into what specific groups of people are doing. So we're going to do a little bit of that. 64% increase humans coming to the mobile site in the world. Even the US, more than 40%. Another lens that we like to put on the data because it supports our mission is global north versus global south. And here's an amazing number, right? Like people in the global south are accessing our content through their phones and it's blowing up to use an industry term. But the reason that we're not seeing that in the aggregates is because of this number and this number, right? Like it's only a 10th of the total in the global north. But obviously you can look at this as an opportunity. There are a lot more people here than here. Another picture of the global north, just keep the slope of these lines in your mind, global south, right? So we wanted to sort of, you know, people like league tables and like to see rankings. So we pulled out the top growers controlled for page views. And again, what's cool here is, you know, India, huge country, part of the top 10 or 12, nice month this year, also Russia. And then Iran, I think we have to dig into Iran a little bit more, but that's pretty amazing. And that's probably a country I think that might be opening up a little bit and is really thirsty for knowledge. So that's really cool. Decliners is interesting because it's all clustered or mainly clustered in Latin America. I think we need to dig into this a little bit more. There's nothing that really comes out and that really is striking about this other than the geographies. So that's the change that's happening, right? We're moving from desktop to mobile north to south. But we are not alone. And I wanted to talk about a few of the big internet trends over the last maybe five years and try to help you understand where we are in relation to those trends. So I kind of think of leading trends and following trends and trailing. So I think with mobile and international, we're definitely going to talk a little bit about what we're actually doing in order to leverage these trends. Particularly international, our efforts in Wikipedia Zero are addressing reaching four billion unconnected users. Social, we're really trailing. There's no other way to put it. When I was at Yahoo, fully half of our traffic came via Facebook and Twitter. It's much, much lower here. So it's a big opportunity for us. And then structured data, which isn't exactly like a hot trend. It's sort of been envisioned by the people who created the web as where it needed to be. We're actually leading there with Wikidata. And Catherine wanted me to mention that Wikidata won the Open Data Publisher Prize from the Open Data Institute recently, which is awesome. And one thing to check out if you haven't already is Mary Meeker's amazing internet trend reports. They're here. They're just a really good overview of what's going on and I read them a lot. So finally, case study. So what they've been doing. Yeah, I know, but we're sort of in the same league. We have similar goals, so work with me. So this isn't a great chart because it looks like the mobile monthly active users and the total are pretty much in sync. But what's really happening is this line this segment is growing about 20% a year and this is growing at about 10% a year. And a lot of the reasons why they're close is because Facebook addressed this pretty aggressively. So here's what Facebook did in the face of seeing their world change. I have been, Facebook Zero, which is their version of Wikipedia Zero was actually not launched in 2010. It was launched in 2012. Sadly, the Wikipedia, what? Oh, they launched in 2010. Okay, sorry about that. Right, so they launched their program to reach out to the Unconnected in 2010. In 2012, they bought Instagram, which was a mobile only photo sharing application. 2013, they reorganized their entire engineering organization around delivering mobile products and experiences. 2014, they bought WhatsApp mobile only messaging company that was popular in Asia. So in summary, our world is changing and that can be a little scary, but the opportunities particularly around mobile and international are amazing and I'm really excited to meet those challenges. And we can look to other web properties and see people who've met these challenges and I'm confident we can do that too. So because this is the Wikimedia Foundation quick word around the data, we used the data from the sampled logs along with a new page view definition that we're rolling out. We're gonna be looking at the old numbers versus the new numbers to find the discrepancies. We still gotta work on Unix. We had the app traffic, we didn't break it out because it's pretty small. So that's it, thank you. And next, Jared and Abby. Thanks, Toby. So one thing that the user research and UX group has been working on is something that we're calling Reflex. It's remember the name, Reflex. This is something that I wanted to bring that was kind of a best practice at Autodesk where I was at previously measuring qualitative user experience over time in a quantitative way. Our goal is really to create a metric for usability readiness of our products. Initially, I think this is gonna be used to inform our process and I hope eventually it will actually be used to guide whether we release things or not. So the reason we're doing this is we make a lot of change to the site. A lot of times they're very iterative changes, they're small, people don't really think about them unless they think about them in aggregate. We wanna better understand our user's experience. We wanna focus on places where that experience is changing and could be increasing their productivity or decreasing it. And we wanna evaluate every change we make and how it affects our user's experience of the site. The way we'll do this is we're gonna take a battery of tasks and we're gonna measure those in a qualitative way but report them in a quantitative way. The things we're gonna focus on are the confidence of task completion which is different than actual task completion but it's about people how they see their experience of the site. The ease of use and their enjoyability in doing these tasks. We're gonna have these tasks and then we're gonna roll them up to task group levels and gonna use a standard metric called net promoter score for doing task groups. A task group could be an example of reading and searching on the site, doing simple edit tasks, doing complex edit tasks. Initially we're gonna focus on readership tasks but we're gonna expand this to editing tasks as well. The last part is a sentiment matrix. This allows users to pick from natural language descriptive words but then we can roll that up to an actual number that we can say this is a positive sentiment or a negative sentiment. The last thing is although reflexes primarily qualitative there are a few quantitative measures built in. Success for our failure rate which is just a binary time on task and a click path aggregation which Abby will show a little bit later. So to make sure we measure the, oh thanks, make sure we measure the tasks accurately no matter what, if it's mobile, if it's desktop, if you're using visual editing. Next, oh, so to do this measurement we're gonna use the possibility of one of two tools. We're evaluating right now user zoom and loop 11 and we're gonna do some, a pilot to see how these two tools function in our code and we're gonna do a pilot and understand which of these is best to use and how the outcomes look. So then, yeah, we're gonna implement a tool in a snapshot of our site and then we'll move up to like 100 to 500 users per quarter. And this is kind of how it might look in user zoom. The user will be actually functioning in the site, their question that they're addressing will be under there and then they'll say either success or abandon but there's also validation within the tool if they actually did succeed or not. And this is a click path aggregation that Jared was talking about. We'll be able to see a click path of everyone's movement through our site and all those click paths will be aggregated. So for example, if a whole bunch of people went one way to accomplish the task, some other people went another way to accomplish the task, we'll be able to see that. We'll also be able to see when people fall off and aren't able to accomplish the task. Then we can hone in on looking at those experiences and aggregate and going in and deeply and doing analysis on what actually happened so we can see patterns to then be able to make recommendations of how to improve the experience. And this is one of the ways inside both of these tools you can choose which of the tasks were not successful at some certain rate and then we focus in on those tasks. We can sort and search and find those experiences and aggregate, look at those closely. So for reporting analysis, I'll turn it back to Jared. So the kind of artifacts that we'll produce from this are obviously the videos that Abby just showed with the actual screen overlays, the click paths. This will allow the user research group to actually go in, find places where there's problems and really focus our research on talking to users about these areas. I've prepared a few kind of prototype artifacts for this. One is a task detail card and this will allow someone in design or product to quickly look at this and say, for this task, find the average low temperature in San Francisco in February. They have the reflex number which is a quick roll up. We'll say that we don't ship something without a reflex of X, whatever that ends up being. Time on task, success rate, and then what people think about in the actual group level. This group is about reading, it has an average score of six and a sentiment of seven. We know this thing is not ready to ship and we can go in and dig in on what we need to address to make it ready. Back to the more quantitative things, the success rate, binary, yes, no, we can establish something where we say if there's a success rate less than 90, we don't ship something and this will allow us to look at over quarter over quarter, year over year, test session or test session, how these numbers are changing. Same with reflex. Reflex is more about the quantitative measure, the success, whether people feel about something, we can see how that will change over time as we make changes to the product. Thanks guys. Clicker, thanks. So, we wanna talk a little bit about how we're responding to the trends in mobile that Toby just talked about. And so on the product side, we actually have been thinking about readership now for some time. This is not something that we just started talking about last week. We've had multiple conversations about what's happening and what the priorities should be between like growing contributorship on mobile, growing readership on mobile. And the way we think about mobile right now is that we look at the apps as a great playground for ideas. So the native apps, the iOS and the Android apps only have about 1% of our total traffic. So most of our mobile traffic is actually by far on the mobile web. More than 99% of our mobile traffic is on the mobile web. But the apps give us a lot of capabilities. Their native code, we can evolve very quickly. They're not tied into MediaRiki, our core application. They're completely independent, so we can try out new things, tested, measure if it works, because apps are very cleanly sort of isolated. It's very easy to measure things like session length and unique user behavior. So it's a great, great, great environment for experimenting with completely new ideas, radical readership experiments, and seeing if we can increase usage on mobile. And then some of those ideas may make sense to actually launch then on the mobile web as well. There will be things that will start on the mobile web for readers, but there's going to be a lot of stuff that we will pilot first within the app plan. And Mariana is going to talk a little bit about some of the things we're currently thinking about. Actually, I think I have one more side that I just wanted to show, sorry. So we can go. Bragging rights right here. Yeah, so I just wanted to highlight one trend that we are starting to see in social media that I think is super interesting, which is that readers are switching and using the mobile side as their primary experience increasingly, and they're commenting on what a great experience it is if you're a product manager, these wow comments are sort of crack cocaine for you. It's like, wow, the mobile vision is great. Wow, wow, wow. This is pretty amazing feedback. And people love the work that we've done on mobile already and they're actively taking the initiative to switch over. Oh, crack cocaine, that's a strong metaphor. All right. So let's talk about what our readers are actually doing on mobile today to start this conversation. So if you take a look at some of these examples, these are actual examples taken from the mobile teams reading history in the Wikipedia app. So these fall into two basic categories, right? We all go to Wikipedia to look up a fact really quickly on the go. We also sometimes go to Wikipedia to learn more about a topic and you can see examples of each of those. What is norm core? What are these kids doing today? And if you look at the actual quantitative numbers here, they're very interesting, right? So on the left, you're seeing the number of times that the average user is opening up the Wikipedia app per month. And on the right, you're seeing the industry average of the number of times the average app user is opening up the average app per month. Now what you can see is that in Toby's leading following trailing metaphor, we're kind of in the following to trailing side of things right now in terms of our engagement. So the industry average is 13.7 sessions and we're at 4.1. And this of course includes apps like Facebook and gaming apps, which are incredibly addictive and are actual crack cocaine for your brain, which you will open multiple times a day because you're obsessed with them, right? And we're not like that, right? So it's kind of an unfair comparison. But in another way of thinking about it, actually what's more important for you to do every single day to learn about the world or to play Farmville, right? We need to do more to actually be the Farmville of knowledge for the world. So, seriously, right? So how do we actually do that, right? Like we've taken the Wikipedia experience and we've shrunken it down. We're done, right? It's small and handheld. Well, not quite, right? We need to do more. So some possible approaches to reader engagement to try to up these numbers are the following, right? We can do one thing. We can take the quick lookup people who are just coming to Wikipedia to learn about a fact and get them to learn a little bit more right on that page, right? We can also give people a reason to come back that's not just, oh, I want to find out what year that movie was made. We can make sharing easier so that people can actually share our content with their friends. We can give people tools to browse and discover our content, not just have to type in the title of an article in a search bar. And we can actually make the content nice to look at, which is kind of good. And the whole point of all of this is we're trying to get Wikipedia to be something a little bit different from it. What it is in the minds of readers today, which is, well, I go to Google and I look up this fact and it takes me to this thing called Wikipedia and I read about it there, right? That's not what we want. What we want is for people to go to Wikipedia, specifically to learn, to be immersed in our content, to get pleasure and joy and knowledge out of it, right? So these are some concrete examples of things that are actually happening on mobile today to try to address some of these feature buckets. And these are all ideas that we're targeting to try and see if we can actually move these numbers. So the first thing, pretty simple, basic idea, right? You come to Wikipedia article, let's say it's the Darjeeling Limited and you wanna find out what year that movie was released. But while you're there, perhaps you would also be interested in reading about the Royal Tenenbaums, another movie that Wes Anderson made, or the screenwriter who wrote the screenplay for the Darjeeling Limited. Or maybe you wanna find out what the hell a screenwriter actually does. I don't actually know what the screenwriter is. These are just some examples of related content that we can show you right on that page so you can go and visit related articles and read a little bit more, learn a little bit more about the topic you're looking at. A reason to return to Wikipedia beyond just searching for stuff that you already know you wanna find out about. We know that things happen in the world and those things are reflected on Wikipedia in the form of trending articles. So we see that when an article suddenly gets a lot of edits or suddenly there's a spike in page views. Usually when somebody dies. And what we can do is actually start surfacing that information a little bit more actively to people. So we can even use notifications and things like apps to alert people to the fact that hey, like there was this earthquake that happened. Maybe you wanna read about that and you're located near that general area. Maybe it's kind of relevant to you. Another thing that we're particularly bad at as Toby mentioned is sharing, right? So you're having an argument with a friend and you're trying to prove a fact to them. What do you do? You send them the entire Wikipedia article. I mean, that's like sending them a giant manual to read through, right? It's not gonna work. What you actually want is to be able to pull out a fact and send it to your friend to say, hey, Wikipedia proves you wrong sucker. And that's exactly what we're doing. So we're actually building a feature to allow you to pull out information from a long, long article and just send it in the form of one little blurb with a reference obviously in actual linked Wikipedia. And finally, browsing and discovery, right? We've got bajillions of articles. I don't know how many Toby said, lots. There's lots of articles on Wikipedia. We could try to go through and categorize all of those articles by hand to put them into high level categories so you can browse them like Pinterest, right? But that would take us approximately 16.7 years. I just made that up. What we could do instead is something that we're talking about is letting readers create those categories for us and for other readers. So you have a list of books that you like, right? You start collecting that list on Wikipedia. You share that list with other Wikipedia readers. All of a sudden, you're sharing knowledge. You're letting other people discover the stuff that you like. And boom, problem solved maybe. Ooh, beautiful content is obscuring, beautiful content. Well, if you go to the mobile site today, what you'll see a lot of the time is a big block of disambiguation text and page issues. And only then maybe do you get the lead section of the article that you're looking at, and maybe then you get an image, maybe. What we're trying to do is change that, make the information more visually appealing. So in the Wikipedia app, actually, you'll soon start to see an experience that's like this one, where instead of having to wade through a big block of text and info boxes and templates to finally get to your content, the first thing you see is an image and a Wikipedia descriptor that tells you what that content is. So you can actually enjoy reading Wikipedia rather than struggle and fight through the interface to try to find your information. And so how do we actually measure that any of these things that we're doing currently are working? Well, we look at some numbers, the number of people who read Wikipedia, the number of times they come back per month and how many articles they're reading every day. And the point of all of this is really to make Wikipedia a part of everybody's daily life, to make knowledge a part of everybody's daily life. As I glibly mentioned earlier, we are the Farmville of Knowledge. We should be the Farmville of Knowledge. People should be coming to Wikipedia every day to enrich their lives. And that's the challenge that we have on mobile and it's also the opportunity that we have on mobile. So yeah, that's it for me. Thank you, Mariana. Okay, hello. So it was exciting to see Toby's numbers that a lot of our growth is coming from mobile in the global south. What we do know, though, is that billions of people are still not connected and so they don't yet have access to knowledge. And that's why we do Wikipedia Zero. McKinsey did a study recently about the barriers to connectivity. So basically they have categorized the factors in incentives that's basically giving people a reason to come online, affordability and low incomes, user capability and also infrastructure. So this is an interesting report I would recommend. We are focused on three areas that we can address directly and that would be the cost of data plans, awareness of content and also providing local language content. So Wikipedia Zero sits squarely directed at the affordability barrier to access. We are now up to 41 operator partners across 34 countries. In the last few months, we have launched in Myanmar, in Ukraine and just recently in Morocco. We know, though, that just waiving the data charges isn't enough to get people using Wikipedia. We have to make people aware of the fact that the resource is available to them and that it's free. And so I wanted to share with you a couple of examples of how our partner's awareness campaigns are actually driving the growth of Wikipedia as a whole, Wikipedia usage. So this is from Nepal. We launched with Encel back in May. They did a lot of like big advertising campaign of beautiful billboards about, not just Wikipedia, what it's about about knowledge, but also that it's free. And they more than doubled the mobile page views in the entire country, not just their page views. We, that went up to over 4.5 million, almost 5 million. And it's been sustained in the country. And we can see now that mobile usage in Nepal is higher than desktop usage. Similarly, in the Philippines, our partner SMART did a huge campaign for us in June. And they also drove total usage in the country, total mobile usage by almost 50%. Again, that we've been able to sustain that level of usage. What we'd like to see is that we see, start seeing more organic growth, not just relying on the advertising campaigns, right? We know that we can't always get partners to do big ad campaigns and we need to find other ways to raise awareness. So the Wikipedia Zero team is working to find other partnerships that would help us drive awareness. And we're working with the communications team and the grant making team to refine our messaging so that it'll be more compelling to new audiences. So the last focus area for us is local language content. And this is what we do, right? Like, so much of WMF activities is in support of our local communities who are writing local language content. But the problem is getting this content to the people who don't even know it's there. And there are a lot of reasons why it's hard to reach people in their local language. I wanted to just show you a snapshot of what's going on in India. So we know that there are, like, more than a dozen major Indic languages that people speak. And only about 10% of Indians speak English and those would tend to be the privileged classes, right? So when we look at our page views in India, though, we're delivering mostly English page views. That indicates to us that we're not yet reaching the people that Wikipedia Zero is trying to help. And it's not just us. The whole mobile industry is aware that in India, the people that they're trying to bring online are the people who don't speak English and they're the ones who need content in their local language. And so in that regard, Wikipedia can be a really important part of the solution. We are doing some things on the product side to try to address this, some simple things. Just this week, Adam Basso released some code that would detect the default language on the handset and automatically redirect the user to the right language version of Wikipedia. Yes, yay. Now, okay, so we're starting to see, you know, right away we can see some impact from that. And it's not gonna drive page views on mass yet because many handset, or there aren't many handsets in the market that have regional language defaults. People use their mobile phones in major languages for the most part, but that's a trend. I mean, it's certainly in India, a lot of the second tier OEMs are delivering or are shipping handsets with Indic language defaults. So we think as more people come online, this will become more important for them. And I would just encourage everybody to think about ways that we can improve the language UX because this is gonna be increasingly important as people come online. So thanks. Thank you, Carolyn. Thanks, Carolyn. So glad to be representing the 1.2 billion Indians who we're all talking about, yay. But as we think about the future, I think it's really important for us to think about who we know of the Global South readers and contributors right now, the 30% that Toby talked about. So we just launched and completed, as I said before, a Global South survey of readers and contributors, which is the largest ever survey we've done of the kind. And thanks to everybody who did the survey. So that's a slightly old slide, but forget the text. Most importantly, in terms of what we did, we surveyed in 11 countries and 16 languages. India, of course, being one of them, I think we did eight languages, including English in India. And we had 96,000 responses with a dropout rate of 51%. And so 47,000 completed responses, kind of, hopefully, statistically significant. At the same time, caveat, caveat, caveat. This is very preliminary data. We'll be digging into it much more over time and we'll come back to you with a much greater sense of analysis soon. We ran the survey on desktop and mobile as well. So in terms of access devices for readers, this is all reader-related material right now. Obviously, multiple responses were possible. Again, as we're talking about, there's a strong, strong trend to mobile. Smartphone usage is 66%. Interestingly, for our current readers, we're still at a low 5% on feature phones. At the same time, people are still using 55% and 40% laptop and desktop. This is interesting and really important new data. Again, we'll be digging into it much more mixed bag of news. In terms of readers, we have a gender gap of about one is to four, right? 20%, 21% identify as female of our readers. Now that is significantly lower than what we think overall on an average worldwide, which is about 50 or more percent of readers are female or identify as female. Now, the good news is we equally think that we have about one is to 10 contributors being identified as female across the world. In the global south of the respondents we have, we have one is to four. Now, that's, again, a significantly interesting data point and we'll dig into more and understand it better, but that's an opportunity for us. Now, here's another opportunity and it comes out of some of what Carolyn has just said. As Wikimedia Projects, Wikipedia and Sister Projects, we have a tremendous opportunity and advantage, first mover advantage of content in local languages. And so it's really interesting to look at who's reading what from our respondents. Wikipedia, obviously, but 22% are reading Wictionary and using Wictionary as well as some of our other key Sister Projects. And that might be something we want to dig into further in terms of entry points, motivations around Sister Projects. Many of our contributors, for instance, are starting with Wictionary and Wikisource because it's around issues of language pride and making sure this content in those languages equally readers are obviously wanting to access that. The last thing I wanted to leave with you is that we wanted to check on reach in terms of offline Wikipedia as well. And many different versions of this, distributions, some of you, of course, know Kivics as the best known. Now, there's a mixed bag here as well. No, we've never heard of it and no, we've never used it, but we've heard of it and about 10% who have used it. And this will be interesting for us to think about in terms of reach as we go forward, is it the Wikipedia zero model that is the most efficacious versus the offline distribution model? Or, for instance, are we just talking about the current readers who are still relatively privileged as much of our movement worldwide? 75% of them access the internet at home. So our next billion users are not necessarily represented by our current readers and users. So again, really interesting and important data points for us, digging into it more and we'll come back to you with much more soon. Thanks all, and thanks particularly to Haitham and Asaf for leading this on our team and everyone across the organization who helped with this. Thank you, Anise. So what do I want you to take away from all of this? Like, what are the sort of main themes from all of this? Like, the biggest thing that I think we need to take away from this is that if we're thinking about our readers and if we're thinking about a growth of readership, then the next billion readers are going to be in the global south and they're going to be on mobile. Like, there's no question about that. And as we pointed out, organizations need to retool themselves and rethink how they do their work when they meet with challenges like that. So we have to do even more than we already have to get ready in all our products, in all our work. Not just engineering work to really sit and think about these two challenges on the global south and mobile. I wish I could have shown you more, for example, on the product side about specific things that we're doing to make sure that our mobile experience works well on the lower end devices that are more common in the global south, or showing you data on performance and bandwidth considerations that apply when you're accessing the internet in developing countries. We haven't done as much here as we should. We have done some things like improving image compression, but there's still a lot more that we need to do to make sure that we meet the specific contextual challenges of reaching readers in the global south. But at the same time, this trend isn't hitting us as a surprise. We have been working in the last couple of years very, very, very much on making sure that we actually can grow with the mobile trend. That is why we are seeing the numbers that we're seeing in mobile. That's not random. That's the result of us actually building a pretty awesome mobile experience already. But now we have to double down on that and really respond to the growth opportunities that we have. So with that said, in general, we want to make sure that we have room in these meetings for a bit of a larger conversation. So I want to turn it over to you both for questions, as well as also comments and observations that we have. And I want to turn also over to the presenters and the senior team to respond to questions and comments. I've got a quick question here. Mike, Mike, Mike. OK, a quick question regarding what you were just mentioning. In the global south, I've seen papers showing our page load performance being slower through RIPE and some work that Faden did on the Ops team. It seems logical to me that a cache pop located closer to our users in the global south would be beneficial. I haven't seen that in any roadmaps for our buildouts. Do you have any insight on what we might be thinking about for that? Yeah, I know it's absolutely something that we need to do more of. So we have a caching center in Europe right now, which actually speaks to some of our sort of historical bias of serving readers in specific locations with faster access to the site. One thing that we are doing right now is we're building out a new data central location in the United States. And part of this process is actually automating a lot of this to make it really easy when we decide that we want to have a new caching center location or a new DC location to not have this be like a one-year process, but actually have it be like a caching center in a box that you can decide to set up somewhere. And that's definitely something that we need to think about on the infrastructure side. Hi, a question from Aaron. Asking Marianna that I guess Eric might be able to answer is do we know that people are struggling to find related content and how so, and aren't links like what this is good for? Marianna, do you want to take this one? What did he say I couldn't hear? I said how and do we know that people are actually struggling to find related content and other stuff? So yes, I think Aaron is correct in the sense that blue links currently are the only way that people can find related content. But if you look at any other mobile product that gets people to read things, they don't just rely on a system of links. They rely on surfacing related content in various ways. And I think we may not have exact metrics around how difficult is it for people to find related content. But we do see those session numbers, and they are quite low. And we would expect that they would be much higher if people were actually able to serendipitously find and discover things that are in the area of knowledge that they care about. So I think the problem exists, and we know it exists. We're trying to address it with more. Thank you. Other comments, questions, thoughts? Anything on ISE? Question for Anasuya, can you talk a little bit about how we measure gender gap? Like is it surveys? Or how do surveys even reach people? Or what is the process of collecting gender data? Great question, and one of the things that we've been digging into for a while. At this moment in time, this is one of the few surveys I know which actually have this large sample size in which people have self-identified. In most cases, there have been very sophisticated analysis done, which is projections of what we assume is our female contributor or reader rate. And I think Toby can talk more about readership and gender, because he has some of the latest data on that. But in this case, this is self-identifying in the demographic section of this survey. Do I identify as female or not? OK. Toby, do you want to talk about readership and gender in general, because I know you have. Where is Toby? Right there you are. I don't know of the data that you're speaking of. I mean, as far as I know, we actually track very little about our readers, and as a result, know very little about them. We do like editors self-report, but that's somewhat dicing. So we've done some previous readership surveys that for the United States didn't show quite as strong a split as what we're seeing in the Louisville South here with the 76% male for readership. That is new data. What we're seeing, Toby, in the readership services, I think closer to 55%. I'm sorry. It's the comm score numbers. Sorry, Anna, so yeah. The comm score numbers show like 45, 55, maybe a little less. And it's remarkably consistent across the world. So yeah, we need to kind of, I mean, comm score in and of itself is not sure if it's dubious, but they may use different methods and get different results. So yeah, we need to dig into that. It's actually very little data right now. And you're seeing 35,000 responses, the largest sample space we've ever had in terms of readers. Yes. Wow. Thank you. Are there comments, questions before we break? Adam? Yeah, I just wanted to ask about one set of features we might be able to add. I see that we're talking about usability. And I wanted to add that maybe we should concentrate also on fun, since we do want to be the Farm Bill of Knowledge. Thank you, Mariana, for that. And this could be as simple as the idea that's been floated a lot of times of maybe showing recent changes in a different color or something so that you can see that this is a dynamic place where people are actually involved and things are changing. Another idea which will probably cause me to be fired is to say that we could surface commentary about articles so that there could be a mix of original content and trying not to see a recurrent content. But there I did. Adam, so fun. So we might have talked about it in kind of a sterile way. But for the reflex, enjoyability is actually one of those aspects. And enjoyability goes beyond, is it easy, but am I enjoying doing this thing? And if we apply that to tasks like editing or finding related content, I think we actually will start getting a measure of are people, is there delight in our system? And we can find something that's easy and successful but not delightful. And that's actually a place where we can start digging into how do we make our experiences delightful? Yeah, it's delightful, 4.7. Exactly. So the one thing I want to add on this idea of giving readers something more than statistics, for example, about article content. One of the interesting things that you can do with the Wikipedia article is you can break it into its parts. And its parts are things like images and tables and citations. And once you do that, once you start to analyze the things that make up a Wikipedia article, you can actually create notifications just on recent images that have been added to articles that you care about or recent citations that have been added to articles that are relevant to you because of your location. There's all kinds of interesting variations on this theme that you could think about that are less about things like page views and more about actually content potentially in a very immersive way. So there's definitely lots of stuff that we could do in this category. We need to wrap up for lunch. We are done for today. Thank you so much for coming. One quick announcement for the staff. There is a holiday party on the Book Media Foundation on 1217. So please sign up on OfficeWiki if you have not already.