 Okay, we're live. Hi everyone, welcome to the Wikimedia Foundation monthly metrics meeting. It's July 27th, 2017. I am Jake Orlewitz. I run the Wikipedia library program at the Wikimedia Foundation. And this is a neat week for two reasons. One, all Wikimedia staff almost all are working remote this week with all of the perks and challenges of that, which is why I am in an exotic shed in Santa Cruz, California. And many more staff than normal are likely wearing pajamas. This week, we're gonna look at outside our contributor core to ask questions of why people come to Wikipedia, how do they learn about us, and how do we reach them? So how do we look? I'm gonna open up some slides here so that you can see the wonderful deck we've put together. One of the perks of being remote is technical difficulties. So yeah, this is our theme, how Wikimedia is perceived by the world. So everyone outside us and around us. Today, we are going to do some welcomes for new staff and also celebrating people who have been with the Foundation a long time. We've got an exciting update of happenings from around the movement. And then two highlights. One, a study on why people read Wikipedia and why they don't. And two, a very interesting exploration of how we may be revamping the wikimediafoundation.org website. And then we'll end with some questions and the wiki love. So first of all, welcoming new hires. Sandra Focanier from the Netherlands, Crystal Stegenberger from Germany, Morgan Jew, who converted from a contractor in San Francisco, Cindy Chickalizzi from Maryland, Daylin Mazza from Florida and new contractors, Swati Goel from San Francisco, Sebrin Maiseland from the Netherlands and Katie Perry from Maryland. Kate Perry, I can't believe I just did Katie Perry. Kate Petty, I'm so sorry. I was like, don't do that, don't do that. And I did it. I'm sorry, Kate. I'll apologize to you in person. Also some very exciting anniversaries. The biggest, Aaron Schultz is the winner this month with nine years. Arthur Richards at seven. At six years, Michael Beatty, Jeff Green, Nicholas Laxtrom, Tilman Bear at five years, the wonderful Lynette Logan, four years, C. Scott and Ennion, Dennis Porter, Nick Wilson, Ty Flanagan and Brian Davis. Three years, Kristen Lans, Josephine Gulligan, Katie Francis, Trey Jones, oh, into two years. Katie Francis, Trey Jones, Mikhail Popov and Emerald Ross and one year, Gretchen Yen. So congratulations to everyone for staying with us and for all your contributions. And I'm gonna hand it off for the movement update to Maria Cruz. Hi, everyone. My name is Maria Cruz. I'm communications manager for community engagement department. And we are here to listen to a few stories from community members. So for this movement update, we have Daniel Metshin talking about WikiCite. Daniel, are you ready to present? Yes. Cool, thank you for joining us. Do you want me to share your slides or are you sharing your own? I'll share my screen and the slides. Hopefully this works out. Okay. Okay, I'm screen sharing now. Yes, can you see my slide, my screen? And then I'm going. No, I don't think we can. We can't. Well, then can some of them just... I can share the slides for you. Yeah, slide eight. So the point behind WikiCite is it's the idea that as a companion to the sum of human knowledge, we want to build the sum of bibliographic data that is the data about references being cited. And this project is called WikiCite. It is meant to handle the bibliographic data for all references cited across Wikimedia projects and possibly beyond. So that is references to books, to articles, also to data, court cases, patents and all these kinds of things. And what WikiCite does is we basically build data models for like a book, a book should have an author, it should have a publisher, a publication date, these kinds of things, which should have a title. And those things differ slightly between, yeah, let's say different books, different editions of the same book or between books and articles or articles and data and software and so on. So we're working on data models. We're using WikiData as the backend where all these data are being stored. And on that basis, we're building a citation graph between works, like if one book cites another book or if one court case cites a patent, things like this. And we are coordinating this with external efforts like open citations and the initiative for open citations. And yeah, the base for the project is on WikiData but we're also building tools which makes it anywhere. So for instance, there's shape expressions linked here that we are using to actually control like how a data model should actually work, like that a book should actually have a title, things like that or workflows. So you can now translate your Zotero library into basically WikiData information. And we're also working on pilot corpora. So for certain areas like the Zika virus, they're compiling the literature and the ranging it such that it can be explored in new and interesting ways. We've organized two meetings so far, one last year, one this year and this year drew 94 participants from all around, let's say bibliographic information. So librarians, Wikimedians, tool developers and others. And yeah, we're very open to collaborations to input and we're hoping to make good progress in the near future. Thank you, Daniel. I think up next is Ilya and Ilya, are you ready to present Ilya? The next story is about, can you hear me? Yes. Okay, so Ilya, we cannot hear you, can you hear us? So for everyone watching, the next story is going to be presented by Ilya Kornikov. It's about Wiki, the photo contest, Wiki Loves Earth. And we're waiting to see if Ilya can present. Yeah, hey Ilya, we can't hear you. If you could refresh your browser and check your sound input settings, please. I know, it's my Linux, it's a Western. Oh, there we go. We can hear you now. I'll summer you. Okay, we can hear you. And do you see the screen? Okay, so Ilya was born in 2012 and in 2013, Ukraine was running the first pilot Wiki Loves Earth. It's a contest of natural photography. It's the best to judge it, not by numbers, but looking on the winner photographs. And we had like growing number of countries this year, 37, including one Wiki Loves Biosphere Reserves. It's a collaboration with UNESCO in which 120 countries which have Biosphere Reserves recognized by UNESCO participate. Also the number of uploads, newcomers, uploaders is increasing. The only thing that is decreasing is images used in the Wikis from those numbers. And maybe because like many of these articles are already illustrated and probably we have to focus more on writing articles. I believe many articles are still missing about natural monuments, Biosphere Reserves so they can be used in a article that's still absent. And by the way, in Ukraine, we are going to write articles about every monument. We have like collaboration with dozens of ecologists. And I think we can promote this around the world also. So if you have any questions, I think that's all. Thank you for sharing, Ilya, about the Wiki Loves Earth Photo Contest. I'm going to leave this slide a little bit longer because I think I didn't share it in time. Okay, so next on movement update is the Wikimedia Foundation Highlights. We have the Abuse Filter Conversation is part of the Antiharazment Tools project. The team has begun a structural conversation about the Abuse Filter tool on both meta and English Wikipedia. This is an important preparation stage for prioritizing work to be done by the team and gathering input from users who write and manage filters. The Developer Relations team has finished a three-week project to document and organize guidelines and resources around Wikimedia hackathons. The resource is meant to be a guide for anyone who's interested in organizing a hackathon or learning more about how to participate at one. However, event organizers working on other types of events should find it useful as well. We published individual graphics about community engagement insights surveys to community. The report is still being finalized. Some data was shared with Wikimedia Foundation staff recently and we will be hosting a complete presentation of the report to communities a few weeks after Wikimedia. And finally, the Wikimedia Resource Center, we have after processing feedback from community members, we redesigned the space and launched a version 2.0. Resources are now filtered by user types and we also enabled contributions from anyone. And coming up in August, we have Wikimedia 2017 from August 9 to 13. This is the 13th annual Wikimedia conference and it will be held in Montreal next month. The pre-conference program is quite busy and it includes Hackathon, Learning Days, Wikiconference North America, Wikiproject Met Conference, Wikimedia Conference follow-up day. And Neri Dathan for First Nations on Wikipedia. So we hope to see many community members there. And coming up is why people read Wikipedia and why they don't. Every now, I'm just going to share my slides here. I'm John Morrison. I work as a project associate with the movement strategy team. Just give me one second, volume up my slides. Just trying to make sure here. Can everyone see my slides here? So we're going to be talking about movement strategy and brand research today. So I've been working with Juliet, Barbara, and Caitlyn Virchin, the entire movement strategy team to get a better sense of where we are in our high awareness regions. And we've been working as a little cross-functionally with a lot of different teams doing a lot of different research. And one of the things that just came in over the last couple of days was a brand research study that we commissioned. So I'll talk a bit about that. And then Layla will be talking a little bit more about why people really read and don't read Wikipedia. So I'll switch to scope and methodology. So we commissioned a study, kind of thought about it for a little while, that the movement strategy team and really wanted to understand kind of Wikipedia's brand in high awareness regions. So regions where we either generate a lot of funding or get a lot of traffic and stuff of that sort. And we chose Wikipedia specifically because it's our most noticeable brand. And we're really interested given kind of the limited scope that we had and the ability to do this to understand what our reach was for Wikipedia. So we solicited some competitive bids from different sorts of market research firms and selected a firm called Wellspring Insights and Innovation, who has been doing this type of research both in the public sector and the private sector for over 25 years. So we worked with him and collecting seven countries to do a run a survey, a series of surveys in each of these countries in the US, the UK, Spain, France, Russia, Japan and Germany. Worked with the firm over the course of several months to kind of develop the survey and the questionnaire and worked with members of the Wikimedia staff to kind of curate this questionnaire based a lot upon our past research from the New Breeders Program and phone surveys and some of the qualitative design research we have done in ethnographic research as well. And came up with this survey and got it approved and worked with the small team, released that survey in June and collected all the data and are just getting in the results in the last couple of weeks. I just want to speak a little bit to the methodology we decided to do a non-probability kind of quota sampling at the recommendation of the researcher, which kind of gives us the best balance between getting kind of a representative kind of directionally driven sample and as unbiased as possible. It's not a probability sample, so we can't discuss kind of margins and error and kind of that type of stuff, but we can get really solid directional data and it's a good balance between kind of cost effectiveness and trying to get really good information to kind of be actionable. And we chose ages 13 to 49 and kind of controlled by quota in this survey for age, gender, geography and also split it up by a generation, Gen X, Gen Y and Gen Z. And we surveyed 1,150 surveys for each country from four independent online opt-in surveys and it was administered online. So that's a total respondents of a little bit over 8,000 surveys across the seven countries. All right, so I want to move to kind of very much stress that these are preliminary insights. We just kind of got this data about a week and a half ago. So it's a little bit like, you know, trying to print something up for a king, as you can see in this image here, which I quite enjoyed trying to find on Wiki Commons. So we're kind of going to be doing serious analysis of it in the next several months and sharing this more widely. But we do have some kind of key, you know, high level insights that our researcher has been able to put together and we've been able to kind of go through Juliette and Caitlin and the rest of the movement strategy team that I'll now chat about. So, but it is very, very preliminary. So the first thing I think is kind of really interesting and it's broken down and kind of our surveys broken down into awareness, attitudes and usage. So on average across the seven countries, people think of Google about nine times more often than Wikipedia. When internet users want to find information online, they think of Google 64% of the time they mentioned as their first and Wikipedia about 8% mentioned Wikipedia first is where they want to go and get information. And only in Japan and Russia. So Japan had a very, very high awareness for Yahoo and in Russia, Yadex, I think I'm pronouncing that right. Google was sharing kind of high level first awareness of those very similar. So one of the kind of questions we also ask is when you want to find information online, what three websites would you say you go to the most often? And as you can see in this chart here, Wikipedia is fairly well positioned in kind of the second tier. Google, as I mentioned just before, is kind of far and away where most people go for information. And then YouTube is kind of close in line with Wikipedia. And depending on the country, you kind of see some differences. Spain has a very high awareness for information and kind of at the opposite end for Wikipedia at the opposite end, Russia and Japan, only 29% mentioned Wikipedia is where they go for the top three websites. I think kind of another couple of things kind of on awareness that was very, very interesting was kind of among all of the websites we surveyed, Wikipedia is among kind of a small group with about 70% greater awareness when we showed our kind of brand across all southern countries. And in kind of order of awareness, those are Google, YouTube, Wikipedia, Yahoo, Facebook, Twitter, so we're gonna pretty small set of people who are aware of us. Moving on to the next slide. We kind of crossed this data with some demographics and one of the main ones was age. And it was pretty fascinating to see Gen Z, which we defined as anyone aged 13 to 19 across all of these countries, had some pretty different behavior patterns and different usage and attitudes and awareness. And one of those was that 50% of Gen Z, only 50% knows that we are a nonprofit. Found quite fascinating. On average, six out of 10 people across all age brackets and in all countries know that we're a nonprofit with Spain being the highest at 71% and Japan being at a low of 44%. Generation, Gen X knows the most, 65% know that we're a nonprofit and Gen Z is the lowest at 50%, I don't know. I think kind of going on again with Gen Z, only 40% of Gen Z across all seven countries know that we are funded by reader donations, which just kind of highlights how little they actually know a lot of our users know what we stand for, what we do and kind of how the onion behind Wikipedia really works. So I think it's definitely kind of something we can think about and marinate on for a bit. And then I think kind of on the highs and lows here, Spain knew that 61% across all generations knew that we were funded by reader donations and then a low was in Russia. Only 27% of people in Russia knew that we were funded by reader donations. Move here to a little bit more kind of on attitudes and we kind of asked on a scale of one to 10, zero being the lowest, 10 being kind of the highest, please rate how strongly you associate Wikipedia with each of the following words or phrases. And this kind of gets at what's most associated with Wikipedia. And as you can see kind of on the top here, what we were most associated with was free knowledge for every person and then second was useful. And I think that kind of makes a lot of sense. What is pretty interesting here across the board Gen Z has lower scores. You can see kind of eight going down to 6.2. And then our research are also highlighted and we were kind of expecting somewhat free of advertising would have been much higher than it is here and kind of across the data. People, our researcher noticed that that wasn't as important to people as useful or free knowledge for every person. The other thing our researcher mentioned in his kind of 25 years worth of experience is usually for kind of their top one or two things. A brand has scores in the 9.5 or above. So overall these scores are fairly low for a large brand like ourselves in Wikipedia. Which as we kind of further slice the data by demographic and income distribution and stuff of that sort, it'll be interesting to kind of see where all this stacks up. I'll move to the next slide here, which is another one kind of on Gen Z which we really found kind of quite fascinating, very different and you're kind of thinking about the future here in the movement strategy, where our future readers are gonna come from. If they're not already reading it, how do we kind of market to and talk to to who are gonna be making the world a different place and hopefully a better place. So for this 47% of Gen Z only knows the content is volunteer created. So there's definitely kind of a disparity to what we do and what they know and their awareness, which I think is quite interesting. Moving on to the next slide, which I found was quite interesting as well was that we asked a question, which of the following best explains why you've never tried to edit a Wikipedia article to people that knew you could edit an article. So three quarters of people know that you can edit a Wikipedia article of those who were surveyed across all the countries. And among those who know, 80% have never tried and we asked why and about 40, on average about 40% of them said they were concerned that they would make mistakes. And I just kind of found that and our team found that quite fascinating is that people are afraid that they're gonna break Wikipedia if they edit. So how do we think about ways and to make that not the case? And I think that is quite a fascinating thing to think about as well. Kind of through this presentation about the kind of outliers and kind of the bans. And I think Spain is kind of very much on the kind of model end of there's very high awareness. It's people really kind of know, compared to all of the other countries, what our mission is, what they associate us for that we're a nonprofit that we're volunteer driven. I think Spain was the most aware in kind of across the demographics of, and stuff of that sort. On the other end, Japan and Russia have kind of very low awareness. Japan has a high usage of Yahoo and a lot of people indicated that they were driven to Wikipedia from Yahoo, which was kind of unique and kind of most people kind of mentioned Google. In Russia, across the generations, 74% access via desktop. So we have a lot of data that we haven't quite been able to tap into yet around that information of what platform they use. We asked about Siri and Alexa and there's a decent proportion of Gen Z that gets a decent minority proportion of Gen Z that gets their information from Wikipedia through that. So we're gonna be posting this in the next, that we have an executive summary up now, which is linked in this slide right here to go explore. And you can click on the explore for yourself once this is posted and go to that. We're also gonna be posting all of the data and country reports in the coming weeks, hopefully be kind of doing further presentations in the next several months in relation to the movement strategy, really kind of diving in and slicing this data and working with our product and tech and kind of across the foundation to use this for the future. I'm gonna pass it on to Layla now and she's gonna talk a little bit about her research into why people read Wikipedia. Thank you, Dawn. Sharing my screen. Okay, yeah, thank you. So it's very fortunate to have kind of the more qualitative slash quantitative research on kind of the brand and the awareness around Wikipedia that have been presented hand in hand with kind of a deeper understanding of why people read Wikipedia. I'll be talking about this topic in the next 10 minutes. This is a joint collaboration with our former collaborators in GASIS Institute, EPFL and Stanford University. And before getting to more details, thanks to all the teams who have been involved in the past, I would say a year and a half around this research, the teams are reading legal security analytics and the broader Wikipedia communities who have been working with us on this research. Okay, so we're gonna be focusing on readers and specifically the readers of Wikipedia. And I wanna start with this stat. Every second, 6,000 Wikipedia pages are viewed by people. And at the beginning of this research, we didn't know why. The issue is that providing education on content and effectively disseminating it requires understanding the needs and motivations of the people who are behind these page views. And this is what this research set out to do. The issue that we faced early on was where to start. Basically, if you look at the kind of data that Wikimedia Foundation collects, the biggest assets of data in terms of readership that we have is web request logs. These are the kind of logs that will help us understand that a user has come from the search engine. They have gone to an article, such as Oceana article on Wikipedia. They have then gone to, let's say, an article about Australia. They go outside back to the search engine and then they come back to read about Nauru, right? The problem with these logs is that we have simply too many of them. We have somewhere between 150 to 200,000 requests or seconds to our Wikimedia servers. And this data is being stored. We can analyze it. The challenge that we have is just that there's too much data and there are too many dimensions to look at if you don't have any clues about the users and the type of things you're interested in. So we started thinking about how we can address it. And we tried basically to address the issue of finding clues in web request logs by mixing web request log analysis with reader surveys. So the idea here is that we go to the readers of Wikipedia and we ask them some questions. And we ask them in ways that they can help us combine the survey data that we collect with web request logs to make sense of what we have stored in web request logs to try to help us understand who are our readers. Now the issue with the surveys is that we don't know what to ask. If we knew what to ask, we could start already looking at the web request logs. So we started with the first survey and the goal of the first survey was to build a taxonomy. Think of the taxonomy to be a set of questions that will represent the readers of Wikipedia. So this survey was a very simple survey. If you were on English Wikipedia, mobile or desktop, you would be sampled. And if you were part of the survey sample, you would receive one question. Why are you reading this article today? And you would see a box in which you could enter free form text. This is what you, the kind of responses that users have shared with us when we asked them this question. So for example, they say, studying for my med school test. I have previously edited this page. I have personal interests about conflicts in Middle East, East, so on and so forth. Now the challenge that we have is that, okay, we have received these responses from people, but these responses are in free form texts. And what we need to go from here is to analyze these free form texts and create a taxonomy. A set of questions that we are gonna put in front of readers to help us understand them more. What we did is that we did series of hand coding sessions and we talk about this in the paper where you can read more about the setup and how we did them. But those series of hand coding sessions helped us narrow down the scope of the taxonomy to three major areas. We understood that when we asked people why are you reading this article today, there are three kinds of information people communicate with us. They communicate the information depth or the information need they have when they are on a Wikipedia article. So they talk about whether they're here to get a quick answer to a question they have, whether they're reading an overview or a summary of a topic, or whether they're here to do an in-depth understanding of the topic. They also sometimes talk about their prior knowledge. They tell us whether they were familiar with the topic prior to coming to Wikipedia or they were not. And they also talk about their motivation. What has triggered their visit to Wikipedia? They could be triggered by sources of media. They could be interested in reading on Wikipedia because they have some personal decisions to make. They may be just simply bored. The topic may have come in a conversation for them. They may have a work or school related assignment or they may want to know more about a current event or they may just simply want to learn about the topic and this topic may be of intrinsic importance for them. And there may be other reasons. So there's an other box in that category. So we basically so far what we have done is that we have built a questionnaire, a taxonomy for Wikipedia readers. Now what we did is that we had to answer the question of whether this taxonomy is robust across languages. So remember that we ran the first survey only on English Wikipedia and there's a natural question of first, does this survey address all the things that people wanted to tell us when we asked them that question? So we rerun this survey on English Wikipedia but also whether this is robust. So if we run the first survey on Spanish and Persian Wikipedia, are we going to get the same categories and dimensions? We did survey two and we checked the box that the survey questions are robust across languages. Now with this information we went to survey three which is the grand survey. The grand survey is run on English Wikipedia and there are three questions that people will see. Those are the questions that you saw in the previous slide. The survey ran for a duration of a week, one out of every 50 requests were sampled on English Wikipedia. Both mobile and desktop platforms were considered. These surveys would be shown to users on article pages and to those users we do not track feature off and we received close to 30,000 responses for these surveys. Now let me talk, show you some of the results that we got from these surveys. Two things I should say. One is that we have done the biasing of the results. So these results should be representative of the general Wikipedia, English Wikipedia population as much as you can say general, right? So you can read in the paper, we talk about the methods that we use for the biasing and what kind of bias those methods will address. But this is the best we could get so far. The other thing is that before I show you each plot or every result, kind of think about what would you expect to see because some of these results when you see them they're like, sure, but if I would ask you prior to running this survey, we would never get accurate estimates. So the first question was information lead. We asked people basically what percentage of them are here to do overview or summary reviews, quick fact checkups, checks on Wikipedia or reading in depth. And what we learned is that in total close to 80% of the people do overview and fact checkup checks on Wikipedia, but another 22% come to Wikipedia. Again, I should emphasize English Wikipedia for in-depth reading. In terms of prior knowledge, what we learned is that the population is split. A 50% report themselves as being familiar and almost 50% also report themselves as being unfamiliar with the topic that they're reading about. What we also learned about prior knowledge while looking deeper in the logs is that users who report familiarity with articles and topics, they are users who tend to read what we call spare time oriented topics. These are topics which are around sports, 21st century, TV, movies, novels on Wikipedia. They read usually popular articles, they read longer articles. They also read articles that are central in the network. You can think of these articles which are basically less niche. Now let's look at motivation. Here, there's also very interesting. What you learn is that 32% of the people come to Wikipedia when their motivation is a source of media. They have watched the movie, they have read a book, they have heard something in the news and they come to Wikipedia to learn more. This is amazing to see because it's a great opportunity and it's a great insight to learn that media still plays an important role for people's learning or curiosity and bringing them to Wikipedia. What is also really interesting to see is that the next biggest category is intrinsic learning. 25% of people come to Wikipedia because they wanna learn and it's important for them to learn. This is an amazing number to keep an eye on as we think about our users. And another interesting thing about this plot is that the drop is pretty gradual. You don't have any reason or motivation trigger which is so dominating compared to others. Now let's look at what reprequest logs tell us by focusing on two motivation triggers. One is for people who come to Wikipedia to basically do a school or work related project. What we learn about these users is that they read articles about war and history, mathematics, technology, biology, chemistry, literature and arts. They generally read topics that are related to academics and professional activities rather than leisure activities. They stay longer on an individual page. They are more likely to use external search engines. Both were coming to Wikipedia initially but also within their sessions. And they use Wikipedia desktop version primarily for the activities they do. What we also looked into was the category board and random. These are people who report that they're on Wikipedia because they're bored. What you learn about bored people is that they read articles about sports, 21st century TV, movies and novels. They read topics that are more spare time oriented. They spend only little time on individual articles. They come to Wikipedia more often both within the survey session time but also during the week that we studied the population. And they also switch topics frequently. They're more likely to use internal navigation as opposed to going to an external search engine and coming back to Wikipedia. So the key takeaway from this study that basically if I have one chance to have your attention and have you remember one thing is that English Wikipedia is read in a wide variety of use cases. Readers defer in their motivation triggers. The depth of information need that they have when they come but also their prior familiarity. And there is no single dominating use case for English Wikipedia readers. The next steps for this research, we are assessing the robustness of the observations that we have for English Wikipedia and the prevalence of the use cases of the taxonomy across 14 languages now. We have run surveys last month. We have received close to 250 plus 1,000 responses and we are analyzing those responses now. We will be, we're hoping to share the results of those responses in Wikipedia. Ongoing documentation is on meta and the paper on the English component of the study you can read about it more why we read Wikipedia in the link and the slide. I'll stop here and I'll pass to Larisa. Thanks. Thank you so much. I'm gonna share my screen really quickly. Here we go. Share. All right. Can everyone hear me? Yes? Okay. Yes. Fantastic. Thank you so much for having us today. I'm Larisa Berger. I'm a strategist and researcher here at Newell Design. Newell Design is just up the road from the Wikimedia Foundation office in San Francisco. I'm joined today by Erica and Maggie. You may have spoken with us in the last couple months. We've done a lot of projects with the Wikimedia Foundation in the past. The first project we did with you guys was policy.wikimedia.org, the policy page. We also did Wikipedia 15, which was a really fun project. I'm here today to talk about WikimediaFoundation.org, which is our next project that... Do you guys see my screen still or no? No, we just lost your screen share. Okay. Gotcha. How about that? Yep. I can see that. Oh, there's everyone. There we go. Boom. Okay. Awesome. Great. So I'm here to talk about WikimediaFoundation.org. Here we are in the project. You guys did an internal audit and now we're in the discovery phase. I'll talk a little bit about what discovery is. So discovery is the first phase of work for any design project that we work on here at Mule. And in design overall, form follows function. So we begin with discovery to understand the functions necessary before we design or make anything. Ultimately, we're gonna make a communication platform. So first, we wanna understand what needs to get communicated with that platform. So for this phase of discovery, we interviewed staff and community and we wanted to find out firsthand from people in their own words about the work that they do. And we found out it's much broader than just Wikimedia. We knew that before because of our other projects but we found out about all of the amazing work that you guys are doing across disciplines. So we found out about Wikidata and the research happening with new readers. And we found out that you guys are really a global movement unlike any we've ever worked with and it's amazing. And basically through this work, what we're looking to do is to find out what the information is that we want to communicate and the different structures that it takes so that we present that information as authentically as possible. We also interviewed external audiences that we could discover their behavior patterns and get a better sense of how we could piggyback different conventions and patterns. So for this project, we talked to 40 members of foundation leadership, staff, contractors as I mentioned. We also conducted a comparative review which I'll jump into a little bit later. And lastly, we reviewed the current site and some select pages on the meta. We did not get through all of meta. And at a really high level, here's what we learned. Like I said, you're doing work relevant to the whole world and there are many channels of communication. The key audiences we identified for the Wikimedia Foundation org website are the general public potential donors. And this includes people who may have never given money to the Wikimedia Foundation yet to date because they don't necessarily know about the work that you guys are doing. Press, job seekers, volunteers. And this includes editors but also includes everyone who gives their time, helps put on events, contributes content to comments that we've seen in the slides today. And also partner organizations. So Ford Foundation, EFF, ACLU, all these organizations who are doing similar work that can partner with you. And what we found overall is that there's a huge need for accessible communication about the work. So in our 40 interviews, 40 plus interviews, we ultimately end through all this work. We examine how you find, organize and distribute information. And we found that there are basically three modes. Oh, there we go. There are three modes for sharing information and they're kind of conflated in one medium right now in Wiki, but there are three characterizations to the kind of communication you're sharing. So sometimes you're documenting and you're really good at this. You have a whole encyclopedia to show for how good you are at documenting. And just so we kind of establish some first principles, the goal of documentation is to create a shared memory of the work. And really, really great documentation also enables people other than the people who wrote the documentation to continue that work. And documentation may get passed on but ultimately it isn't communication. So here when I say documentation, I mean something like storage or a read me, a top page in this case is not documentation. Again, you're a really fantastic at documentation. I would say the Wikimedia movement is an engine for generating documentation. The next, oh, there we go. We also looked at collaboration and how that functions in the movement and at the foundation. And we noticed that there is a role that documentation has in collaboration and that people are highly participatory. But we also noticed that not all participation is necessarily collaborative. And again, if we look at the root here, collaboration is working together towards a shared goal. And sometimes documentation is important to collaboration. Sometimes it's really necessary for it, especially for the kind of work that you do where you're collaborating across time zones. But what often is the case is that excess documentation can impede collaboration because there's a super high barrier to entry. There's lots of stuff you need to read just to know about different work that's going on and how you can get involved. It's kind of like a dusty record collection. And the thing about digging through documentation is that it's intrinsically solitary work. Kind of in the same way that we learned at Editathons and other places where you collaborate in person, pizza intrinsically enables collaboration and is a really important part to the budgets of what make this work possible. Well, it's my slide. Here we go. Oh, there we go. So communication is going to really help with collaboration. And communication is about conveying an intentional message with a purpose to specific audiences. And that's what we're here to help you with because communication has a point of view. And that's why it's really tricky because so much of your work and Wikipedia prides itself on having a neutral point of view. And that's a really important contribution to the world. So because so much energy is focused on that, it's definitely scary to create other art facts that do have a point of view. And maybe this is surprising or maybe it's not, but for everyone that we spoke to, 100% of the people said that the Wikipedia is neutral, but it is important that the foundation does in fact take a radical position to fulfill its mission and to fulfill the vision that aligns all of you with a shared goal and a shared purpose. And is why you wake up every morning and spend time on this amazing resource. So again, just to reiterate, documentation and collaboration are no substitute for communication. You achieve a very high level of transparency and overall what we observed is that people often think that the work itself will communicate for them. But again, documentation is not a replacement for communication. So we found there's, while there's a really high level of transparency, there's a low level of visibility. So the Wikimedia Foundation website that we're going to set out to design as the next phase of work, we found that it does really have a job to do that's not being done by anything else right now. And that's so exciting for us. And we're really excited to work on this with you and collaborate. Right now, this page, the Wikimedia Foundation Wiki, it looks like another page of Wikipedia. And so when people come upon this, people outside of the organization and the movement, they think that they're just on another page of Wikipedia about something foundation related. They don't identify it as being specific to the Wikimedia Foundation. This is WikimediaFoundation.org should have your outside voice. And this page speaks with your inside voice. It speaks with the voice that you use to communicate internally. So it's really confusing for an external audience. We also found that everyone's job is much harder because they have to constantly communicate basic things about what their job is at the foundation, what the foundation is, what its mission is, because there's no one place to point to for this information. Nobody knows where to start. If you land on the Foundation Wiki, you don't necessarily know where to dig in if you wanna get involved with the movement. So our goal is going to be to provide people with a starting point. Because when people show up to a website, they arrive with a question. Maybe it's someone from the general public who's heard about Wikipedia in the news and they wanna know more about the foundation. Maybe it's a potential donor who's never heard of Wikimedia Foundation, but they Googled, how do I give my money to Wikipedia? And they've landed at this page. Anyone who wants to support the projects and wants to know how to volunteer should be able to start at WikimediaFoundation.org and find the information that they need. It could be a program officer from another foundation who wants to understand how grants are made and see if they can contribute to that work. It could be a prospective employee who's been contacted through Recruiter or never even heard of the Wikimedia Foundation, like so many of you who we interviewed. And it could be as simple as the families of people who are already working at the foundation who wanna make sure that their child doesn't work at WikiLeaks. That was an actual story that we heard. Now really quickly with our last two minutes, I'm gonna jump in really into the comparative review. We review a lot of sites, many more than we'll look at today, but in the design process, the comparative review is very useful because it lets us see how organizations with related goals and audiences communicate about their work. So this is not about visual polish or style. I really wanna emphasize this. It's about how these organizations have structured information and they often had to deal with similar communication challenges that you have. So we wanna identify conventions that are being used so that we can piggyback those conventions and make the information accessible and findable. And we also wanna see where there's wiggle room to really differentiate the foundation and its work because there is no other organization in the world that is doing what you are doing and that's amazing and we wanna be true to that. So Creative Commons, of course, was a starting point. Specifically here, what to look at is that they communicate very quickly and effectively why you should care about Creative Commons, what you can do, how to get involved. They even mentioned what you could do right now about around net neutrality and it's very clear how to donate. A really interesting case was the Mozilla Foundation. Lots of people know about Firefox. They don't necessarily know about the Mozilla Foundation. This page also interacts with a wide range of audiences. You could dig in at any level and it's all achieved here on one page. And last but not least, a local organization, 826 Valencia, while still kind of letting their personality show through, it's a very professional page that achieves all the things that a nonprofit website needs to do. And Wikimedia and the movement and the foundation and the community all have a unique voice and so we wanna be true to that. So this is just a quick overview of how we're thinking about this problem and we're going to join you at Wikimedia. We're super excited, none of us has ever been. So please find us there with any questions that you might have or any input. We're really looking forward to it and I'll hand off to Jake. So if you could just, yeah, perfect. Great, so thank you very much to all of our speakers. It's really fascinating research. We have a little bit of time for questions and we may go a little bit over for those who can stay. So in the tradition, I will hand it over to James who is monitoring the IRC thread and start there. Hey there, so hi. I've got a couple of questions for John. One from Chris, which is, how much do folks know about the other nonprofits they work with or use? I mean, do we have any understanding of whether there's a baseline to compare against? No, not at the moment. It's probably something we can do and one of our researchers said for further research to recommend kind of diving into trying to get a little bit more of a comparative analysis, but it's a little bit difficult at the moment and we're gonna figure in that out still. If I can jump in, this is Julia. The survey really focused on general internet users and their perception of Wikipedia, something that they use online. And so, as John said, this one didn't focus on us. As a comparative brand to other nonprofits specifically, but further research, as he mentioned, could. But I think what was interesting is that so many people did not know that we were, did not even know that we were a nonprofit. And so that tells us that they might not compare us to other nonprofit brands because we're not in their minds classified as one. Understood, thanks. Next question is from Matt. So didn't you say that the data is not a scientific sample because we can only use it for directional data? And given that, is there enough science in the study to draw finer distinctions around like 8.0 versus 8.3, that kind of stuff? Julia, do you want to take that? You want me to do it? Sure, I can take that. Yes, so this was intentionally a non-probability sample and the reason for that, there were many reasons for that, including that it would have been outrageously expensive in working with this consultant to do that. And so you can't talk about margins of error with this data. I think the point about kind of 8.0 versus 9.0 on those word associations was an insight that we learned from our research consultant who again brings 25 years of experience on this. And in his experience, he had not seen, in his experience, typically brands had higher association in the 9.0 range with certain words. And so that was really kind of from him, but it's not, as you mentioned, a scientific study. And we made that decision also after receiving feedback from internal stakeholders and speaking with various people and product and communications and realizing that a directional study would give us the information we need to make general decisions and that we wouldn't need to go and do this more complex and very expensive study. Thank you. And yet another one for John, sorry. Where do we want our awareness of brands to be? Oh, sorry, not for, yes. I can imagine that we want 100% of respondents to know Wikipedia is written by volunteers, but do we also want to know the same ratio for knowing the foundation exists, that it's a nonprofit? What are our objectives around that? I think, and Julia, feel free to jump in when you want to. That's a conversation we want to have as a part of the movement strategy. And kind of an understanding of, as we further and analyze this information and further presented and discuss it with people across the organization, we're going to have to start assessing and making those decisions based partly on this data and potentially on more studies. Julia, did you have anything else to kind of touch on that? I think that's right. It's something that as an organization we need to decide. And I know that this research is, built on some existing research, but in many ways it's a starting point for understanding where we sit as a benchmark, as a brand. And it'll give us a lot of information to have informed discussions about that. And I know that the communications team has some work in this area planned for the next year. But I think we need to really discuss and do more research to understand what information is critical that people know about our brand. Is it critical that they know we're written by volunteers? Is it critical that they know we're nonprofit? One of the most interesting insights is that people didn't really, didn't care as much as we might think they care that we're free from advertising. So, yeah, it's a discussion and we need to really figure out what is the most important, one of the most important things that we need people to know. Great question. I've got one more question from Pine for Laila which has been mostly answered in IRC which was asking what are the principal primary objectives of the readership research? Yeah, I can address that really quickly. So basically it depends, it may depend on who you ask if you're asking about the reader research, why we read Wikipedia, what we did there. The primary goal is to understand readers. As Dario mentioned in IRC, there's no way to have an informed product features, understandings, programs to be built for people who we don't know much about. So that's basically understanding readers is the goal of that program. I'll stop here. Thanks. So we are about five minutes over time. I'm not sure if we have another forum for people to ask questions as we're all remote. Does the YouTube stream have any questions or is that all for today? I don't see any questions in the stream but I'm scrubbing through right now. Okay. Yeah, it looks like there were some questions in the stream but they were answered in the stream thread. Great. And if you have questions, feel free to ping people and follow up. Everyone's excited about the work they've been doing and wants to talk more about it. We like to end with the opportunity for WikiLove which will be a little more difficult in this format but if there is anyone who wants to call out and appreciate someone else in the movement something that they've seen happen now is traditionally the time when we do it and if that's not possible in this format then I'll just share a little WikiLove with everyone and appreciate all the great work today from our speakers. So WikiLove will be possible. We'll just have to wait about 60 seconds for the people to see that in the stream. Got it. Victoria says some WikiLove to Lynette and the Move team. Zach MC says WikiLove for Ellie and everyone working on Wikimania. Pine mentions that WikiLove would be that I'm glad to see the work happening with structured data on Commons and on the recent changes feed or WikiLove for Lynette and Robert for their work on the Move. And Edward is very glad that WikimediaFoundation.org is being redesigned. Thank you for doing that work. All right, let's wrap it up there. Really just packed metrics meeting. So much good stuff happening. Thanks for attending. Thanks for speaking. Thanks for all the work everyone is doing around the movement and at the Foundation and enjoy the rest of your day or evening. Thank you all.