 That's okay. We're officially on air. Yeah, welcome to the metrics meeting, where we make the world a better place, five, 10, and or 15 minute increments of time. Let me get my clicker. I'm your host today, Trevor Pascal. So we have some interesting people talking about interesting things today. I have full trust that it is. We also welcome some new people, some new hires here. Yay. Welcome to the foundation. And actually, three of these are local. So bump into them and say hello. And also some contractors, which is great. And some anniversaries. Asaf has been here for five years. As well as Lisa, Pow, Dan, and the other Dan. How many Dan's are on this board? Ellie, Ambron, Yuri, Adam, you can read. Raise your hand if you can't read this list and you need me to read. That's cool. All right, we got that sorted. Five years is a long time, Asaf. We've been putting up with you gladly. So without further ado, I pass it off to Alex. Hi, everybody. I'm Alex, part of the community resources team. And as always, there have been a ton of interesting and exciting things going on in our communities around the world. February was Black History Month. There were over 23 editathons in the US and Europe. And there were three in Washington, DC alone. And we had over 550 participants and over 550 articles created throughout those editathons. That's pretty awesome. If you don't know about the work that's going on in New York City with the New York City chapter and Afro Crowd and the work they've been doing to support more writing about Black history, Black culture, ask us, because it's pretty March was Women's History Month. And also, when the Art and Feminism Project organized its global editathons in 100 different events, there were editathons, there were training the trainer workshops. This is a project that we have been supporting here at the Foundation for the last two or three years. And it's super exciting. It's one that I, it's actually my favorite grant that we give. But it's a core team based out of New York City. And they support nodes in many, many countries, all over the place, different communities, different languages. There were 2,000 articles and nine different language Wikipedia. They're a pretty awesome group. Also, many of you may have read about one of our star Wikimedians, Emily Templewood, who is a medical student. She, on the Wikimedia blog, was a great story about how she turned online harassment into positive response through writing articles about male scientists. And this story got picked up at a lot of media outlets. And it's just a really super inspiring individual. And she's kind of spreading her message and her method at her local university and to different communities all over the place. It was also the International Francophone Contribution Month. Dozens of events, trainings, workshops, conferences, photo walks, and 13 different Francophone countries over a number of different themes. March was the 15th anniversary of the German and Catalan Wikipedia. The German Wikipedia has over 2 million articles now and has basically replaced their version of the printed encyclopedia, which stopped being printed in 2014. Catalan Wikipedia now has over 500,000 articles. They celebrated their anniversary with a hackathon about the Education Collab, which is a group of education program leaders met in Sweden. And they were looking at the lessons they've learned over the last 10 years of doing education programs in Wikipedia. They're going to be focusing on their program and events dashboard, which is a way for program leaders to help track activity in the classroom, mentoring, documentation, and information. You want to learn more about that in this link. Other project milestones already said that Catalan Wikipedia has reached 500,000 articles. Danish Wiktionary has reached 30,000 entries. The Kanada Wikipedia, which is a language based in India, has reached 20,000 articles. And Russian Wiki News has reached 2,000 articles. That's right. Hey, guys. Hello. So I'm here to give an update on the strategy and annual plan, which I think some of you may be familiar with. It's a quarter back. All right. So here's where we are at in terms of the strategy planning process. Over the last couple months, as you know, over the last 18 months or so, we engaged in understanding the issues, testing different strategic approaches, putting them out in front of our community, getting feedback, working on building the strategic plan, which was out on Wiki last month for commentary, and finally developing the annual plan and metrics, which each of your teams and departments have been involved in over the course of at least the past month, and probably has really ramped up for you, many of you, at least over the course of the last couple of weeks. I want to say a really big thank you to everyone who has been involved in that process, and encourage everyone to stick around after the metrics meeting this month today to talk a little bit more about the various different narratives, what the teams are doing, and hear from our CFO, Jaime, on our revenue projections and a little bit more detail about the budget. So what is our strategy? It's a guiding direction for the next 18 to 24 months. It's what allows us to sort of shape our annual plan and understand why it's going to have an impact in our movement and how it responds to community priorities and needs, as well as our broader overall vision. The strategic three strategic priorities we identified were reach, communities, and knowledge. I'm not going to read these, you guys have probably seen them before. So strategic priority number one was reach around reaching more people around the world, and some of these points below speak to how we're going to do that over the course of the next year. They go, we go into a lot more detail into all of this in the annual plan, which will be published tomorrow on Wiki. It is about 60 pages long on Google Docs right now, so it is as rich a resource as you'll ever find about all of the work that we're going to be doing over the course of the next year. But some of the things that we're looking at are evolving our user experience to meet user needs based on research and testing. Engaging new readers from new countries and communities are going to hear a little bit more about that later today. Improving research and analytics capacity for increased understanding, and improving performance and reliability for infrastructure of our sites. Strategic priority number two is around communities. So we are going to work to increase volunteer retention and engagement through improving programs, experiences, and resources. And really this is looking at increasing the retention of existing volunteers and empowering our volunteers with function, in key functions with tools and trainings. Improving outreach to new potential volunteers, improving community culture and resiliency. The community engagement team has already deepened some of the efforts around this, and improving collaboration with our communities in product development. So it's a really holistic approach of trying to bring communities in and focus on their priorities. And just a big thanks to all the teams and their thinking on this, and the ownership of this across the organization. Yeah, heard a little clap. You guys can clap for yourselves. That's awesome. All right, and then knowledge number three. This is really where we look to our communities and want to support them in improving the quality, diversity, and breadth of knowledge across the projects. So what we'll be doing here is improving mobile engagement and contribution. Again, you're gonna hear a little bit about the importance of mobile later this morning. Services to help volunteers improve quality. So this builds on some of the great work that the research team has been doing with ORS and other experiments in artificial intelligence. Developing and supporting administrative and other volunteer tools. So this builds on some of the great work that the community tech team has been doing, among others. Facilitating institutional knowledge partners such as our relationships with Glam institutions and really sort of structuring this in a more formal way and improving tools for multimedia and wiki data integration and reuse among many other things. So definitely take a look at the annual plan. The annual plan, speaking of. So this is just a brief snapshot of where we're at. We have set a target of 63 million in revenue for the coming year and budgeted around that. So 63 million in annual operating expenses. This is about 2 million down from last year where we were at 65 in 15, 16. We're publishing it tomorrow in adherence with FDC structure and deadlines. So this is gonna be open for community review from tomorrow, April 1st until the end of April. And then it's gonna go to the FDC who's gonna give us feedback on strengthening the plan overall before we bring it to the board. It lays out our programs in details differentiating between core, which is the day-to-day important work that makes sure the sites run and strategic work, which is that that we do to strengthen our community and improve experiences and make wikimedia projects, bring them closer to our overall mission. And program work, which is both core and strategic, counts for about 70% of our overall foundation budget. This is a sort of important, if a little bit abstract concept, it means that most of the work that we're doing is really in service to the communities and to our readers and overall in the direction of supporting our mission. The other 30% of the work is the essential work that the administration teams, the finance teams, HR and legal provide us so that we can do that program work. So big thanks to all of you guys. And here is where we're at. So April 30th, or first in the process for publishing tomorrow, it'll be available for comment. Both the annual plan and the revised strategy are up. Please do encourage folks that you know and talk to to provide us feedback on the annual plan. It's what's going to make it stronger and it's what's going to create more buy-in and support as we go through actually deploying some of those programs over the course of the next year. And then you can see FDC deliberations in May. That's right, yeah, May and then in June, the board will vote to approve the annual plan for the fiscal year beginning on July 1st. Coming up pretty soon. I will leave time for questions, I think we're leaving time for questions at the end, but it's pretty straightforward and I'm gonna hand it off to Adele to give us an update on strategic partnerships. Hi everyone, good morning. I am Adele and I am here today, oh this one, to introduce you to a new team and that's the Global Reach team. So you might be wondering, yeah, like I know, I know there's some of the spaces at the list and some of us have been here for four years or three years, like Ben, Ben Foy has been the mobile partner technical manager for four years now, I have been here for three years and a half, Samuidi Gupta is based out in India and has been managing our partnerships for all Asia in Eastern Europe, Jorge Vargas is based here now but he's Colombian and he's our regional manager for Latin America. And our newest member, Jack Reba, that you probably are getting to meet now, he's pretty new and he is backfilling my former role as the regional manager for Middle East and Africa. And you were like, yeah, I know these guys and they were the zero team, right? Yeah, that's how we were formerly known and we all know here that Wikipedia Zero is our more mature program in the global south but we are broadening the scope of our work. So I'm gonna talk more about how we came to the Global Reach name and why we are broadening the scope. And from I think three years and a half working in the new readers' territories, we have learned a lot, we've talked to a lot of people and we came to talk to them and learn that we need to improve our awareness, right? There was lack of awareness, we also need to reach out to marginalized voices like more women and more diversity in ethnicity and race and LGBT and all the voices and then we need to just improve the way people are coming to read and edit out our sites. And we were bringing that to the foundation and people are coming back to us with questions and asking about data. So where is the data to prove that there is actually lack of awareness? Where is the data that people are facing all these barriers for cost to access our content? And it was hard for us to make the case, right? And then we decided, okay, let's work on data. And now we have been surveying people in this countries and the countries where we have Wikipedia Zero partnerships or countries that we don't have but we would like to increase our presence like Mexico. And we're talking to people that are not online and we're not reaching them to the internet or to our Wikis, we're calling them using their mobile phones, feature phones or smartphones, whatever phones they have and we're asking questions about have you heard of Wikipedia before this call and not to our team's surprise but some of you might be surprised that almost 80% in Ghana they have never heard of Wikipedia before. And in Mexico it's a little bit better, right? Like we can see that 55% have heard of it before but we see that in countries we are going to survey more people in Africa. The next one is Nigeria but it's a really high number of people that have not heard about us. And then when we talk to them about feature phones and is smartphone usage to gather information that they access the internet then we can also see that there's a lot of feature phones is still in this market. And when we go even further and ask them how they're using their smart phones many of them have smartphone usage but they're not actually using the internet because of the cost and because of other barriers like digital literacy. So this surveys they're drilling in and they're really gathering information on our new readers and potential editors. And they are all like the Ghana surveys now posted on Meta and our office page as well. So you can just go there and take a look at the other questions and results that we have posted. And then this slide here is not to talk about the piracy issue and or the non-neutrality discussions. Our team is more than welcome to talk to you about that. I really wanted to make the time today to talk about our work and expansion of our scope. But if you have questions, please come to us. Go to our office page, go to our Meta page and give feedback directly. I would not welcome just like some snarky comments on social media before you come and talk to us, right? Come and talk to us. We are open. We want to talk to you, community and staff. But through Wikimedia Zero, we have learned a lot and we have learned that the challenges that we are up to are so complex that one tool is not gonna cut it, right? Wikimedia Zero, it's not the solution to all the barriers that we see in the markets that we are working and we need more tools. And we are expanding the toolkit and we work in partnerships. So our job is to find other partners and to do other partnerships that are gonna increase our reach and that are gonna increase readers and potentially and hopefully our editors in this new readers' territories, right? So we are gonna be talking to local nonprofits, to private sector, to governments and to educational institutions. On this last one, we are gonna be tagging team who already are, but more officially with the education program too. So we are already working together in these regions with them and we wanna just really reinforce our work there. And for private sector, I can name, for example, the work that we're already doing with Wi-Fi providers. So just yesterday Jorge closed the partnership that is now live in Mexico for a Wi-Fi distribution of Wikimedia for free. So people in Mexico can access now Wikimedia for free to their Wi-Fi. Yeah. And then we are also working with local providers, local manufacturers providers to have our app, our iOS and our Android preloaded. We are giving emphasis to, of course, our Android since Android usage is way higher in these countries and we hope to have some news there soon. So next steps for us, we are going to share more broadly the results of our phone surveys in Ghana and Mexico and pretty soon we are gonna have the surveys being run in Nigeria. So we are going to meet hopefully with all of you or some of you to release the results and then discuss the results in a brown bag. We, as I said, we have our meta page now in office page. You can just go and look for global reach. It's there and please give feedback. And then we have also been increasing this new global reach team, increasing our office participation or wiki participation, sorry, and also increasing our cross functional work. So we are working closely now with communications, with research, with reading and you're gonna listen and hear way more about that but we have a cross functional work being done with focus on the readership and that's how I'm gonna handle that to end. Hi everyone. So for those of you that don't know me, I'm Anne Gomez, I'm a product manager with the reading team. Ah, there we go. And Abby and I, Abby's the lead design researcher here at the foundation. We're here to talk to you about some research that we did in Mexico in February in collaboration with Adele's team that recently known as Global Reach with comms, with reading design research and with community engagement. So the project goals were two fold. We wanted to first of all figure out how to do these kinds of research trips here in the wikimedia context so we can repeat them more on that later. And also more importantly, we wanted to start to understand how people in Mexico think about knowledge, learning and the internet. Particularly how that relates to our context here and around the world but for people who are really familiar with wikimedia so that we can kind of abandon some of those assumptions and start to learn about these people more directly. So we were particularly interested in digging at how people think about learning both online and offline. What kinds of devices they use platforms. Do they like web versus mobile apps? How much internet access do they have? Are they constantly online? The way we all are here, checking their phones all the time. Are they on wifi? What, if anything, do they know about wikimedia and sister projects? So, in order to do this, oh, you can't see my map. There's a map of Mexico there guys. Imagine it. So what we did, there were a team of six of us. It was me, Abby, Daisy, Joaquin, Shira and Jorge, a diverse variety of skill sets. We went to Mexico for two weeks. We only did in Mexico City which as you guys probably know is an enormous city. We then went to Puebla, a smaller city, and then north into Fluxcala which is a state that's more rural, more agriculturally driven with smaller communities in general. We asked a set of questions. It was about 50 questions took between an hour to two hours. We met with 15 recruited participants who don't really know a lot about what we do. We met with them in their home so we could hear about their context and see their devices, see how they use things on the day to day. These people were a mix of men and women, a mix of people who were working or in school. Many of them had really consistent internet access like most of us are used to. Some didn't, some had intermittent or barely got online at all. And their ages ranged from 17 to 41 years old. We also met with a number of subject area experts so Doc James was awesome and introduced us to people, local community members in Mexico, Lee Thelma Daughter and Nancy Gertrudis were really helpful. And we talked to people about areas that they're really familiar with, particularly around some use cases that we might be able to explore. Medicine, education, and also communities that are just getting internet access right now. So then you guys in the office have probably seen this if you've been on the third floor in the last while. It looks crazy. So after each of the interviews, we took our observations from onto these post-its and gathered them and put them from each interview onto a bigger piece of paper. And then from that, we're able to look across those and try to find patterns and extract those observations into findings. Those are these purple post-its here which is what Abby's gonna talk about right now. Thanks, Ann. So, oh, I need to click right here. Okay, so I'm gonna talk about some of the things, the observations we had, some of the patterns that we saw. Thank you, thank you, Provena, in these interviews. So for internet and technology use, most of the people we spoke with have multiple devices. They all have 13 of the 15 have smart phones and two of them have feature phones. Many people rely on Wi-Fi at home for downloading content. The cellular networks aren't as reliable and nor is Wi-Fi in public places or at school or somewhere like that. Also, many people have laptops also, but they don't leave home. People don't necessarily, at least the people we spoke with, don't take their laptops out in the world. And what else? Oh, also, everyone who had a smart phone uses WhatsApp a lot and also Facebook a lot. And everyone, most of the people talked about using YouTube to learn how to do something. So YouTube came up a lot in our conversations. In this picture, you can see people at an internet cafe and as we were walking around the place we visited, we saw a bunch of internet cafes that most of the people we spoke with hadn't really used them that much. So about trust of content, it was interesting to see how people came to trust the content they were looking for online. People will look for something and read it from one source and then if they see it several places, they come to trust it and they think, okay, I trust that content. Most people said they don't trust the major media. One of the participants we spoke with said, I get my news only from my friends because they know it and I trust them. Also, a bunch of people described that they get news from Facebook. And so this is curious to us. We'd like to understand what that means better. We can do some follow-up conversations with people. Is it because people trust the people who they know on Facebook and then they trust the content that those people are sharing? One person also mentioned that if there's a whole bunch of likes on a post, they might trust it. So this is something we're curious about that I want to dive into a little bit more about. Okay, next. So six of the 15 people we spoke with were either in higher education or in trade school or high school. And we also spoke with 24 students who are in med school. These are some of those students here. So all of the teachers or almost all of the people who are students said that their teachers said advise them, don't use Wikipedia for school. But all of the students do use Wikipedia for their schooling. And it was interesting to kind of note that people in higher education, they had a different concern than the students who are in high school and trade school. So in high school and trade school, the teachers had repeatedly seen copy paste plagiarism. So they're telling their students, don't do that. The students use it to learn a definition or get a high level idea of a topic or a word or something. But in higher education, the concern was a little different. People use it the same way to get a definition or to have a high level understanding of some topic. But then when you get deeper, like if it's a scientific principle, they're not necessarily trusting the deeper content about how you apply that scientific principle or something like that. So there's a little bit of trust in the deeper content for the different concerns about trust for different students. So we also learned about people's mental model of Wikipedia and how articles are written. We asked in our interviews how directly, how are Wikipedia articles written? People, as you can see, seven of the participants thought that experts write articles. Other people thought that there was a process to vet the content before it was put on Wikipedia. So we would then, at the end of the conversation, tell them, well, it's not written by experts necessarily. Anyone can edit Wikipedia. And a few people, whoa, whoa, I like had reactions like I don't know if I can trust it anymore. So we would then have a conversation with them describe referencing and how there's robots that sometimes look for vandalism and help human curators understand if something is okay, that it's vandalism or not or usable content. They were on down a little bit. But this made us think about what if we up-level the understanding of referencing so people can see it more at a more high level. And what about how can we communicate more clearly about the quality of an article? Like, is this article fully written? Are there enough references? We have the banner, but I'm not sure people see that all the time. But this is another area that we wanna dive into more and start thinking about. That start, I'm sure people have been thinking about for a long time, but we wanna focus on it a little bit. So four of the people had edited before. Only two did so in good faith. But two of the participants built biographies about themselves. And then it was deleted and they were okay with that. It was an interesting thing and a fun thing for them to do. Another person who, he worked on writing an article about Tlaxco where he lived with a group of friends and they did a lot of work on the content and got it to a high quality where they were confident that they thought this represents our town. And so then they moved on and they haven't edited or at least he had not edited before again, but he was happy with the work he did. One other participant edited the Transformers article when he was a kid. Unlocked down, no, I noticed, but I think he did in Spanish. So also finding content online, as we all know, Google is a gateway to Wikipedia a lot of the times. We observe this in these participants also. And also most participants had very little trouble finding any content that they were looking for online or some cases where people described, oh, it's really hard to find this or that, but most of the time people get to what they need online. And then, so all of the participants were fluent in Spanish. Few were very fluent in English and there were degrees of fluency in English. Like some of the participants didn't speak English very much at all. So we observed that when people described how they would search for something in Spanish and then they would maybe not find the content they were looking for, maybe they didn't find a way to define what they were looking for. And then if they didn't find it in Spanish, then if they're fluent in English, they would put the English word in the search bar and search for it and then maybe find that content. But if they weren't fluent in English or they didn't know the English word for that Spanish search that they're looking for, they would employ Google Translate, then translate it into English and search. And then they would find stuff in English if they are fluent, that's fine, they're reading it. But if not, they're gonna again employ Google Translate, click to that link that's in English and use Google Translate to understand that content, either a quick word or two or a paragraph. So this area is really interesting for it's sparking our interest and I think other people are interested in it too about how can we support multilingual people or frankly anybody looking for content from one wiki to another, even if it's not the language that they necessarily speak. How can we offer the content in a language that people can't understand? All wikis are not the same as we all know, like English wiki has a set of content, Spanish wiki has a set of content, they're not the same content, there's a big variation in the content. Okay, so I'm saving content for later. So people like I said earlier, many times are reliant on downloading data and heavy data downloads with their Wi-Fi at home. So part of the reasons they don't wanna go over their data limits, part of the reason is that it's not really reliable at school or in the public square where there's free Wi-Fi to get big downloads. So even students had described downloading full books to be able to read it later and not have to pay too much. So this is also an area that we're interested in, like offline access. One of the participants had the Wikipedia app but then he deleted it because it wasn't offline. So is that before it was offline available? Was he not able to understand and save for later? Is offline? We need to understand that. And also he had downloaded some of the other offline Wiki apps but then deleted them because he didn't like them. So anyway, this area is also a really interesting and potentially fruitful area for us to support people in getting content available offline. I'm gonna pass it back to Ann. All right, so Abby mentioned a number of these areas that we're interested in potentially focusing on both for more research and potential opportunities for comms, for partnerships and for product to explore and trying to serve our readers better. The reading team is already working ahead on some on load speed, size of content with performance improvements, also on the language issue with raising the visibility of the language switcher on mobile web. But there's a lot more that Abby mentioned that we can start to explore. These are just some of our ideas and you can see all of them on the Wiki that we're building out. So just in the vein of wanting to learn more as we're in the early planning phases of some research trips to Nigeria and India, I sent out an email announcement about this yesterday to WMFOL and to WikimediaL. So you can find that and click through to learn more. Get in touch with us about that and we'll also be doing a longer brown bag about this work in particular. Sometime in this quarter, we have to schedule that and we'll send out an announcement about that too. All of the findings, we're working on getting everything, the raw content, all the analysis, all the findings, everything online on this media Wiki page that we're working on. It's taking time, but please feel free to check it out. Let us know what you think. We'd be happy to hear from you. And I will pass the mic on to Dan. Thanks. So I'm here to talk about metrics. So a little bit of an update from Discovery. We, on the 10th of March, we pushed changes to wikipedia.org, portal site, launched new search and space improvements. So you're seeing that here. Previously, if you typed in a search query, you just got a list of article titles, nothing else, just text. And now we've added in images, we've added in wiki data descriptions, and we, well, why would we want to do that? Why would we want to improve wikipedia.org? That's why I'm going to explain here. So Abby actually did a very good job of pointing out that multilingualism is important and that a lot of users might speak one language, they might not speak another, they might speak two, they might not be aware that there is a wikipedia in their language, they might not be aware that there's a wikipedia that's in English, you know, things like this happen, or even that they're necessarily separate wikis. So having a multilingual portal to enter into wikipedia is very important. And it's not just some page somewhere, it actually gets around 14 million page views per day. It's a fair bit of traffic. So this isn't just something that exists and no one uses, lots of people use it. So why search though? Of all the things we could do with wikipedia.org, why search? Well, search is actually the most common action that users take on the page. So if you look at this graph, by far the biggest action that people take is doing nothing, just leaving. And search isn't right here. And then there's other things like the primary links and secondary links. So the primary links were a couple slides ago, the links that are around the globe and secondary links typically below the fold, but not always depends on your resolution. So what were the effects of the search improvements? Well, the early data actually shows an increase in the click-through rate as predicted by the AB tests. You can't really see it on this graph because of the magnitude of the effect, but it did actually happen. So this tells us good UI and UX actually makes a quantitative difference to people's ability to find things. This wasn't just adding an image and oh, it looks awesome. It adds an image, it looks awesome. People that previously did nothing now search and actually engage with the search results and go through to the site. So the change, as I said, it was small, but it was statistically significant. So, and it's good that our AB tests that predicted that this would happen actually was correct. So that's why we pushed this to production. So this graph is kind of confusing, so I'm gonna explain it. So localization matters. So behavior on the portal is actually very, very different for people that don't speak English compared to people that do. So how we quantified whether someone speaks English or not in this graph is to look at their accept language header. So that tells you whether they have English set as one of the languages in their browsers. So it's not perfect, short of actually going and asking every single person. We'll never know for sure whether someone actually speaks English or not. And it's interesting that people that don't have English in their accept language headers actually search less on the page and are more likely to click on the primary and secondary links, which take you to particular language wikis. So, do these people, are they affected by the fact that search isn't working for them? Is it that they're actually using wikipedia.org to go to the Spanish wikipedia or whatever wikipedia? We're not sure exactly why that is, but it does tell you that this page which is just in English, the people that don't speak English interact with it differently. So that's something we're definitely going to have to look at in the future. And now I'm going to talk about the completion suggestor, which is another search improvement which was pushed out on the 17th of March. So what the completion suggestor does, this is a wonderful if or jif, depending on how you like to pronounce it. Previously, the completion suggestor that's on the top right of articles was completely useless if you made a typo, unless a wikimedia had already created a redirect. So this is a kind of implausible typo but it illustrates the point here. And so this is what it used to look like when you type this in. You got nothing, you had to do a full text search. And that's a big deal when you consider that, as I mentioned, this is all wikis and platforms. So if you're on your phone and you made a typo, zero search results, unless your phone could correct it for you. So now search is actually a lot better at that. So this was pushed to all wikis except wiki data, for technical reasons, and all platforms. So what I mean by all platforms is the search box at the top right and desktop, search interface in mobile web, search interface in mobile apps, link inspector in visual editor, link inspector in flow, pretty much anything that uses prefix search. So why are we really concerned with the completion suggestor? Well, the zero results rate is the proportion of queries which gives users zero results is problematically high. You can see here it's around like 37% at the minute. From previous talks, you've probably heard me say that it should not actually be zero percent because there are a lot of people searching for things that just don't exist and it's correct to give them nothing. A good example is there's a tool that searches for a particular DOI that you can put in. It's like the ISBN for a scientific paper. And most DOIs that people are putting in are not in Wikipedia. So telling them that it's not in Wikipedia and giving them zero results is correct. But still, 37% is too high. So what were the effects of the completion suggestor? Well, you can see the blue line here is the zero results rate for prefix search. It dropped significantly when we launched the completion suggestor. It's actually lower than full text search right now. So we've seen a significant drop in the amount of queries that we're giving people that have zero results. And this, so that graph that I showed you before the zero results rate, cut off the end, because otherwise I would reveal this. The zero results rate has actually gone lower. This is the overall zero results rate for every kind of query. And the click-through rate on search results increased. So this wasn't just us giving people results that weren't useful to them. They actually engaged with them so you can see the increase here. And people were staying on the page so they weren't just clicking on the results and bouncing back. So this tells us not only are we not giving people zero results anymore, but actually finding them useful, which is nice. And there is a slight data collection issue. So don't worry about that. So we actually fixed, the completion suggestor was a beta feature, or beta, depending on your choice of accent, for several months. And we actually fixed a number of bugs in a number of different languages. So the suggestor actually ignored stress marks in Greek. It didn't index cross-name space redirects correctly on the Hungarian Wikipedia. And there was a one-off technical issue that broke it completely on Hebrew Wikipedia. By actually having this out in beta and getting feedback from people, we fixed all of these issues. I tried to thank people in their languages. You'll have to tell them whether I did it right. I did get people to check for me, yeah. So yeah, thank you. And if you're interested in finding out what we're gonna be doing next quarter, you can check out our goals. You can email us. You can contact me. And I'd like to thank everyone in the Discovery Department who made this possible. And one of the people that made it possible includes Thomas, who will be speaking to you next. Do you need the clicker? Nope, okay. Just fixing the batteries in the microphone. Please be patient. Excellent. Thank you so much, Brandon. All right. So as Dan was able to tell you, Discovery's been spending a lot of time really trying to understand what our user habits are or international reaches and how to make search better. But we of course know that search is only one of the ways that people access information. And maps and graphs are some of the other areas I've been thinking a lot about and trying to improve as a visual way of organizing content. So today I'll be giving you guys an update on where maps have gotten to. We've done some amazing improvements on what we're calling the interactive team right now. That's Yuri, that's Max, and Julian who has joined the team recently. So I'll show you some things that you've already seen. Hopefully some new things that you haven't yet. So we're starting on mediawiki.org on the help extension cartographer page. Cartographer is the media wiki extension that gives you access to two really interesting tags that you can use on the wiki right now. Map frame tag and the map link tag. We'll be going through both of those. So let's start with the map frame tag and the getting started section. So taking the chuck of wiki text and I'm editing on my user page here. Of course I'm gonna be doing all of this live so let's see what explodes and doesn't work. I love live demos. Who knows what'll go wrong. So out of that little bit of wiki text, everything goes well. You can see a map show up on the right side. We did great. I can interact with the map and zoom in. I can zoom out. I can pan. You can see our tilesets from our OSM dataset loading really quickly and looking beautifully. But let's say I don't wanna use wiki text. Let's say I've been using the visual editor and I wanna go in that modality and think about I do the same things. So pick a point here. I'll insert it up here. And click on insert. Go down to more. See that in partnership with our friends on the editing team, we've been able to integrate visual editor maps as well. So let's say I wanna insert a map my home country of Poland. Let's say I'm feeling especially national pride and wanna make it a little bigger. You'll see that as I update the values, everything is changing dynamically. I hit insert. I hit save. I think it goes right. It's there. You can see the new VE map. You can see the wiki text map. We have to give many a thanks to the editing team and especially Ed on the team who worked with URI to get this collaboration happen. First, interact it. And now, just a little bit of visual polish. You can also see a nice beautiful large map where our tilesets are showcased. Everything works just as before. So that's the map link tab. Now, let's take a look at the map frame where we start to be able to put on individual points. Let's start actually working with the map. So let's take a look at this one. Yeah, it has matched correctly pointed out map frame. I'll get to map link at the end. So now let's add a point. So we added a map here on the right side for one of my favorite places to visit in San Francisco, the Exploratorium. You can see that we're able to add an individual point and even add an image for it. Now, what does this look like? What does this look like under the hood? Well, let's take a look at the visual editor. And what I have as an option is a little edit button on the right. It may be a little hard to see since it's coming in right at the very bottom screen, but you can see that I'm seeing all of the information about the map point right in there. And let's say that I wanted to be able to start adding a couple more points to this area and highlighting the Exploratorium in that part of town because it's one of my favorite parts of town to visit. So start off by using the interface and add a little square here to draw the user's attention about what's going on here. I hit done, I hit save. We'll hit a little bug after this is done, I'll have to refresh. But after that's done, you can see that I have that main point there. It's edited. Let's keep going. Let's go back to edit. Let's say that I want to be able to change the, let's say I want to change the color of this point. It's blue. Let's go back up here. Live demos are fun. So let's say I want to be able to change it to a yellow color. I type right within the editor, our icon gets updated. Let's say that I wanted to make it a little smaller, give it a little less prominence. I can type that in and we'll fix that. And it changes right within the size. I hit done, hit save, refresh, everything's working just as it should. But let's say I'm taking a trip to the exporter, I'm really highlighting that area. And I want to be able to identify a parking space, right? Let's say that we're driving to that area. You can of course use public transit, but let's say we wanted to highlight that area a little bit better. So what I can do is I can select a point. Let's say there's a parking spot there. I'm sure there isn't. Don't trust me with that at all. So I get a point and if I scroll within the editor, you can see that I have all of the data for it here. But I don't have any information about it. I don't really have, I haven't set a color for it. So let's start adding some of that. Selections are tough sometimes. So let's go ahead and add a marker, dash symbol. Let's put that in quotes. I want to make sure that we form a json here. Let's call this parking. You can see there's a little key that dropped in. Let's keep going. Let's say that we want to add a marker. Let's see, yeah, it's just out of size there. Let's make this just a little bit smaller than, actually, let's make it bigger. Let's make sure that we want to find our parking space. So we made it really big there. And let's add a marker color as well. And we'll make this one, let's just make this one, let's make it a big one there. And I can hit done. I can hit save. We'll refresh after. And you can see that all the points are there. It's still interact with the map and it's really, really easy to do. Last bit, let's say you're working on a wiki page and you really like the idea of this map, showing you everything that you need, but it just takes up a lot of space. And you want to be able to make it just a little bit smaller, about a little less prominent, but still not lose the effect. So, what you can do is you can take the map frame tag and change it into a map link. We change that. And we change it down here. We hit save. Everything has worked out well. We should get a set of geo-coordinates that are showcased right on the map. You've seen wikipedia pages at the top right have something very similar. We're looking for that same experience here. You click on it and you have a beautiful full screen map that has all of the data that you've added, obviously. These are some of the early features that we're adding at the request of our users. There's still so much more that we could do. Some of these are more friendly edges so we would love for you to keep playing with this, giving us feedback and we look forward to finding more ways of our users visually browsing a lot of our content. Thank you. Awesome. That actually was interesting. So that was good. You're well, yeah. Thank you very much. So now is the time when we take questions which we'll go to IRC first. James Alexander, do you have any questions? Yeah, we do. So I think starting up actually just for Katharine at the beginning was wondering where the $2 million in savings came from. I'm coming from pie and this is the first big question that I'll put. Please be like. Don't say Tony in the audience. No one from the finance team. That's convenient. Tony's back there. Tony, you're hiding. So I'm sorry, I'm a little slow today. I hurt my back. The $2 million, well, it came from you. Thank you all for being such thrifty people who are such excellent stewards of donor dollars. In all honesty, that is exactly where they came from. We took a look and thanks very much to the finance team for modeling some of the projections for this year and we found that we were actually underrunning slightly against our $65 million budget. So we were looking at some of the places where we tend to underspend including in areas like travel, for example. And we went back and worked with all the different teams and departments to identify places where we could sort of extract cost savings. So very broadly speaking, I think we took a 20% cut to most of the department's travel budgets. That sounds really big, but as it turns out, most folks weren't actually using their travel budgets anyway. We had been calculating them in different departments on a per person basis. And so we're just looking to get a little bit more sophisticated about some of that financial modeling. Some other areas were in outside contract services. For example, as the organization has grown more mature, we do more things internally. And so what are the things that we can continue to do internally and take on versus working with outside vendors? There are lots of different other areas. And I just wanna say a huge thank you to all the different departments and teams for their collaboration on this. You really did actually turn over your couches and shake all the cushions and find the places where we could have savings. And that has allowed us to really preserve our focus on our programmatic work and make sure that that actually gets the resources that it needs while just being a little bit more efficient across the board. So $2 million sounds like a lot. It is, this is a really significant accomplishment for the organization. And we should all be proud of the fact that we worked together to do that. Congratulations to all of you. I am sure, and when we release the annual plan tomorrow, we'll have a more detailed breakdown that talks about specific areas where we've done some offsets. And so please do take a look there. I don't have it all just at the top. More questions from James Alexander? There they are. Next we'll be for Deli and the partnerships. We actually have two sawdust put them there. One was if there were any plans for China yet, we were looking at it there, and also sort of what the definition of Eastern for the position. Sure. So in China, we are attending a Shanghai event. It's really still focused on mobile operators and handset manufacturers. And we are trying to get our app being preloaded, but not much more than that. We are coordinating with Chuck in the legal team and trying to understand the China scenario better, but not really beyond preloads. And for the Eastern Europe definition, we are, so Sridhi is the partner manager that is working there. And we work with, I think it's Vimpo Com. I'm trying to remember the two huge mobile operators that we work with there. It's Vimpo Com and Talonore, and they have some countries that are considered Eastern Europe, and that's why we added that to Asia. So some of them, we are working with our community and asking them if they actually are welcoming Wikipedia Zero there, because they might be part of the European Union. So if the country is part of the European Union, they cannot really have Wikipedia Zero partnership. So that's our cutoff. All the countries that are considered Eastern Europe that are not part of the EU, they are being added, and the countries that are part of the EU are not. Now, that'll be it from me for now. Any other questions? Come to the next voice. Hey, I have a question. I don't know who should answer this, but maybe like Adele or Anne or Abby. It's about the research that you guys did. And I was wondering if we have any plans to run any advertisement ads in emerging countries to kind of tell about what Wikipedia is, because I feel like there's some very common misconceptions about Wikipedia, especially in those countries. I think advertising there should be a lot cheaper than running an ad in the US, probably. Yeah, I can take that. I'm not on the COMS team, though. So we're doing some research. That's where we're at right now, is trying to learn if awareness is really the problem, and the surveys are helping with that, and we're getting a little bit more qualitative as well. Looks like Zach might be making his way up here to actually answer this. Yeah. Hi, I'm Zach McKeon. I just did a Kanye. I'll let you finish, but I'm gonna let you finish more as after this. I'm Zach McKeon on the COMS team, and definitely a part of this project on new readers. Very excited to work with Adele and Anne and Abby. And the short answer to your question is yes, but the longer answer is when and how, and that's something that the research is informing. So we hope to find different ways to raise awareness and then raise usage in these countries, but we don't wanna rush in to do that. We wanna do that by understanding what are the current perceptions, what are the current misconceptions, and then making messaging that can help that. Otherwise we think we'd be running in and we wouldn't necessarily know if that's a good use of funding or time or energy. And we also definitely wanna work with our communities to make sure that they can support what might happen in terms of an influx of editors or an influx of readers, so that this is really buttoned up from community to messaging to our partnerships and product teams. So that's what's gonna happen. Mewzer. Hi. So I had a question about annual planning and one of the things that keeps floating on top which I'm really happy to see is the user experience and that's gonna be the focus. So I just wanted to see if there was any basic plans or any force of direction for basically achieving one of the main goals that we have for this year. Yeah, I'm not sure. I'm 100% the right person to answer that because I think it's kind of on a team-by-team basis. I would be happy to, someone wants to pull up the annual plan document who has actually overloaded this morning with all the folks working on it. But yes, the strategic, first the strategic, excuse me, first strategic priority is around understanding our users better and I think that is a combination of user research and some of the work that the research team is doing, both including the user research team but also, I don't know if there is in the audience today, some of the work that the research team is doing more broadly in order to understand some of the needs and priorities for different, for different types of users and some of the work that the cross-collaboration team is doing around how do we reach, for example, will include some of the product research and user research component. So yes is the answer. There are a lot of different, the way that the annual plan's broken out is, I'm sorry, it really is like 60 pages so I'm trying to remember all the different goals and objectives but it's broken out by programs and then by objectives and then goals and the program speaks at the very high level to, for example, user research or understanding from a testing perspective or doing quantitative or qualitative research and then they set out objectives that are specific to either the product in question or the team in question and then they get into goals which are what are we actually trying to get? Oh, no worries. Then they get into goals which is what are the things that we're actually trying to achieve and how are we gonna measure for being successful? So apologies, I don't have it quite at the top of my head. I think there are folks who can speak to it in greater detail than me but it will all be, it is actually mostly on OfficeWiki now and the document is open to the public so I can share it over with you when we're done. Just a follow-up on that. So I think there are two parts to user experiences when one is understanding what people actually want. The other one is improving the experience of products that we already have and what I'm hearing right now is basically this is more on the research part, understanding the user part, knowing what to build, those sort of things. I am happy to tell you it is actually both. So the research team I was looking in, do you wanna speak to this Abby? Cause I know it's part of your core work. Yeah. So we are gonna do a bunch of value of research to iterating with the designers and the design teams towards improving the functionality. So that there's half of our work, I don't know if it's exactly half but we're doing generative research where we're going on understanding needs and how to better improve the experience, what the experience is and what some problems are and then also when we all kind of evaluating concepts and evaluating functionality that we have already to improve the current experience. So for the design research team we're gonna do both of those kinds of research very collaboratively with the design team, all the product teams and the designers. Abby, if I can just that. So just to pile on, but this is super important for the product teams as you know. There's actually one of the strategic initiatives that came out of the community consultation was understanding our users. And so Abby and I have written some goals around that. It's in the product narrative. So definitely check that out on Office Wiki and make sure that you're down. Thanks. I'm just gonna make a plug. I know I mentioned it earlier but this after metrics is over in about two minutes everyone's gonna be able to go get lunch like usual but then the finance teams are facilitating and actually thanks to Lynette and Toby and Trevor facilitating some narrative sharing. So if you do have questions as what's in the different teams and departments narratives this is a great opportunity to hang out and ask some of those questions and learn about what it is that all the different departments are gonna be doing. So might be able to get into more specifics in that venue. More questions? I'll just add, yep, we're about a time here but all of the narratives are also available on Office Wiki to read. So if you wanna do that async it's also available for you. And that's it. Everybody thank you guys for coming and enjoy lunch.