 Hello everyone, that's all looking down at their computers. Welcome to the May metrics meeting. For those of you who don't know me, I'm Sati. I sit on the community resources team within engagement and I will be your DJ S-A-T-I for this metrics meeting. The theme of this meeting is going to be something that's become a growing part of our conversations in the last few years and definitely come to the forefront recently. And that is inclusivity. How we don't just say everyone can edit but actually create welcoming and safe spaces for people. How we don't just say everyone can participate but ensure that their voices, their ideas and their contributions are seen, discussed, amplified, reconstructed, improved in a way that makes them feel that they are a part of the space that they occupy. And so what we'll talk about is a movement update from Maria Cruz who will talk us through some work going on with Wikimed and R Plus Feminism. Then Joe will talk us through some work that's going on with the collaborations team. And then Emily Templewood, or as you might know her user Kay Lana and Erin Haffeker are gonna talk about an interesting story of the intersection of their work that's been going on them over the last few months. And then our wonderful ED will have an executive update and then I'll DJ us through some Q&A and Wikila which I expect, Wikilove from all y'all. So with that, welcome to a few new people who have joined us recently. Eileen, James and Valona who join us in Legal and TNC. Welcome. As well as Sohyun, Dina, Diana, Alex and Lisa who are joining us across the board in Product, Legal and Talent and Culture. And Anniversary. So, for those of you who don't know me, I suck at names. So I did research to ask people how to say their names because my names get butcher all the time. I also usually don't know all the people who sit on the slide so I've just kind of added where they sit so if I'm wrong, that's metasphal, just saying. So at eight years, we have our anniversary for Roan who sits in editing. At six years, Pat's in fundraising, Maggie in community engagement, Daniel in technical ops, and Katie Horn in discovery and fundraising tech. And then at five years, Sabu who also sits in editing. And a lot of people still to go. So Anna Launce in talent and culture and Dros in technical ops. Congratulations of being here four years. And then at three years, these are a lot of names. Filippo in technical ops, Bernd in reading, Mukunda, and Daniel in release engineering. Abby in design research, Maria in learning and evaluation, Elena in talent and culture, and Dylan in fundraising tech. More to come. Volker in editing, Casey in fundraising tech, Jaime in technical ops, Sam in editing communications, Benoit and Johan in the community liaisons team, Darian in security, and Joan's support and safety. Two years with that. And last but not least, Vahid in education, Leon in community tech, and Joe in mobile apps. Congrats. And Emily just pounding in the background. We love you. Wiki love all the time. So then I'm gonna hand this off to Mira Cruz who's gonna walk us through and move an update. Hi. Happy 25th of March to any Argentinians who are watching this. And Moriel, who might or might not be here. My name is Maria Cruz, I'm communications manager at the community engagement department. Never know where to stand here. And for the movement update today, we have two community for centers, Shani Evanstein, who is a medical educator at Tel Aviv University and also vice chair of the user group Wiki Project Medicine Foundation. And Mackenzie Mack, who is co-lead of the Art Plus Feminism Global Campaign. So without any further ado, Shani, you are first. Hi guys, can you hear me well? Yes. Yes. Perfect. So thanks, Maria. Very happy to be here today to present Wikimedia, which is an academic for credit elective course that I've designed and lead since 2013. To understand why Wikimedia is important, consider two things. The first is that Wikimedia has become the number one source for medical information online. However, we're still very far from good coverage of medical info. English Wiki has about 35,000 articles, but other languages are very far behind. Hebrew, for example, has less than 3000. The second is that from an educational point of view, Wikimedia can help medical students develop skills, literacies and competencies that are required from tomorrow's physicians. However, to do that, we need time with students. While we have many educational collaborations worldwide, we're usually limited to one to two meetings, which is simply not enough. Next slide, please. That is why I developed the first of its kind semester long course that is dedicated to creating high quality medical content while helping students develop various skills that are needed in the 21st century. So far, the course had about four iterations in which my students wrote over 200 medical articles, which is about 8% of medical content in Hebrew. And those articles got about 4 million page views. So that's actual impact. As you can see, in addition to writing articles, we focused on various skills and the result is a very positive and impactful learning experience for students. Next slide, please. The course is being inclusive on various levels. First, almost 50% are women, which is great. And students speak a variety of mother tongues, Hebrew, Arabic, Russian, and French, so quite a diverse crowd, just the way we like it. Another aspect of inclusivity is that I don't only teach Wikipedia, but also discuss the open knowledge movement or the media movement and other wiki projects like wiki commons and wiki data. I also make sure to collaborate with other projects in the movement like wiki project medicines translation efforts and Women in Red by wiki women. Can you believe we didn't have any women's health category till this year? So it's important to note that in 2015, a second course opened at Tel Aviv University based on the same model. So now every undergraduate student at Tel Aviv University from any discipline can take my wiki course. Hopefully the course model will be adopted in other institutions around the world. Finally, some exciting news that you'll be the first to hear. Next year, I'll be opening a wiki interest group for the American Med program at Sakura, which will focus on Wikipedia, wiki data, and epidemic research around it. And what can you do to help? Join us in our efforts to enhance medical content in wiki projects because every single one of you counts. Thank you so much for listening. Thank you, Shani. And up next is Mackenzie. Mackenzie, are you there? I'm sorry, I still have myself muted. Can you hear me? Yes. Okay, perfect. So hi, my name is Mackenzie Mack and I am one of the co-lead organizers with a group called Art and Feminism. Art and Feminism is an initiative that was started in 2014 to increase coverage of women and women editors on Wikipedia, coverage of participation of women and women editors on Wikipedia. So this is me and I love metrics and activities. So it's perfect that I'm able to present for this meeting today. And before I kind of get into talking about some of the things that we've accomplished with Art and Feminism, I wanted to share a tweet with you that went viral about two weeks ago. And this tweet particularly was shared by someone who was looking at O Magazine and saw this photo essay by the photographer's name was Chris Buck and it was this photo essay called Let's Talk About Race. And one of the reasons that this photo essay was so compelling was because it was switching, reversing the roles of women of color and white women. And people found that to be particularly interesting to think about this world in which you show up to a store or you show up to work and you don't see yourself represented. And so when we think about that, the question that we have is what happens when people don't see themselves in history? What happens when they don't see books that are written about them? What happens when they don't see themselves on television? And they only see really histories and narratives of a dominant group or a dominant class. So in 2011, the Wikimedia Foundation did a survey and it found that less than 13% of editors on, less than 13% of editors on Wikipedia were women, right? So that's a really interesting number. And when you think about that, what happens to content when it isn't written by persons who have experiences or have similar experiences, for example, women who have very specific kinds of experiences that aren't written about that they're not writing, what you tend to see is then content that is skewed and content that doesn't cover them as often or doesn't cover their accomplishments. And you see histories in which those histories are attached to their husbands or attached to their lives as mothers, but we don't see as much of an attachment to things they've accomplished in their lives. So in 2014, a group of friends came together and out of this conversation about women and editing in Wikipedia, they started art and feminism. And so what we do is we hold editathons around the world and these editathons teach women how to edit. And so this year in the month of March, because we hold our events simultaneously in the month of March every year, we've had over 200 events and that has been in 42 countries with over 2,500 participants and 6,500 articles that have been contributed or improved, which is double the amount of what our output was last year, so we're growing. But when we think about the ways in which we wanna improve, our goal is to continue to grow in terms of getting more trans women involved in editing, more women of color, more queer women involved in editing on the platform. So some of the ways that we've done that is we've created online a toolkit for organizers who want to organize with us. So anyone at any time could have access to that toolkit and could organize their own editathon, can use the resources that we provide us so that they can hold events wherever they are in the world. And so that work has led us here. And so I'll end just kind of by saying that our impacts are really what we're really proud of and the work that we're doing currently is being able to have the opportunity to see new editors come into a room, not thinking that they could ever write anything on Wikipedia, let alone open a browser and go to Wikipedia in an effort to not really find information, but to add information, to see them go from not thinking or not confident that they have that ability to editing on the platform, telling their friends about it and seeing how important this kind of information activism is on the internet has a really incredible experience. So thank you so much for taking this time to listen to my presentation. Thank you. Thank you all for presenting. And we have some foundation highlights now. So in the past month, we had a media wiki documentation day and write the docs as a meetup. There are some efforts put into documentation for media wiki and we are going to be hosting maybe an event about this in the near future to share what we learned at Write the Docs and also to understand better how we can collaborate better on documentation. The 2017 board elections are, the voting has closed on May 14 and thank you all, to all of you who have voted because your voice matters always, all the time. And we welcome our new board members, Maria Sefidari, Marius, and Tokchains, thank you. I had a blank moment. We celebrated a media wiki hackathon in May 19 to 21 in Vienna, Austria. And the wiki media foundation engagement survey is closed on May 20. And this is a means to identify ways in which we can increase individual and ongoing organizational engagement with the foundation. And coming up in June, we have the funds dissemination committee. This is a volunteer group that, sorry, that seeks to vote on assignments of wiki media foundation grants. And Iberokonf 2017 is celebrated June 10 to 12 in Buenos Aires. This is their regional meetup for chapters and user groups from Latin America, Spain, and Italy. And with that, I pass it on. Testing, am I good? Am I live? Here we are. Hello. I am Joe Madazzoni. I'm the Collaboration Team Product Manager and I'm here to demo something today called the New Filters for Edit Review, which the Collaboration Team released recently to beta. It's a suite of improvements to the recent changes page designed to help edit reviewers and today's theme, new users. But first, I wanted to impart some wisdom that I learned in the course of this project from our esteemed colleague, Aaron Halfacre, whose work features prominently in the new filters. And it's that good is more common than bad. In fact, it's a lot more common, as you can see from the numbers on the screen and I hope that we all think this is true in our lives. But from Aaron, I know that it's true on English Wikipedia, where some trollers would be surprised to learn. I think that good edits actually outnumber bad ones by more than 99 to one. Aaron, of course, is the data scientist who is behind OARS, the Machine Learning Program. That's a big part of the New Filters beta. And this, in my roundabout way, brings me to my subject, which is that the new filters is really designed to around two main problems. The first has to do with this. And it's that, you know, it relates to the fact that every day on English Wikipedia, I'm sorry, every day on all the Wikis, there's human editors make a half a million edits, and that is a big pile of edits that need to be reviewed. And while it's reassuring to know that only, as we saw in the figures just now, 0.4% of them are vandalism or otherwise bad, that's about 2,000 edits, and a half million. That's also a problem, that's a needle, if you can't tell. And, you know, so problem number one is that we need to help reviewers to find the needle of bad edits in the haystack of good edits that are out there. And to do that, warning, metaphor switch here. Edit reviewers have always used power tools of different types, like this man who's clearing down trees with a chainsaw. But of course, you know, so power tools like Huggle or RTRC, et cetera, but, you know, that brings up the second problem, which is that when not used with care, power tools can do harm. And again, as many of you probably know, Erin has done a lot of research in this area, showing that new editors in particular who are summarily reviewed and rejected have a tendency, a much greater likelihood of withdrawing from Wiki work. And this problem, of course, is more acute when this type of power tools that I just mentioned are used, which can be a little indiscriminate. And of course, the power tools I'm talking about don't lead to the type of devastation we just saw, but they do contribute to the type of anemic new editor retention figures that we have seen in recent years. This is a graph of second month active editors, which means editors who keep editing a month after they review, sorry, registered. And as you can see, without going into a lot of details, it's not going in the right direction. So, over the last months, the collaboration team has been working with these two goals in mind, to increase the efficiency of edit reviewers and help them work through that haystack while also protecting new users. And we've done this. To do this, we started with the recent changes page, which is one of the main locations where edit review happens in MediaWiki, one of the primary tools. And the improvements we made fall into three main categories. We completely redesigned the interface. How do you know? I know, over your sales last day. Completely re-imagined the interface so that it's both much more user friendly with a lot more description and logic to it, but also more powerful in that it now really gives editors a lot more control over the types of edits they include and exclude in their searches. We added a lot of new features that I'll demo in a minute. And this is really the first project to maybe most notably or most excitingly. It's really the first product to fully make or is available for use by general audiences. We put a lot of effort into making it easy to understand and to use. And in particular, it's the first time that Orr's Good Faith test has been available to the public. And of course, that's important in finding the users that we just talked about. So now I'm gonna go to demo. If you can switch me over, Brendan, you got it? Okay, so that's the recent changes page. For those of you who aren't familiar with it, it's basically a search page. The bottom is a reverse chronological list of all the changes that have happened recently to the Wiki. We did not actually touch that part. Up above are the tools that let reviewers filter that and narrow their results. This menu, which is new, lists all the various filters that let you do that. Some of these are new. As you can see, if I make a selection, it's reflected up here in what we call the active filter display area. And that lets you get a quick read of what your settings are. This button here is the highlight menu, or brings up the highlight menus, which lets you add color coding. Oops, that was a bad choice. Color coding to your edits in order to, see what's one that I don't know. To highlight the things that you're most interested in, the edit qualities you're most interested in and make your results more meaningful. These up here are the filters that, sorry, are powered by ORS, the contribution quality prediction and the user intent predictions. User intent predictions makes predictions about edits as to whether they were made in good faith or in bad faith. The quality predictions make edits about whether the edits are good quality or bad, or have problems. Problems can be anything from outright vandalism to formatting or something like that. So both of these are making probabilistic predictions. As we said, we put a lot of work, or as I said, we put a lot of work into trying to make this understandable and useful to a general audience and the solution we finally arrived at was to sort of define a series of filters along a range of probability from, in this case, from good to have problems that are very likely have problems, going through may have problems and likely have problems. And now I'm gonna give you a quick demo of how you might use these tools, I'm gonna destroy the defaults, to do a little vandalism hunting. So I'm gonna get rid of these two because I'm just not that interested in them. For now, I'm gonna select the broad may have problems filter. So now everything I'm looking at is a edit by human that may have problems. So that's a good list, but I might wanna prioritize a little more, so I'm gonna use highlighting to do that. And I'm going to color code my problems kind of in order, I guess, of urgency. I'm gonna make the likely have problems yellow, the very likely have problems orange, and the very likely bad faith, red. So now everything I'm looking at is, as I said, a may have problems, but the ones with most intense colors are the ones that are really probably bad. That's page blanking. Here's a, I'm afraid to bring one of these up, because of course you never know what it might be. And just to show you the difference between sort of coloring it versus, I could have filtered it, very likely have problems. And I could do that by doing this, just doing its magic. And now you see, so you never wanna filter and color at the same time, is there? Now these are all edits that very likely have problems. And you can see over here in the left side, the dots that are multiplying show me the different characteristics that apply to those edits. So that's one way of using this, but as I said, for a second demo, I'm gonna do something a little bit different. As I said, traditionally, the Edit Review focuses on problems and bad actors. But of course, we're also interested in supporting people who are trying to be helpful and do good. So for this one, I'm going to, and so we've added a number of features that help highlight that type of thing. And I'm gonna use this now to find new users to thank and I'm particularly interested in new pages. So let's see, I'm gonna just leave up page edits and page creations this time. I'm gonna search for very likely good faith and very likely good. And I'm gonna highlight my page creation. By the way, if you ever forget what these are, there's rollovers that tell you all what the different filters do. I'm gonna highlight my page creations in blue. And my, just because I wanna know who's a newcomer or not, I'm gonna use the newly defined newcomer filters. I'm gonna make that green and I'm gonna bring another new category we defined called learners. They're also pretty new. I'm gonna make them green. Okay, so now everything here is a good edit in good faith by human, not a bot. And the page creations, we see any are blue, right? And if I see something that's blue and green, it means it's a page creation by a new user which I'm not seeing any right now, but trust me, it happens. So, and so that's how I might, and then I might go and decide to thank all these good users who are making good and good faith contributions to Ricky. So, back to the slides. So this is still in beta. We're still working on this over the next quarter. We're gonna add a lot of great new features. We're gonna, it's kind of complicated. We're gonna let people save settings. That should come out in a week or so. A reverted filter, which will be super useful, but might have to wait a little longer. We're actually gonna add a real-time updates thing to it so it'll work a little more like RTRC for those of you who know that. When we get this all working the way we like it, we're gonna bring it to the watch list and maybe some other review pages and a lot more other features that reviewers are still asking for. Want to give a little credit to the collaboration team who are an amazingly talented group whose working style matches their team name and of whom I'm very proud to be a member. That's a couple of Meredith members there too, but they're still in our hearts. And give it a try. As I said, we're still working on this. Very interested in hearing your opinion. There's links here to our talk page. We're very much listening and to the project pages and our fabricator list and all that stuff. And that's it for me. Thank you very much. Okay. Hello. Hello. Okay, so now over to Aaron and Emily Templewood for their piece. Aaron, do you want me to click for you or? No, I got this. Thank you very much. So today I'm gonna show you folks a measurement which is perfectly in line with the metrics meeting, something that I'm really excited to show you. But before I get into this, not everybody would be familiar with Emily Templewood. So Emily, why don't you introduce yourself? Hi. So I'm Emily. I live in Chicago or the suburbs thereof. I'm a first year, no, second year medical student. I just finished my first year. And I'm the founder of Wiki Project Women Scientists and kind of the, that's been my focus for the past five years on Wiki Media Project. I've been editing English Wikipedia for the past 10 years, 10 years and a couple months as of this April. And I also serve as an arbitrator and an administrator and some other fun stuff. So yeah, nice to meet you all. All right, and there's a big story that Emily is gonna have to tell you. But before we get into that, I'm actually gonna tell you, we're gonna put this together a little bit backwards because when I took this measurement, I wasn't that familiar with Emily's story. So I'm actually gonna tell you this from my point of view where I took some measurements, found some interesting stuff and then became familiar with things later. So you're gonna follow the process the same way that I did. Okay, so let's see. So let's start with this measurement. So there's this thing called ORS and Joe was just talking about this. Essentially what it does is it takes a Wiki thing and makes some prediction. And so like for example, it can take a diff. And so on the left we're seeing here an edit to a Wikipedia article about dog intelligence and it changes a reference to Lama's Grow on Trees. We can send that edit to ORS and ask it if that edit was good or bad and ORS will tell us this edit's probably bad. So, but we can also actually have ORS make predictions about a lot of other things in the Wiki such as an entire article. So for example, if we give ORS the article on biology and we can ask it to make a prediction on this Wiki project assessment scale which is a scale that Wikipedians use to assess articles by their overall quality level. And in this case, ORS will tell us that this article is a borderline featured article quality level. And so it turns out that currently biology is labeled as a good article, but that label came back in 2013. And so ORS is saying, you know, I'm not sure but maybe it's time to reassess this article. Maybe it's time to promote this article to a featured article class. And before I go forward with this I gotta give a shout out to one of my collaborators, user Netrom. This is the guy who runs this awesome tool on Wikipedia called Suggest Bot. He's a researcher. He's currently working as a fellow for the research team. And so there's this paper that he published called Tell Me More, An Actionable Quality Model for Wikipedia. And this model that we're using in ORS is largely based on his work. So I wanna talk to you a little bit about what quality looks like without ORS. So if you go to the top page of Wikiproject Biology you'll see this set of templates that I'm showing you on the left. These Wikiproject templates are used by subject focused Wikiprojects such as Wikiproject Biology to tag an article as within their subject space. They also use these templates to rate the article by its quality and importance. And so in this case, Wikiproject Biology said that this article is good article class GA class and top importance. But it turns out that this doesn't happen that often. And so the graph on the right is showing you over time what the assessments of this article biology look like. There were a couple of assessments that happened towards the end of 2006 just a couple of months apart. One put it at a B class and then the next one put it at a good article class. And then there was another assessment at the beginning of 2010 and that put it at a good article class. But the rest of this dashed line space is we just don't know. And so that assessment that happened in 2010 it was actually repeated in 2013. We just don't know what the quality of the article was after that point. And a big reason is because this is exhausting. It takes a long time to read through an article especially a relatively high quality article and assess it for its overall quality level. And so it takes a huge amount of work we just can't do it that often. However, with ORS and with this prediction model that I'm telling you about we can do assessments for every single version of the article. And so we have with ORS on the left we have three assessments for this article on biology but on the right we have, gosh I never actually counted this up. Let's just say it's thousands. Thousands of assessments of the article on biology and we can see a much more nuanced picture of how that article has developed over time. So there's a cool thing that we can do with this model. It's not just about assessing individual articles but we can assess content spaces. And so when I was first playing around with this model and playing around with ORS and measuring the quality of things in Wikipedia I thought that, hey what would be a fun content space that I could look at? I know that there are some initiatives around Wiki Project Women's Scientists but I don't really know much about them. I just know that that was kind of interesting and it came out recently. How about I compare the overall quality level of articles about women scientists to the overall quality level for all articles in the encyclopedia and what I found was actually really striking. So in this graph we're looking at that quality gap, the quality difference between articles about women scientists and articles across the entire encyclopedia. And so from basically 2005 until 2012, 2013 the gap gets wider and wider and wider and so Wikipedia is growing in quality a lot faster than this particular subject spaces but then something happens towards the end of 2012, beginning of 2013 and that shifts dramatically so that now we're looking at a very big surplus where the average quality of an article about a women scientist in Wikipedia is much higher than the average quality of any random article in Wikipedia. And so I actually presented this at the Wikimedia research showcase. This is a monthly event that we do from the research team and with our collaborators. This one was in the December 26th, 2016 research showcase. The title was English Wikipedia Quality Dynamics in the case of Wikiproject Women Scientists. And I got to the end of the presentation and I asked the question, does anybody in the audience know what was happening here? And there was this interesting interaction that happened on Twitter where SICO shows up and immediately says, this is SICO Virtus, said, wow, Erin Haffiger's research shows big increase in quality of women scientists articles starting in mid 2013. I think the reason is Kailana, which is Emily Templewood's username. And after a little bit of discussion, we decided that we were going to refer to this sudden shift in quality around women scientists in Wikipedia, the Kailana effect. And so what the heck happened here? Emily, do you want to tell us? Yeah. So as soon as I saw the data, I was amazed because around, I'm pretty sure November 2013 was when I first got involved in the area of women scientists history. And Erin showing Anne Bishop who was a really cool parasitologist and I could go on a whole long rant about women and malaria science, but I'll spare you that. She was a fellow of the Royal Society and for Ada Lovelace Day, the Royal Society had released all of their documentation about women scientists that year. So that's kind of the perfect starting point for me. And when I dug a little bit deeper, we found that there were thousands and thousands of articles missing, not just, you know, fellows of the Royal Society or the obvious candidates, but, you know, layers and layers deep. So I asked Sarah Sturch for her advice on how do you do this whole Wiki project thing, despite the fact that, you know, I've been a Wikipedia fan for five years already. And she and I got together and started Wiki Project Women Scientists. It was pretty incredible how we went from approximately 1600 articles at our inception to almost 6,000 now, if I remember my numbers right. And one of the things that really helped was this IEG, the Women's Scientist Workshop Development. So I ran a pilot program at my university, Loyola University, Chicago, where I built on the art and feminism type model that we talked about earlier in the session. To get small groups of women really invested and producing really high quality content in a really short period of time. So at each one of these workshops, we would create, you know, a dozen, 20, that, you know, number of C or B class articles. And this started to be replicated and, you know, the principles of how to, you know, bring people into the project on a really high, you know, achieving level very quickly, started to become more popular. And with the advent of Wiki Project Women in Red, founded by my dear, dear colleague Rosie at Wikimania 2015, we've seen a huge increase in quality because, you know, we've not only have, we created more articles, but we've created more quality articles. And yeah, like I said, at the end of my grant, it was a really hard road. But working with the grants team was incredible and having the support of the foundation made it all possible. I wouldn't, you know, I could not have bought that much pizza by myself, nor could I have, you know, done the project management and organization that I managed, you know, while in undergrad and without the support of the foundation. So it's a really nice example of, you know, foundation and community working together to produce a demonstrable increase in quality content. Yeah, so it's kind of awesome. So thanks, Emily. So, you know, as soon as I, as soon as like we had this conversation on Twitter, and, you know, I learned more about Emily's initiatives, I changed the figure as it was uploaded to comments. And so now if you go find the Kailana Effect figure, it has these annotations on the graph. And so, you know, reading through the blog post about Emily's efforts, I was able to, you know, put the timeline on this graph. And so it helps you understand just how abrupt the change was once Emily started her work. So, you know, we showed you the article about Anne Bishop and that blog post that was announcing Wiki Project Women Scientists. And I really think that the shape of this then just speaks for itself. So one more thing that I want to say is that I didn't tell you very much about this graph. There's a lot of work that goes into taking these measurements. And I'm sure that there are people in the audience who are really interested in how this came together and looking at how they might take measurements themselves. There's this page that's up on meta, interpolating quality dynamics in Wikipedia and demonstrating the Kailana Effect. So go check that out and you can see how this all sort of came together. Okay, so there's a couple of takeaways that I think are really important to take from this analysis and what we were able to recognize about these kind of outreach events. So the first one that I want to push on is, like this was really cool. I don't think that we really understood how effectively this grant money was being spent. I think that we were perfectly aware that Emily's efforts were being successful. But I think we should really ask the question about what other projects are being successful like Kailana? It's like what other Kailana effects are out there that we don't know about? Other initiatives that are like hers or maybe even not like hers that are also having such a substantial impact. And so this is something that I'm hoping that we can do with this article quality model and this measurement methods. Emily and I talked about building a Kailana Effect detector. And so maybe if I can find time around all my other projects, I'll dig into that. And if you're out there and you're really interested in this and you want to work with me on that, then that would be great. But the last word I want to leave for Emily. Okay, I'm gonna read it, sorry. So yeah, Seco is the reason this all happened. You know, I conceived of this whole project when I was at Wikimania, Hong Kong. And I want to thank her and all of you for giving an untested weirdo a chance because without that, this probably wouldn't exist. So thank you. And with that, super interesting story. I'm gonna hand it over to Catherine. Emily, you're amazing. Lots of wiki love to you. Amazing too, Catherine. Aw. I like your cat too. Thanks for that awesome presentation and I can't wait to see you again soon, wherever that is. All right, cool. Hi, I don't have a deck or a presentation. I'm just here to chat a little bit. I thought it might be a good idea to just check in. It's been about a year that I've been in this role and I thought it would be nice to provide an update and perhaps some reflections on how that year has been. And with apologies to our community members who are hanging out on IRC and watching this, I'm mostly gonna talk a little bit about how things have been at the foundation because we know that you're doing phenomenal work and I want to reflect, like, that's amazing. But I wanna talk a little bit about what's changed here at the WMF and where we're going. And I really don't know how to follow Kalaanik. She's totally awesome. So I have a couple notes and I'm just gonna pull them up. So the reason I wanted to chat, too, is kind of, I get this feeling that we've been moving really fast lately and that things have been, there's been a lot on. No matter where you look, people are busy. I hear all the time from folks that they have so much going on that it's sometimes hard to sort of take a moment to stop and breathe and I think that it's important for us to actually create these moments in order to recognize all the incredible things that are happening, right? Whether it is stories around the Kailana effect, whether it's a new, I'm sorry, not a catchy title. I hate to say that, purposely not a catchy title. That's great, okay, cool. New filters for edit review, thanks, Joe. All of the things that have been happening at the foundation and all the things that have been happening in the movement are actually really remarkable when you sort of stop to think about this incredibly vast spinning, expanding ecosystem that we work on and work in and support in this remarkable community that we're a part of. And the reason I wanted to pause today, sort of a year into sitting in the executive director's seat is because a year ago we were in a really different place and I just want to sort of take a moment to acknowledge that, right? We just heard this presentation about what Kailana has done and we talked a little, and she was talking about how important it was for her support from SICO and it gave me this moment of sadness because SICO's not working with us anymore. She's still doing great stuff in the movement but she's no longer day in and day out in our offices and part of our community here as a staff community. And a year ago it was, we sort of found ourselves in this tough place. We'd lost about 30% of our board members, almost 80% of the executive team and a number of other folks who are big holes even today, folks in bases that we miss every single day. And we've come a really long way at the same time and that's just sort of what I wanted to speak to and we needed some time to sort of pull ourselves back together and I am excited to see that that seems to be happening. So we just took the engagement survey results a year in. They are actually really consistent with where they were a year ago, which is great. We've made some real sustained and lasting progress. When I started in this space, I just want to say thank you too for supporting me and trusting me to step into this role. It meant a lot. The board gave me three or four, excuse me, priorities and responsibilities and those were to rehire the executive team to launch a movement strategy process, to partner with the board for its success and to write the ship at the foundation. So any one of those things in a year might have felt like a lot. So it's been a really busy year and I just want to talk a little bit about from my perspective what we've accomplished because that is a ton of stuff. And the first is really just around building structures and improving the way that we work at the foundation. I was amazed when I joined, I think there were like 180 people. We're now at 300 people and we built a huge amount of that growth out in those last two and a half, three years that were really hard years. And a lot of that structures that you need to sustain an organization as it goes through that transformation weren't built alongside that growth. And so part of what we've been doing over the course of the past year is trying to rebuild some of those structures, try to understand what has been missing and backfill in ways that allow us to run our organization more effectively, provide support where we need it, understand how we like mentor and grow and develop people, train our managers so they have the support that they need, give people clarity around the work that they're doing, refactor our quarterly reports so they are quarterly check-in so they actually bring value to teams. I mean, there's just been a tremendous amount of work that's been done under the surface. And I want to call out specifically the incredible work that the talent and culture team has done around this, right? As like I've got this partial list, just the benefits that they've afforded us in the last year alone around remote working parental leave shared and pooled sick time and equitable holidays have been phenomenal. So thank you, TNC. And I don't know how many of you spend time reading the FDC feedback on our annual plan. I was right before coming into this meeting. And yes, the FDC made some really good points about how we are measuring the work that we do and the smart metrics that we can use but how we've already made such an incredible improvement there. I want to specifically call out the reading team which introduced as part of their quarterly check-ins, a separate meeting all on the metrics that they're seeing and their products. And that is something that I'm seeing adopted by the editing team as something that they'd like to introduce and that's phenomenal. Like we should be spending the time to understand the impact that we're having. And I know that that's not just these teams. I know other teams are doing this too but we've made real progress there as well. And not just like neoliberal crazy metrics around like project success where everything has to be measured. Like these are empathetic metrics that incorporate both qualitative and quantitative aspects to them. And I think that that's really important because we're not just here for the KPIs. We're here for the community but it does help for us to understand where we're making progress and where we're not as we just learned in terms of the quality of articles. The values process that we went through, that was phenomenal. Almost everybody in this organization participated in that in some way, shape, or form. And members of our community participated in that too to help us understand not just sort of what organization we want to be but also the organization that the community would like us to be. And I was talking about this a little bit with Victoria this morning and she rephrased it to me as the community wants us to be the highest integrity institution we can be. I thought that was really lovely. And I think that's something that we can all aspire to. I'm not suggesting that needs to be a new value Anna, don't worry. But I did like the way that that phrasing that she phrased it because I felt like that was in some way capturing what it was that we got out of that values process. Staff inclusion and diversity. I have been thrilled with the fact that we've launched this. I am so appreciative of all the work that Angel has done to support the, SNAP's awesome employee resource group. I want to encourage folks if you haven't registered to participate in Pride yet, the deadline is it today? It was yesterday. Oh no. All right. Well, for all of you who are participating, that's awesome. Please do. And I also want to acknowledge that there have been some parts of sort of restructuring and organizing that have been difficult. I want to say thank you to all the people who've participated in the product and tech team tuneup. I know that there were more than 90 people who spoke with Anna. I know that that was something that not everybody is fully on board with some of the suggestions that came out of that. And I just want to respect and acknowledge that. But I also want to say thank you for moving forward. And I am looking forward to what we learn as those new sort of structures fall into place and what we get out of them. And hopefully recognizing that if we do move into this sort of new structure and it's not working for folks that we can also continue to evolve and iterate and change as the work dictates and as our needs and our audience dictate and as we learn about how to be more effective. But thank you. I know that it hasn't been always easy. So I really just want to appreciate that. And I also want to speak to the fact that we've hired this new, well we've hired three new executives and we still have some work to go and I have some work to do around bringing on a full executive team. With the addition of Victoria and Eileen and the confirmation of Jody and Anna and the talent and culture team that's actually at a rate of about one person or one team every two and a half months which is, it feels like it's been a long time and it has been a long time. It might email the other day about the chief of communications role. I just want to acknowledge that this departure of our executive and leadership team over the course of the last two years really had a sustained impact on our organization. And we have had phenomenal interim leaders who have stepped up to the plate and have done an incredible job leading their teams through and leading this organization through. And I often reflected as we first got sort of coming out of this transitional period on the fact that that team together had more collective years at the Wikimedia Foundation. I think it was like 25 years of collective experience at the Wikimedia Foundation which is incredible and I genuinely don't think we could have done it without them. So I just want to say thank you to all of the, to the entire executive team, interim and otherwise, for being just great allies and I really appreciate the work that they've done. And I hope that, yeah, sorry, okay. All right, so that was just the first bullet, wasn't it? So I, yeah. And then the other things that the board asked for and all of these are really important are the movement-wide strategy process. So I know we've talked a lot about strategy so I'm not going to talk about the process itself. I'm going to talk about how incredibly amazing it is that we are talking to thousands of people around the world to understand what their vision of the future of free knowledge looks like. Thousands of people in dozens of languages on every continent. Experts in technology, experts in cultural institutions, experts in the arts, experts in diversity and inclusion. We have gone out and sat down and listened to or interviewed or had conversations with. Our communities have had conversations with. We've had conversations with our communities. We've flown people into Berlin. We've done workshops with open technology. What's it called? Open space technology. We have no idea where it's going. We have every idea where it's going. It is, I was talking to a strategy expert the other day who was like, do you see it as a blob on the horizon? And I said, yes, a blob on the horizon sounds right. She said, great, you're on the right track. We have no idea where it's going. We have every idea where it's going. We know it's going to be completely aligned with our values and our future. We know that the people who show up are exactly the right people to be there. We know that the only outcome that comes out of it is exactly the outcome we need it. And I can't wait. I have seen it as this tremendous exercise in aligning ourselves around our values and vision, in having conversation with community members that we haven't spoken to in some time, in elevating the perspective of community members who are not always heard, and in reaching out to community members who aren't yet a part of our community. And through the process, at least for me, I have come to this conclusion that if you share our values, you are a Wikimedian and you might not even know it yet. I think that's a line I stole from Guillaume. So thank you, Guillaume. And so I see this incredible strategy process not just as something that allows us to start building quarterly check-ins, which we will do at some point, and annual plans, which we will do at some point, but as an opportunity for us to clarify who it is that we think we are and where it is that we think we're going, and at the end of the day, come away inspired about that future and have something that when we go out into the world that we can talk to people about what it is that we're trying to do. Not just build the world's free encyclopedia, but have a sustained, meaningful impact on the availability and quality, breadth and distribution of free knowledge and who participates in it, who builds it, and where they come from. So that is how I've been seeing the strategy process, not as a process of measuring and metrics and building a strategy about where we're going to invest and what we're going to do and how teams are structured, but who we are in the world. And I just want to encourage all of you to please continue to participate. I know that there's a ton of work that's going on, and I know that this is in addition to that work, but I'm so grateful for all of the things that you're bringing to the table already, because this really is about our future. Okay, so that was the strategy process. Hiring executive team strategy process and working with the board. And I just want to say thank you to our trustees and the new trustees that have come on board or are going to come on board through the election. We've been doing work at the board level around thinking about governance and improving the work there and making sure that the board is more transparent and accountable and we do now publish all of our board meeting notes and all of our board meeting decks and all of that. So if you're interested in that, please do take a look. And I know that our board has made a real effort to come to the foundation and understand who we are with our chair and vice chair attending our all hands. And I think that that's a phenomenal practice and something we want to continue because we never want to find ourselves in a position again where we're so separate from our trustees and they don't understand where we are and we don't understand where they are. I think having those open lines of communication is critical to making sure that our movement works because at the end of the day, we are an organization that is in service to our movement and our movement is an organization that stewards us. I did have a long list of things I want to thank people for. I don't know. I think I've run out of time, right? Think I've run out of time. Okay, well, I do want to say thank you for all of the incredible work that you've done. I want to recognize the fact that we won this remarkable victory. Was it yesterday, the NSA case? So that was awesome and really the person who should get tremendous credit for that is Michelle Paulson and wherever she is, I hope her ears are burning because that was her work, her leadership and her vision to make that happen and to not understate it. That is the first time that an organization has established standing against mass surveillance. It is a tremendous legal victory, not just for us but just in terms of overall precedent in the law in this country. So phenomenal. Y'all are amazing. All right, I'm not going to try to go through the list because there is food out there but I just wanted to acknowledge that one thing. All the other work that I have not mentioned is amazing as well. Thank you guys. Thank you. Satya. So I think we have some time for Q and A. There's a couple of questions from IRC. First one, actually two of them for Joe on the new filters for edit review. Is this, this is one from Leila. Is this, sorry, is this tool only on Wikipedia or all projects? It's on everything. Yeah, it's on everything. The OARS portion of it is only on Wikipedia and only on currently on, is that right? On 10 projects right now. There was a recent blog post about the new filters so any community that wants to get OARS going on its Wikis, there's process for that and it just takes volunteers, volunteer editors to go in and score some thousands of edits so the system can digest them and recognize the patterns but yeah, it's on all the projects. Cool. Say Joe, don't we have that on Wikidata? I'm pretty sure that we have it on Wikidata too. I'm sure. Yes. Okay. Another question. This is from Tillman. Do we already know which combinations are most popular among editors or which they find most useful? That is so interesting. No, not yet. It's only been out really a few weeks. We have instrumented the tool and do intend to do that analysis and I'm sure that will lead us to some interesting insights and maybe new features or something. One thing that we are going to do is, as I said it's coming out quite soon is the ability to, because there's a lot of tools there so people are going to be able to save their settings and save sets of settings by name and to declare one of them a default so I hope to be able to mine that data in particular and I think that should give us some interesting insights into what people find useful. That's all from IRC right now. Any other questions from those in SF or all y'all looking just real hungry? All right. So, nothing else from IRC? Nothing from IRC. Cool. So, I want one person to do some Wiki Love and then I swear I'll release you. Please. Anybody else? Edward or Toby? I got two. Thank y'all. Totally cheating here a little bit but I want to send some Wiki Love to everyone who has signed up for Pride and to add that the registration is still open so you can still sign up. I'm going to give some Wiki Love to Halfacre who's just everywhere. Good job, Aaron. Oh man, thanks Toby. Awesome. Oh, I want to give some Wiki Love to Catherine for everything that she's done last year and for investing in us and our mission and trusting in all of us and thank you for everything you've done. And now with that, this concludes May Metrics. Thank you all for coming. Go eat.