 Welcome to January 2017's metric. My name is Josephine, I'm with the IT help desk. So today's theme is building our future. So pretty much what that is, is that we're gonna delve into finding ways how we can, as a community and WMF, build our future together. So here's our agenda for today. We have welcomed our theme, our introduction. We're gonna go over movement update. We're gonna be reconstructing media Wiki history and then next community capacity development and then movement strategy update and then afterwards we're gonna do questions and discussion and then move on to Wiki Love. So today we're gonna be welcoming some contractors, interns and volunteers. So we have Allison with Legal, Omar with CE, Biza with CE, Anthony with Product, JC with CE, Jean with Technology and Veronica with CE. And so we have a lot of anniversaries today. So for one year anniversary, we have Deb, Jack, Nathaniel and Chris. For two years we have Arian, Cory and then for three years we have Ann Smith or Sam Smith and then, oh wow, there's a lot of three, Alex, Giles and then for four years we have Bruna, Doreen. For five years we have Jody, Steven and Andrew and for a good six years we have Dario. So next we're gonna go over the movement update with Maria. Hello everyone. So as we've mentioned before, the theme of this meeting is building our future. My name is Maria Cruz. I am communication project manager in the current engagement department. And I'm here to share a few stories about the movement. Next please. In the theme of building our future, is doing this by enabling students to learn values of collaboration and cooperation with weak media. And one example of that is Corphopedia. Next slide please. Corphopedia is a project that was proposed by students to their teachers after noticing there is not much content about Corphoo on Wikipedia in an introductory course about the encyclopedia. It involves 60 students ages 12 to 68 from traditional and evening high schools. This involves two junior high schools, one evening high school and one vocational evening high school. And that's why the age range is so wide. Another story from the movement, next slide please, is Wikisbeak's language. Wikipedia will talk to you and it will teach you pronunciation. Next slide please. This is a project started by shared knowledge of Macedonia to document spoken examples of different languages. The goal is to enrich content on Wikipedia with multimedia files. And in this way increase the quality and educational value of the written text. They have been partnering with Glam institutions, education institutions and other Wikimedia affiliates as well. I'm trying to promote this initiative through the cooperation of Wikimedia Central and Eastern Europe. Next slide please. The Hungarian Revolution of 1956 is an editing challenge. And I thought of sharing this because I think the future will certainly be built through affiliate collaboration. Next slide please. This is a writing contest to commemorate the 60th anniversary of the Hungarian Revolution. The challenge was taken on by affiliates of Wikimedia Central and Eastern Europe. It had seven participating countries, 390 plus articles edited and only one edited down in Hungary. And the contest of organizers have implemented valuable lessons from Wikimedia's CE Spring writing contest. And finally, next slide please. During January, Wikimdava 2017 took place. Next slide please. This is the regional conference for Africans to strengthen support and share knowledge within Wikimedia communities both in the continent and in the diaspora. It took place in Accra, Ghana from January 2022 and it was supported through a Wikimedia Foundation grant. It had over 45 participants from within countries. And among the highlights of the conference is that increasing visibility and awareness of the movement, of the global movement was identified as a key area for development in the region. And you can read more following that link. Next slide please. From the foundation highlights, we wanted to share that the entire campaign on gender gap published its final report. This is a qualitative report that looks into inspired campaigns as a model for proactive grant making. This is a model created by the community resources team. And the report offers key lessons learned from the first campaign on the topic of increasing gender diversity on the Wikimedia sites. The Wikimedia Foundation received a $3 million grant from the Sloan Foundation as you may have read on the Wikimedia blog to enable structured data in common. The second annual one live one breath campaign kicked off in January in this month as well. That this campaign encourages librarians to get involved with Wikipedia by adding citations to articles. Next slide please. Many things happened in January as well. The Wikimedia developer summit took place from January 9 to 11 and volunteers and staff, developers discussed six key topics among which were creating a plan for the 2016 community wish list, top 10 ideas and how to grow the developer community. And we also had our Wikimedia Foundation all hands hosted in San Francisco on January 12 to 13 where employees participated in sessions connected in person and discuss issues important to our work. And we also had an amazing talent show. And coming up in February 2017, next slide please, is the movement strategy process, annual planning and board recruitment process. And that is it for movement athletes. Thank you. So the next speaker we have is Dan. Yep, hi everyone. I'm Dan, I'm on the analytics team and I'm super excited to be here today showing you what we've been up to. So basically, in the theme of building our future, one of the things that our team does as analytics is try to make sure that we understand where we come from. So they say hindsight is 2020. And at Wikimedia Foundation, it's not always easy to answer simple questions. When we looked at the data, we found that a lot of the very fundamental things that we need to know about our past are actually pretty hard to answer. I'll give you an example. So how many new editors join all of our projects since the beginning? To answer this question right now, we have to write this complicated query. It has to hit three tables, across 800 Wikis. It's got a bunch of joins and subqueries, hard to read, and ultimately takes five days to run. So this is a problem because it's data that we need to know. It's information that we need to know. What we did is we brought all of that data from all those different places into one place. We patched it up, we cleaned it up, and we're calling it the data lake. So now, when you ask the same question, how many new editors? We don't have to do any joins. We don't have to do any subqueries. We only look at one table, and it takes five minutes to answer the question. So more than that, we built a process by which we can start with a question that's otherwise hard to answer, and we can build infrastructure on top of infrastructure that we've been building for years to make it easier to answer that question. So I'm going to go over a few questions that are easier to answer now with the data that we've already built, and then I want to show you ideas for what we can do going forward and how we can go about asking more interesting questions. So what if we wanted to know how many of those new editors, how many of them were bots when they joined? Today, we know, based on our database, we know which editors are bots right now, today. We don't know who was a bot back then because we don't store that data. So we dug through the logs, we patched, we historified the concept of being a bot, and we have two fields. Some technical stuff here, but easy to understand. There's an event user groups and an event user group's latest field that's there in the same table with every edit. So we know whether the editor was a bot at the time of the edit and whether or not they're a bot today. And instead of joining with the tables and trying to pull this information from hard to get places, it's right there in the same row. Other things that are historicified that we tracked down backwards through time are things like what titles, pages have had over time as they get renamed and redirected, what names users have had over time. And these things are maybe not too interesting on their own, but what if we wanted to ask something like do people who get reverted or people who have harassment directed at them, are they more likely to change their username to try to hide from that? So this is data that's now letting us ask those kinds of questions that are really important to us as we try to figure out what we want to do in the future. We also have some interesting new insights about our data that, again, really hard to get if you're just looking at the current structure and digging through the tons and tons of data that's available without prioritizing looking at it from a question perspective. So for example, we have this concept of a revision being productive. That means that someone made an edit and it was not reverted within 24 hours. To figure out whether a revision is productive today, it's a really hard process. In this work, it's just a field that says yes or no right next to the edit. So you can use that to ask questions about the productivity of the edits. What we want to do next with this data we have lots of actually, I guess other things that we track down are and we have contextualized to the edit what registration date people have so you can ask questions about what they do relative to when they were registered, how many bytes a revision adds or removes which is another thing that's a little tricky to get today. You can take a look at this and ask us for more details. But what we're doing with this data next is we're using it to update WikiStats which is an amazing project that started from before Wikimedia Foundation even existed and is a critical resource for our communities to do their work. We're going to update that with this data and hopefully lots of interesting new stuff that they haven't been able to use until we did this infrastructure work. We're going to publish this so that it's available for public research in collaboration with the labs team. We're going to make it available in this interface where it's really easy to slice and dice. I'll show you an example real quick. Going forward even more we're going to look at the revision text itself and parse that because there's lots of really interesting things I'll give you an example and most importantly we want to show that we built a process where you can ask questions so your questions whether you're from the community or from the foundation are welcome and we're going to try to figure out how to build infrastructure to answer them. This is an example of pivot it's an interface that's available right now only internally because of privacy concerns but hopefully this data is going to be available externally soon in this shape as well. We showed this to a few different teams like fundraising and reading everybody's super excited it's really fast to get you insights this is showing the amount of content added per wiki so you can see that wiki data wiki is getting a lot of really interesting work being done. The kinds of questions that we want to answer like get to things that we haven't been able to get to before so how much work does our community do to all of our projects one of the ways to measure is to count the tags that they place where work needs to be done right that's how much work needs to be done so citation needed is you know our classic tag how many of these tags are there in wikipedia across other projects and backwards in time these questions are really hard to answer now and getting the answers for them will allow us to do things like measuring the community's backlog so we can figure out so we can include that in our decision making importantly we want to count what you need so I hope this data inspires you and you come talk to us you can reach us on rc and we can get analytics or on our mailing list or we have all of our documentation on this stuff and more on wikitec thank you very much we need to see all these slides again okay good evening I'm here to talk about the capacity development pilot program and it is a program done by the community resources team CR on the CR this is an experiment it's a program that was built on the premise that there are certain capacities that all thriving communities need to have and that some communities for whatever reason have not been able to develop or grow these capacities sufficiently or have plateaued and can't quite get beyond a certain level of capacity and further that WMF can usefully intervene and help those communities with a targeted, limited time project partnering with a specific community to build a specific capacity to kind of get them back on their way growing and developing that capacity this project I would like to acknowledge and thank Anasuya who approved this almost two years ago and had the vision and the clarity to see the need and further this project is cuteness approved much of it was supervised by Kankunchik here a member of the Wikimedia cuteness association so how do we go about this first we conducted a whole bunch of research with quite long community interviews with 17 different communities across the world most of them emerging communities yes yes I can give you examples partnership building media relations community governance on wiki technical skills all of them are necessary for all communities and some communities have naturally grown those capacities and others not so much and so this research phase was precisely designed to find out what are some key capacities that are relevant for our communities after that research phase which was conducted a year and a half ago we selected three emerging communities to pilot with the three were Brazil with whom we were working on communications and media relations the Tamil community in India with whom we were working on on wiki technical skills and Ukraine the Ukrainian Wikipedia community with whom we were working on conflict engagement so already an example of three different communities that identified three different key needs for development once we had those communities interested in working with us on this we developed a curriculum for how to build that capacity for that community and the key factor in that was delivering that training in person in that country and in that language using translators then it's time to evaluate this program and that's where we are right now we are done evaluating the program so here I am telling you about it after that we will need to decide now that this pilot is complete what conclusions do we draw and how do we move forward so that's just the timeline not my style really but apparently photos are important so here is India that was also by the way supervised by cuteness here is Greg Varnum helping out with the communications training in Brazil in India UV was key to the success of the training and I don't know who this guy is conflict management so does this work the short answer is yes this works we are able to help communities build these capacities if we pay attention to that community the longer answer is not only does it work it also has additional beneficial side effects so I would like to share a few lessons first of all this high touch approach the community is really appreciated working with us the level of attention that we paid we repeatedly heard during our interview phase things like nobody has ever asked us that nobody ever cared how our community selects admins for example and the communities in this pilot did successfully level up did successfully break new ground in their development in these respective criteria what I'm stressing here is that in addition to our efforts to scale and to do everything that serves everyone with the greatest multiplier there are certain things this is one of them specific capacities of specific communities that benefit from a high touch high interaction approach we need to do both this training was effective partly because it was in person and in their language in their own language our post training surveys and interviews have proven this time again people really appreciated the fact we came to them and we made it accessible in their language the materials from those trainings are significantly reusable it turns out that these needs are actually shared across large parts of the movement for example the conflict engagement materials we have developed and it is quite tricky to teach Wikipedians about conflict because most of the standard curriculum on how to run conflict is about people having conflict in person whereas most of our conflicts are online sometimes with pseudonymous people etc so that curriculum has turned out to be quite reusable I have personally given it already at four conferences beside the training in Ukraine likewise the on wiki technical skills training which included tools demonstrations but also a thorough introduction to wiki data was already delivered in multiple conferences and was very well received so to share some concrete examples of the impact the Brazilians following the training certainly correlated the Brazilians who are permitting me to say also caused have revamped their website which was defunct have revived their blog and social media and regularly contribute and use those and they have created a press kit these are things that for whatever reason did not exist in Brazil before this training the Tamil community now regularly engages with wiki data before the training there were zero contributors from the Tamil community except for inter wiki links that everybody had to kind of migrate to there was no editing of wiki data beyond inter wiki from Tamil editors now there is and one of them I don't know how has amassed 200,000 edits manually not using bots to wiki data in under a year the training took place in 2016 yes there are some quotations here I won't read all of them but people were saying I was aware of wiki data but found it complicated confusing to understand now I think it's the future of wikipedia my mind was blown I was inspired and started contributing massively we need WMF to come to communities the quality and depth of the training by experienced WMF staff can't be matched by outsiders I don't know if it can't be matched but anyway warm endorsement one very veteran wikimedean more than 10 years experience in the movement told me I attended lectures about wiki data many times but not one has engaged me and made me actually want to contribute I was finally persuaded that I should invest time and go to actively contribute to wiki data so now what this was a strategic pilot this was an experiment the experiment succeeded this approach does work the report which is on meta is recommending that we scale this up now that we know this works scaling it up means working with additional communities and working on additional capacities we've only worked on three capacities out of six we had identified in the initial research and we could probably work on a few more secondly to develop a kind of core curriculum a kind of notion of what all communities should have and then track that across communities so that we know where are different communities along this core curriculum it should be somebody's job to make sure our active communities are not left behind on adopting wiki data on using Lua I don't see that it currently is and someone other than the community resources team should care that the Tamil community was not using wiki data at all so I'm proposing I'm recommending that we track this that we do pay attention to this and then the constraints of resources and budgeting see how can we help the greatest number of communities progress the greatest amount along this core curriculum and finally and this is with a view to scaling identify some already effective trainers across the movement and empower them to deliver that training again and again in their own communities and in other communities one thing that was observed across all three pilot programs is that the quality of the trainer and the training matters a whole lot to the receptivity and effectiveness of the training some people are better than others at public speaking some people are better than others at explicating wiki data specifically for example I have been told by the wiki data team in wiki media Germany to explain wiki data impressed them and they liked it so much they are in fact adopting it to their own efforts to explain wiki data the point is once you find something that works you need to empower that make sure that has more chances of scaling across the movement I cannot personally teach wiki data to every single wiki media in the movement but we can train trainers make sure they're effective and then send them out to do more trainings so these three key recommendations are now before the foundation and that implies increased resourcing for this program again it was resourced at a pilot level now WMF needs to decide whether and how to increase the resources and the big question is is WMF leadership interested in this now that we know this works we need to decide whether we want to do this how do we want to do this what teams would be involved in doing this we need to have a community resources team and that is again where we are right now that decision is yet to be made if you have more questions want to get involved want to read some of the data and surveys etc it's all on meta under CCD community capacity development or you can write to me thank you Lisa aren't you next is this strategy update is this cuteness approved I think it's lacking can we go to the next slide I think it's lacking a little on cuteness yeah I mean it could use some work I guess black and white slide so this will be very brief and I think most of you but perhaps not of all of our listeners online have met the strategy team so we just wanted to take the opportunity today to reintroduce for some of you and introduce for those who haven't met them yet our four strategy team who is leading the movement strategy process there's a combination here of new faces and familiar faces kind of the the leaders of the group are Whitney Williams, Ed Blan and our own Guillaume and the project manager managers are Shannon Keith and Susie who I think most of you know who's worked with us I guess for well over a year on strategy and annual so just want to introduce they are all working this week in Seattle together and well I'm sure will come out with to share with us thanks to everyone who participated in the workshops that we did at all hands this team was also a couple members of this team are also over in Switzerland with some executive chapter executive directors and started the conversation with them as well a lot more to come from this team so and they are somewhere listening participating remotely in this meeting as well so welcome to all of them hi everyone well chime in briefly from Seattle thanks very much for that warm introduction this is Ed Blan I'm sitting here on the couch with my colleagues Guillaume you know Susie you probably know this is Shannon Whitney is not with us today but she's with us in spirit so we're delighted to be involved in this project thank you Asaf that was very helpful for us here and we've heard a number of other things in terms of a quick background I spent quite a lot of time in the corporate world and have spent the last ten years mostly in the non-profit world working with organizations that are part of movements to help them scale up and be more effective in those movements the largest of which is the microfinance movement so that's my background I've been working with Ed and Whitney and others on the Williams Works team for several years now in collaborating with brands that reach sort of large numbers of people that are interested in engaging those people to do good in the world and so it's a delight for Ed and I to be working with the Wikimedia communities on this project so we're excited to share some of what we've learned from working with others and some of the folks we know and sharing that knowledge and also listening and really learning from you all which we've done for the past couple weeks but are early and intended on doing more so looking forward to that No I think we're doing a lot of great work and we will start posting that on Meta and on the mailing list very soon and we will also be able to answer questions at the end of this meeting if you want to have any Nothing really more to ask Thanks again for the invitation to participate in the Metrics meeting This is Juliette, just while we have you I know that many people have met you but not everyone in addition to getting to know each of you individually I'm wondering if you could give just a general description of Williams Works as an organization and what your focus is and and what you're working on with us here because I don't know that everyone is familiar with that side of things Sure, we off mute So Williams Works as a firm, the reason we're in Seattle we're based here in Seattle but we also have team members in other parts of the world in Africa and Europe Whitney Williams founded the firm 18 years ago she used to work for Hillary Clinton in a fairly senior capacity helping her to do a lot of travel and logistics around the world meeting people learning about problems and helping to address those problems She transitioned to creating this social impact consulting agency and we've been helping for-profits and individuals to do good in the world sometimes it's quite a shift when they're for-profits and they don't really know how to give back and they want a lot of direction sometimes they're individuals who are have done well in their careers and now want to give back in a significant way and would like help and guidance doing that and other times they're non-profits that want to do more or have more impact in the world so we've been doing various projects with various organizations over the years including the Bill and Melinda Gates Foundation early on Tom's shoes we've created non-profits in East Africa and Central Africa all sorts of brands that you would know for profit brands and feel free to check out our website Williamsworks.com let's see this project allows the core group and various communities to pull up quite a few levels from your normal strategic work and think longer term about a 15 year time horizon and where this movement is going that you're a part of where you are going and where you want to go how you want to achieve high goals of knowledge spreading knowledge so that's what we're involved with we'll be publishing very shortly a project plan that will help everyone see the scope is how we're engaging different audiences and allowing lots of participation into that process of developing strategic fields that will help to guide the communities for future context hope that was helpful feel free to ask questions in the Q&A if you have more specific questions more questions from RSE and one was about dance talk but I'm going to talk about strategy I'm going to read one from Joel which was mostly about a self-stock but he says how should the foundation decide whether to spend more resources on expanding this program the community capacity one versus all the other things the foundation could do I'm envisioning something like somebody giving a presentation like this and saying our pilot shows that this program has a cost-benefit ratio between three and ten and then that could be compared to other programs it's the current strategy process coming up with anything that we could use again, apples to apples comparison especially between heterogeneous choices or that's more knowledge to humanity a new server from Africa or training medium-sized community so mickey data how do we make that choice so I'm trying to understand everything but I'm going to give you an answer and you tell me it's that versus the question so the phase for the strategy process that we're in right now and that will be until around wikimedia is trying to have a movement-wide discussion and to sort of define a direction where we want the movement as a whole to go and to try and align between the different actors of the movement and the partners so it's not going to be about should the foundation devote resources to a specific program or a specific feature of the software so it's not going to be that level of discussion it's going to be as a movement composed of the foundation but also affiliates and organized groups and individual contributors and potential readers as the whole movement where is it in what direction do we want to go and once we try to agree on that in a few months then we will start looking at so how do we translate that into action and who is going to do what and then we will start talking about strategic plans with deadlines and roles and specific goals and then we will start trying to prioritize those programs and how we assign the resources but that will come in the second phase of this project does that answer the question yes it does as an option include ORAS scoring on past divisions this could answer questions like where past edits more or less damaging as estimated by ORAS have committed standards gotten stricter over time trying to find what we get for the same level of damage that kind of question the answer is it is definitely possible to analyze revision scores over time we have to build a little bit more infrastructure but that is exactly the kind of thing that we want to know like what should we prioritize what questions should we make easier to answer right now so thanks for that we will follow up on it with Eric any questions in the office yes I have a question about the capacity building I was curious for the three sessions that you ran did you have like specific goals that you were trying to get that community to hit and is the idea sort of once you've got all six I think of the topics that you have planned actually kind of the menu of your topics and then your goals to hit one training the trainings did have goals and some of them were concretely met and others may maybe met that there's a kind of growth towards them some are very hard to measure so the conflict management one Ukraine was the hardest to measure because it's a soft skill to measure the amount of conflict or whether conflict is happening better inevitable conflict is happening in a better way on Ukrainian Wikipedia that's very hard to measure so there's no concrete evidence that we can point to for example that shows decreased conflict on Ukrainian Wikipedia but how people individually behave within that conflict and how comfortable they are to operate in a conflict that has improved and we have anecdotal and personal evidence to suggest that other things are easier to measure I mean Brazil's media development is objectively observable wiki data edits can be counted to your question about the capacities the six that were identified in the research six is kind of arbitrary there could be 10 capacities there could be 20 there are more at the time for example we excluded some capacities that at the time seemed to be addressed by other efforts within WMF focused on organizational development of affiliates those efforts have since been suspended so now it makes sense to actually revisit them as possibly part of the capacity development work so that's not the closed menu of what we would offer even within each capacity we focused on wiki technical skills so we focused on wiki data and bot frameworks with the Tommels maybe other communities will say no we got that covered but what we really want to understand is Lua and Boris and labs it's really interesting thanks this is Dario from research I wanted to say quickly that in response to his offer yes in fact we may have raised now to so we could talk about it and I had an anecdote share in response to the project that Dan was presenting which is particularly telling about the reconstruction of an article history so a few days ago a few reporters were tweeting that the wikipedia article about the 25th amendment of the US constitution was spiking in traffic in response to something happening in the public debate in this country that is drawing a lot of attention I tried to replicate this data and try to figure out that was actually the case and I couldn't and the reason is that there is a 25th amendment article in the media there's a bunch of redirects that were left there over the history of many changes of the article titles and that's something that happens very often in wikipedia I want to story short our PGU API right now is unable to result these redirects and we might be unable to tell the story of what topics are actually spiking without looking into our history of an article changes and title changes so I just want to teach this as a possible use case I think adding the entire article history reconstructed will allow us to have very high quality data about traffic not just edits so I'm very excited about that I said this on our team but I'll just mention here we're super excited to solve the redirect confusion that has played many different questions with hard answers over the years super wanting to solve that thanks Daria for mentioning one more question from RSE from Leila to Asaf and just for the reasons that people brought up whether pilot was not good in some ways or in a different way did anyone come up to you and say for reasons X, Y and Z that this kind of pilot would be a bad idea and if so what were the reasons of your points they had no nobody nobody challenged this idea when it was presented and announced almost two years ago when we presented the results of the research phase in Berlin people had questions but nobody said this approach strikes me as terrible or wasteful or I'm even surprised nobody really challenged the approach maybe it means everybody thought it's a great idea maybe people were not paying attention but the specific programs in the communities they were done were very positively received people did have criticism about some technical details like one of the programs had a real-time interpreter who wasn't performing well enough and was distracting for people because they had to kind of mentally keep correcting the mistakes and adapt the wiki terms etc but that's not a very interesting criticism it's a technical detail that of course we will do better next time so interestingly I have not had a very very I've not had a lot of responses to our choice of model to our hypothesis here nobody has seriously engaged with it there were a few positive comments and that was it are there any other questions any on IRC in the office well next we can go ahead and move on to wiki I just wanted to say thank you to everyone that participated in the foundation including Jack Zach Anne, Saf, Catherine and Casey as well making it a really awesome conference there was a lot of really good workshops and teaching learning, sharing and I really really appreciate everyone's efforts to make it a great success so thank you hi I want to give a shout out to Maggie the NSF team in community engagement a couple of weeks ago we had the Dev Summit and this was really my first opportunity to meet the developed community and I'm just so grateful for the work that they did both to prepare that but also to prepare me along west to have a very positive engagement I felt that some of the atmosphere in general but Dev Summit was really really positive and helped I think have a very positive dialogue and certainly helped me personally begin to get to know people in a positive way so I really appreciate it a big shout out to Kim and Maggie and the rest of the team I wanted to give some wiki love to everyone who worked on All Hands this year which is I know a lot of people and I'm not going to even attempt to name names I would leave really critical people out but we all know who you are you are all awesome and I really thought All Hands this year was a great mix of social time of team building time, of substance and great location, great everything and thanks to everybody else who participated I think people showed up in a really great way really cooperatively, really sharing a lot and I think the whole experience was just amazing and thank you to everyone who worked on it and everybody who participated Anyone else want to share some wiki love? Okay so I guess that's it for metrics