 But it was originally a 90 minute session, and it got rescheduled to 30 minutes, which is not a round table friendly length of time. So I changed it to just a presentation and the Q&A at the end. I hope that's all right with you. So that said, I would like to talk about what problems for the issue. So I would like to talk to you about what problems we have with software RFCs and how I think we can solve them. And to be clear, this is just about RFCs that are used for software changes. So I'm not talking about other kinds of RFCs, like policy changes on Wikipedia's. And so I wanted to say a few words about myself, but I think almost everyone here knows who I am, so I might just skip that. But very short, I work for the foundation, but that's not why I am here. I have been a volunteer for a very longer time than that. And I'm interested in this topic as a volunteer who is heavily involved in movement strategy and as you will see, this is a topic that's very relevant to movement strategy. So the reason I am talking about this now is that as you, I'm sure you know, the movement charter is being created right now. So the movement charter drafting committee is working on this foundational document of our movement, which is a sort of constitution. And one of the movement institutions, it's supposed to define is the Technology Council. And one of the areas that the Technology Council is supposed to handle is how do we agree on what software we should develop and how do we agree on when, whether that development is successful, whether the software is good enough to be deployed on the wiki. And this is a big deal, the entire budget of the movement is between 100 and 200 million dollars a year and something like half of that is spent on software development. So a lot of effort goes into software development, how the software works is very much influences the experience of editors on the wikis. So it's important to figure out a good process for this. And the current process is involves heavily RFCs. And I think RFCs are a truly terrible process for this. They are bad for people on all sides of the aisle. And I think the movement charter is an opportunity to define a good process if we can agree what is a better process. So what are the problems with software RFCs? And I'm using RFC in the wide sense here. So some projects actually use votes, but from the point of this topic, it's the same thing. And I assume everyone knows that, but just in case you don't, RFC is it stands for request for comment and it's a very core governance mechanism of the wikis. It's similar to voting, but it's a little less structured. So the main problem with software RFCs is that they are a huge waste of money. The software development costs a lot of money. A small feature is something like a quarter of a year for a software team to develop a large feature can take multiple years. Given how much a software engineer or other kind of IT professional costs, we are talking about money, donor money ranging from hundreds of thousands of dollars to millions of dollars here. And the current approach is that the foundation or some other organization, but it's mostly the Wikimedia Foundation today. So the Wikimedia Foundation decides what software it thinks should be developed and then works on it for a couple quarters or a few years and then deploys it to the wikis and then the wikis, if they don't like it, they organize an RFC and decide whether they think that is acceptable. And if the outcome is that this is a bad thing and they don't want it, then essentially you just send a million dollars of donor money down the drain. So it's very clearly we should agree on what we want before we spend a year on developing it. And besides the cost in money, there is also an opportunity cost because that means the foundation could have worked on something else that's more useful for the editors, but it didn't. But on the other hand, RFCs let the editor communities get truly involved in decision-making, not just being consulted and then someone else decides what their feedback means. They have actual decision-making power. I think that's something that's very important to preserve. I think that the wikimedia movement is built on the volunteer time and effort of editors and they deserve to have a say in what software they got, what resourcing they got, just a say with a better mechanism. So maybe could we just have the RFCs sooner in the process? Well, not easily. So that isn't really any commitment involved in RFCs. So anyone can just decide that they don't like the RFC and run a new RFC in part because people change over time. So if a software development process takes like two years, then two years later, it's not necessarily the same editor community anymore. There are new, highly engaged editors or influencers. And sometimes even on shorter times, like for example, for the recent development work on Vector, there was an RFC that decided that it should be deployed. And then three months later, it actually got deployed and people didn't like it. They run a new RFC, which had the same outcome, but it could have a different outcome. And then, so you cannot really rely on people actually holding the outcome of RFCs as a commitment. And especially because usually the size of RFC is seen as a way of how influential or how binding it is, the number of people participating in an RFC, which is very reasonable in theory, but in practice, it's impossible to get the same level of engagement when you are talking about a piece of software in the abstract, then the level of engagement you get when it's actually deployed on the wiki and affecting people and they are maybe angry about it. So the size of engagement will naturally scale up as the development progresses, but the commitment still needs to be upheld somehow. And this is made worse by change a version. So change a version is the phenomenon that people are initially unhappy even with good changes. Google did a lot of research on this. They measured the satisfaction of their users over time. And what they found was that development, the satisfaction curve for a good feature looks something like this. There is a stable level of satisfaction for the old feature and then the new feature gets deployed. Satisfaction drops down because people are unfamiliar with the new way of working. Suddenly things are not where you remember that they have been. Suddenly you need to think about how certain things work. Your muscle memory is not working anymore. It's a distraction from what you actually want to do. And then if it's a good feature, then over time people get used to it and satisfaction goes up and people realize that it's actually easier to use than the old thing. And eventually you get a higher level of satisfaction. Or maybe it's a bad feature and satisfaction goes up a little bit, but you end up with a lower level of satisfaction altogether. Problem is that air assistance tends to happen at the bottom of the satisfaction curve. So at this point, just from user feedback, you can't really tell whether it's a good feature or a bad feature. Everyone is angry and that's the status quo that goes into the outcome of the RFC. And another problem is the quality of decisions. So just because something like a vote is a way to decide that represents everyone, it's a legitimate way of decision that doesn't necessarily mean it's a good way of decision in the sense that it makes correct choices. So RFCs are originally invented for things where everyone has a good understanding of the issue. Like you are having an RFC about policy on how sources should be used in an article. All the highly engaged editors are very familiar with the policy and have experience with it. That tends to work well, but with software often you need a lot of expertise to understand the importance or the goals or the problems. And most participants just don't have that expertise. This is worse for software where the main target is not editors, but say readers or it's mainly for mobile users and most of the RFC participants are not mobile users. So you can get a decision-making mechanism which is dominated by people who don't have the right level of understanding. And you can't really expect the hundreds of people who participate in the RFC to read a book before that to understand the issues well enough or read significant documentation that would be a very large waste of volunteer time. Some projects use voting, formal voting, for example, German Wikipedia, and the people who, usually what you get with votes is that the people who are least engaged tend to make up for the majority of the participants. So the outcome is dominated by people who haven't spent much time understanding the issue. In other projects like English Wikipedia, the RFC is a more flexible thing where you need to argue and then the weight of the arguments matters more than the number of the votes. This is a very nice process in NCRA. It was originally copied from the decision-making process of the Internet Engineering Task Force, the standards body that creates Internet standards. So it definitely can work for technical decisions. But with the IETF, there is a chair who makes the decision about the arguments. On Wikipedia projects, it tends to be very undefined. Like any administrator can step up and decide to close an RFC and decide which arguments are good and which arguments are bad. That makes the process very unpredictable, potentially controversial. And it also means that it's a very hard and unfair situation for the closer often. Because it puts a lot of pressure on them and they get blamed for the outcome when they close something in a way that's different from the number of votes. And the final problem with RFCs is just the sheer scale of the communities we have. So there are almost a thousand Wiki projects. Even if you're limited to Wiki projects where there is some meaningful level of community discussion and self-reflection, we are talking about hundreds of projects. So if you could run RFCs on hundreds of projects, but then there is just no way for the organization doing the development to take that into account. And for a lot of software features, you want a uniform outcome. Like if every Wiki works differently, there are usability consequences to that. There are brand consequences to that. Maintenance of the software becomes much, much harder. You could have one global RFC somehow, but then that's even worse defined than Wiki project RFCs. Like how would we decide who gets to close and who gets to check the arguments? Which language would we only use for the global RFC? So that's not really a workable solution. And there is an opposite problem which gets talked about even less, which is that while RFCs give, in my opinion, too much power to the editing communities over whether software gets deployed, they get absolutely no power over what software gets created in the first place. So RFCs are kind of like a veto. You can decide that you don't like what's happening, but there is no way to run an RFC with the outcome that says that we want the foundation to work on, I don't know, IP blocking software instead of a skin. There is no way to influence the selection of new projects and to propose a new project. And that seems like a very important thing to figure out because if you want to give communities real decision-making power, then that power needs to start at the planning level. So how would a better process look? In my opinion, one important part is representation. Votes just don't scale up. And votes are just not good for sufficient level of engagement and expertise. If you want that, you almost always need a small body, a small body which you can ensure that it contains expertise and which you can ensure that it spends the necessary time to engage with the issue, to read documentation, to read research, to understand why something is good or bad or important. But it needs to be a legitimate body that represents the editor community. So it needs to go through some kind of election process. So we should have a group. I am using the working name product council here because I think technology council is not a very good name for reasons I won't go into. And then the editing community should elect that body and that body should do negotiations with the foundation. Basically, they should kind of replace the role of the closing administrator of the RFC. So the community can still have a large-scale discussion, but this committee would decide who is right in the discussion. The other key step, I think, is to have an open planning process. So instead of the current situation where the foundation just decides behind closed doors, what projects are it's going to work on. And to be fair, the last year, the foundation did a little bit to improve that by publishing the draft annual plans. But I think that that wasn't very successful both in the terms of making it easy to engage with and still at the point it got published, there were already plans done. So I think we need a planning process where anyone can make proposals and editing communities don't need to write open letters and things like that. They can just get in their proposals for new features into the process in an equal footing. And then a third key part of what we need is experimentation and metrics because the RFC process doesn't really have the means of incorporating that due to time constraints and due to the large scale being hard to manage. So we get into a lot of theoretical debates about whether something would work or whether something is good or bad for the user. So we need a process where we can actually check that where we can agree on what things we want to measure or what experiments we want to run. And then we can do those things and get an evidence-based understanding of whether the feature is good. That's very hard to coordinate with RFCs but with a dedicated small community, it's perfectly possible. And then the fourth key thing is having targets. So instead of having very open-ended and unpredictable battles over whether something is good enough or whether the development process is finished, I think we need to agree at the very start of a development process on what are the targets that we want to reach. And the target could be something like 90% user satisfaction in surveys or it could be something like the majority of users can engage with the feature successfully and doesn't drop off in the process of going through a dialogue. So measurable things that demonstrate that the software is working well. And the product council or the community with the facilitation of the product council could agree with the developing organization, the foundation on what targets are the ones which show that the project is successful. And then when development gets to that point, the product council oversees the metric gathering and can determine that the target has actually been reached or not reached and further development is necessary. What are the next steps of actually making this happen? As I said, I was hoping for a round table discussion and the timing didn't work out for that. So I think that if you are interested in moving this forward then the next steps are there is a technology council chapter in the current movement charter draft which is currently very under specified and very understandably not the main priority of the movement charter drafting committee because the fundraising and hubs and all these things are more important and more controversial. Maybe not more important, I would argue that software development resourcing is very important but certainly more accessible to people as a topic. So my plan is basically to propose a draft as a starting point for how the technology council could look. I will, in the coming days, I will create a wiki page, I will announce it on the wiki media main list and I hope that you can participate in criticizing that draft and coming up with one that represents some level of consensus and then presenting that to the movement charter drafting committee. So if you are interested, please join that discussion as I said wiki media is going to be the main place where I mentioned it but I will go to telegram and whatnot. Also, if you want to discuss this at wiki media, shout out to the upcoming session called discuss improvements to our collective decision making processes for software releases. This is by different people and from a slightly different angle but pretty much the same topic and it's a longer session, it's going to be an actual round table so I think that's going to be a good opportunity to discuss the shortcomings and possible replacements of RFCs. Thank you very much. If you have any questions, then this would be a good time. Do we have a microphone? Thank you. Hello. It's not a question but more comments. I am a board member and product and technology committee member and just couple of days ago, we had a meeting where the chief technology officer Selina Dekelman proposed a pilot product and technology council which would be doing what you were saying. I know that the global council supposed to have the technical council under it but it will be several years before that so we will probably have some sort of open call so please watch out for it and join the technology council, product and technology council. That's great to hear. Thanks for sharing that. Thank you for this. Thank you. I'm from the movement charter drafting committee and I just wanted to follow up on what Victoria said even though Wikimedia foundation might be going into a certain direction and also thinking along, it is of course also important to hear the voice of the technical community not saying that Wikimedia foundation will not do that but it is important to have this kind of discussion. Movement charter drafting committee is always open to this kind of feedback especially if there is community support for maybe just taking a bit of a different course or whatever. I'm very much looking forward to the exchanges that will be happening in the discussion. Thank you for taking the lead in a proposal like this. You showed the satisfaction curve. How would you see that changing with your proposal? I don't think that would be changing. I think that's just a constant of development. I think the key change here is that the decision is made at the beginning of the, or before we select plans and since we have a small committee and that's unlike the larger editor community that small committee is likely to have the same composition mostly over time. They can be the keeper of the commitment that this was agreed on. So even though right now people are unhappy and everyone finds it confusing, it will get better and we don't change our minds from one day to another. Thank you for coming.