 I'm sorry, but I'm sorry. The goodness is always the devil. It's just ashes now. Yes. Oh my God. Yes, I know. My name is just a devil. I love you too. That's my devil. He's just a devil. I'm sorry. He's just a devil. We were sitting there together. We were supposed to be. I'm going to do this for you. I did. I appreciate it. One more minute. It's good. I don't know. That was supposed to be an open quote. Edit, preview, close quote. Is that about the meeting, Ben-Wang? Yes. The rules? Hi, Adam. Okay. I guess we can start. I guess we can start out. It's great to have everybody here. We hope maybe some other people. I think in terms of the unconferencing, we're supposed to pick these four rules. Can we just have a volunteer friend note taker? Thank you. There's a tiny umbrella up here. I guess we can start out. It's great to have everybody here. I think in terms of the unconferencing, we're supposed to pick these four rules. Can we just have a volunteer friend note taker? Thank you. Aaron. A tiny URL where we already started a page. Google Doc page that should be available to everybody who has that link, I think. Note taker, we have. Remote moderator. I'm not sure what their remote moderator does, but they probably take questions on the ARC. Perfect. Thank you, Ben-Wang. The advocate does what? You are the advocate. You've got the rules. I'm trying to have people keep it moving. Hi, General. And also, all right. We have our meeting rules. That's fantastic. Welcome to newcomer friendly edit review tools. Help us imagine what that might look like. I think I'll ask first off what sort of people are here. Who here is from the anti-vandalism edit review type community? Got a couple who here is just interested in building tools and thought this out a lot. What are you guys doing here? What made you interested in this? If you're not from the anti-vandalism, you're interested in supporting newcomers, maybe. Supporting newcomers. Cool. And from what's your role in that? I could do a round. Sure. I'm Joe. I'm with the collaboration team. I'll tell you a little bit about this project in a minute. Hi, I'm Matt. I'm software engineer in the collaboration team. And I'm here because I'm interested in hiding the nice and easy people to be nice to each other. Nice. Okay. I work with the collaboration team with that project. I'm also active over there. I try to make some projects with a more friendlier environment for everyone. I'm a researcher with the foundation I've been studying, specifically newcomer retention issues for almost a decade. I'm an engineer in the collaboration team so whatever you decide to do, I'm one of the most successful. Hi, I'm Marco from the services team. And so I'm interested in the service side of the industry. I'm just going to join the other team. I'm really used to the the industry. But Hello, I'm Joseph. I'm in the NIT team as well. I'm interested in the topics for the for you to be nice to each other and make sure that we cover something like the NIT team. Sorry. Hi, I'm Diana. I'm a friend. Sorry. Sorry. Sorry. Stop telling people why you're here. Well, because like I wanted to spread video, it seems like making it possible to do edits that aren't nearly as visible in the front page. You were having a little more fun. Yes, I know. And I wouldn't actually love to spread video. We're just saying that we have an interest in helping users or that goes with the drawing and the review. Let's do see people and the same people. What's that? Introduce yourself and why you're here. I was here in the department and I'm interested in the topics I'm interested in. I'm just kind of a career on the science research team. So it's my job to actually find users who are interested. We're just saying who we are and what are you interested in. Okay. Who are you? What is your purpose? To share all the video with everyone. I'm upset I'm getting this a strong interest to meet users. It's both of them. Great. I'm Greg. I'm here because I wasn't sure what I wanted to do this lecture or this talk. So I'm going to test them all out. Fine. We won't feel bad if you leave. Okay. Hi John. I'm the reading team and I'm interested in a product with a meta product. I want to watch this. Okay. When does it end? It's a half hour session? An hour from now. Got it. Perfect. Collaboration team is involved in a project right now called edit review improvements. The goal of that is to improve edit reviewing generally but very specifically to act on the whole body of research much of which comes from Aaron who's sitting over there working. That shows that new users can have a bad reaction to getting a harsh edit review especially when they're very new especially when the edit review is coming through certain automated tools. This can be discouraging to new users if their work is reverted or deleted or whatever. It can be discouraging. It can have an effect on that. Most edit review tools of course have a very important role of safeguarding the wiki and its integrity but they can have this bad effect and so the question is how do you effect that? How do you do it? ERI has so far in its first phase really focused on yes? We need you to input the master in the laptop because it's one now. So now we just need someone to magic everything and so on and so forth. So you know what would be super useful is if you can find this document and you put your name in and if you're interested it would help us to there we go. We've been focusing so far on helping edit reviewers to the hypothesis is that if we can find people who are users who are in good faith but who are making mistakes we can segregate these people out for a gentler and kinder edit review and hopefully have an impact ultimately on retention and good vibes and everything else on the website. So the first phase of ERI of edit review improvements has been about identifying them and so we're using ORAS I think to identify users who are in good faith and we're rolling out tools this quarter on the recent changes page at first that will help you do that and we're also creating a feed that will let you do that review stream which is meant to go to it's actually very germane to this discussion because that feed is meant to really go to some of these third party tools that are really used in edit reviewing like Huggle, like RTRC, like some of the other ones. So we have to go through a process of getting those tools to ingest that feed getting the users to understand that it would be helpful to them to have some of these new tools and to building tools that they find useful so part of what I want to think about today is what would it look like if we could identify these good faith users who are newcomers but are having problems what would we do with them what would we do with them on Huggle what would we do with them on RTRC what would we do with them and all the tools that people use to edit to review edits that's sort of what we're on about today so like I said we're currently working on you know I could I suppose we could show you if people really can't imagine what this you know if you need help imagining what some of these tools are like we're working on putting it out there right now the stuff that helps you identify it on the recent changes page and in this new feed I need if I was clear at all does anybody understand do you guys understand what I'm talking about okay perfect so I don't have a lot of prompting questions or anything but I'm very interested in hearing from the audience like I said especially those of you who are involved in edit reviewing, who are involved in patrolling who have experience with supporting new users and things like that what are the things that we need to know about that you know that work that don't work, what would it look like if some of these tools were more friendly to these struggling new users who are in good faith or is there something else we should talk about who here are reduced in controls I don't understand that perspective I have no perspective I've noticed as a user if you get an edit revert it's actually really difficult to recover what you did so that's the thing that's always on my mind is how to recover maybe to leave a new article which is a little bit similar but more importantly like a change that you made to another, an older version of the article and then finding a way to re-integrate that later in order to fight for your case or to use it someplace else or you'd like a boss to change itself then refine that change and then re-apply to the article that's an interesting idea where would that take place how would that be managed like right now you have to what do you do, you click two versions and you get the difference between them and then you reference that URL so you kind of have to know about fitting in this whole concept whereas maybe in an article you could say your edit was reverted to that, that's friendly you could just point to the album and discuss it and then we have the top pages or click to rewind and then try to software and then say have you discussed this to try to compromise but it might be for a long time if you're coming in a couple days later or a month later, it might be too late so my interest is always from I think we need to come up with you know, on the back end storage mechanisms for reviews that are independent of that single reviews you can see so that in some sense if you grant that there is some sort of these then you can talk about what the UI of the URL would be as a strawman the sort of GitHub distributors that are like the pull requests and when I made the edit there was some names for the namespace something like that that was my copy that was reverted I still own that change and I made further changes to that change so that change lives independently of the artist do you have people working for it by side or everybody can handle it? well, I mean everybody can but it's at their permissions level to convert my point is I think those are separate requests so I'm really interested and why I came here is to hear more ideas for the UI side like what would we like the interface to look like for ordinary people making edits like what's the best thing for the users and have that I think there are general tools on the back end that can be used in one of the storage sites we can solve the name problem we can come up with something about what your edit was the question is how did you get to it? was it on your user page? did they go through your user page? just sort of what you do in GitHub or did they go through the history page? I think someone suggested that when I went back to look at that old version that there were links from there that would let me present it so I don't have one of the questions I have at the angle of the session is is it really about tools or is it really about the people? in the sense that sometimes because too few people are trouble looking they don't have time to actually be friendly next time and is it about the number of people or is it just a question? well I think one of the questions we had was whether like as you say a busy vandalism fighter who's going boom boom boom boom boom boom it's our intention to help that person to add the means by which that person could identify that they're looking at someone who's a newcomer who's in good faith so the question is do we try to focus on that person because that busy huddle user was going along boom boom boom boom boom here's a good faith newcomer I should go slow I should send him a message or should we expect that of the huddle user or should we expect that he's going to say you know what I'm going to skip this and let somebody who's a specialist in that so do we imagine that if we were to provide the tools there would be enough users who would become sort of in other words by creating the tools that allow you to identify your faith newcomer who's having problems we create the opportunity for there to be a new type of user support activity this could now be a thing of supporting new users during editing like edit review has never really been a convenient place where you can support new users but you didn't have the ability to identify these people so is it plausible to believe that people who like supporting new users would come to this if we provide the tools as a separate platform like I'm going to support new users in the edit review process or should we just expect that the people who are already through patrolling should say whoops I need to put my nice hat on now because I'm going a little slower on this one or is it going to be both those things so I think that by surfacing these things we get a really good opportunity to see what happens. One thing that I would really be passionate about is trying to prescribe what should be done but you know I appreciate it as you said Joe that you know well how about we put it out there and see what people want to engage in for what is worth in my interviews of patrollers where I ask them about how they think about good faith editors and that sort of stuff a lot of them did have substantial thoughts about good faith newcomers but they ran into this problem and they just didn't have enough time to deal with it they couldn't really know at a glance if this editor was had a series of edits that were problematic or it was just a swan and so I think that by surfacing this we can find out what would people want to do with it from there I think that we're probably not going to be done engineering just by surfacing this I think that the things that people want to do next we're then going to have to engineer on top of that and after maybe a few cycles of this we actually hope to engineer the right thing So you're talking about one of the things they would want and I'll follow up with that question is they would want the sort of information about the editor they're reviewing that they can get from Snowball for example where you can see something about their record about the things they've done in the past I think that's very plausible but that would be something people would want just that One of the things that I I think there's two issues that I have One is that at the moment even if we can identify the type of editor that the person is making and then also then send them more materials the materials don't exist and that isn't a technical issue that is a documentation problem every length that we ever provide any new editor to is an encyclopedic entry about how to do one thing it is hundreds there is a lot of work that can be done by the community in terms of turning the encyclopedic entry into something that is actually user friendly because we already provide links to all these resources that they are so next and so inaccessible that any person who is saying who gets to that page will go you know what I'm not interested The other thing is that to be an intervention by an actual person we should be able to detect within the interface the type of editor that someone is doing and provide the materials at that point before they even click save to be able to actually identify what this person is blacking the page there's a standard thing everybody tries to black the page and the resources to go hey, it looks like you're trying to do this or hey, it looks like you're trying to add a citation or hey, you're adding something but you haven't yet added a reference if you want to learn how to add a reference then you click on it and it will take you as that walkthrough actually having those health materials being provided to the user at that first point of contact when the person gets to an edit page is the most important time rather than yes, there is always going to be a need to identify those types of people to volunteers so that directs their communications but before that even happens as we're identifying that type of edit we should be providing that there and then rather than hoping that some sort of interaction with the user further down the line go ahead and I'll be back here I have a question about something similar because I see this in the cycle so someone is making an edit and there's an interview so if we're talking about the editing process then we also need to have so how do we see from the editor perspective this newbie who made an edit and then was referred to it so what does the message indicate to them how far do they understand about the process from from this message and how does it work and on the other hand if you are here in review whatever I think it's a question of what parameters or what information you need to know about this newcomer in order to identify them as newcomers so it's like a number of edits or I don't know what kind of factors right now we've established a new filter I think it's a new definition of a new error user which shows that the people who begin editing are most vulnerable really in a couple of first few days essentially I think it's a new cover definition is 10 edits or how many days 5 edits, 4 days something like that pretty well so when this filter but then there's a second level that's also we're calling learner so how do you define this new cover right so right now it's defined pretty low the newcomer is very new just like we just said the learner level is basically the same as automatically which is one and the first one to find where they need some 30 days so there are less than 500 days in 30 days these are new definitions that will show up in a lot of new tools so that when you're editing you'll be able to see these levels of experience as one of the qualities of the edit are these things that show up in the cover it is our intention that they go like yes that would be awesome if there was like a universal language for texting and we also define based on for those of you who view forest we've been doing a lot of work to make the interface more sort of standardized so that it would be standardized across different tools so we've worked on establishing certain categories of how damaging it is how good faith it is and this will be more okay process now we've got anonymous gdocs asks it has been waiting a long time has anyone looked at what wikihau does would anonymous and gdocs like to verify with you but yeah, he's clearly thinking of something that he or she likes I want to get back to does that give us something more next that you're trying to go to which is that you mentioned the problem of when you're trying to reach out and help this person how do you help them because it's not always all that helpful to get a list that brings you to these specific entries I don't know how you can get around that except that one of the things that we've been working about is there's another issue of when people are from our research we found that when people are reaching out to help other people one of the things they want to know is that the person wants to be helped because you can spend a lot of time writing a very polite and friendly answer to someone who never gets back to you never reads it and just goes off into a well so one of the things we've thought about is if we identify that they're having problems as they're saving allowing them to sort of sign up to give them a message that says something like our robot thinks you might be making an error or a problem would you like to have a special constructive review by someone who signed up to do constructive reviews if they said yes those people who these hypothetical people who are doing that would feel a lot more confident in reviewing that queue first because they'd say well these people actually asked to have somebody intervene and help them that's one of the ways we can get around that issue of just giving them a bunch of links because the gold standard of helping people seems to be actually having a human help so with this in saying that we articles would have a measure of how good or bad they are will there be a way of doing that with editors that provides a score for some editors that based on that but essentially where I'm going with this is one of the things that I work with is central editors if I can be able to target editors based on how good they are how new they are then I can be able to provide them resources directly to the point in their progression and their growth and so on with editors we have people who applaud to clearly identify with bad editors but I mean it is there's some hesitancy about scoring editors I think that's likely possible so I have two things that I want to say about that so one is yes we can we can model a relatively high accuracy whether a new editor is editing in good faith or bad faith you essentially just take your edits together and say is this mostly vandalism according to the vandalization model and that gets you a pretty good direction because vandals consistently vandalize and now if that kind of scares you a little bit about modeling editors there's another session I'm going to have on ethics and transparency so I encourage us to be bold and model scary things but also try to make them less scary so the answer to the question is yes and we have some published research on how to do it yeah well if you're as my director I think at the moment we're more considering not doing that but maybe providing some context of this person who has been reverted this many times or this person has you know had a warning in the way that's sluggable without necessarily legally yeah so I was just going to go back to the psychological ultimate focus is the person may get reverted and they feel like I'm talking a bit about separating the idea and the edit from the revision and one of the one of the sort of near-term things that we do is for the revert bot instead of just saying reverted it is it was moved instead of just being reverted it is now at user space slash your editor whatever the place is and that's also we talked about if there are humans who want to help maybe they get it better and that's the place that they need to be pointing to but it also kind of goes back to this every class that you think it also means that vandals now have there's also that thing about the web scene making their head more visible in that sort of way good edits are more visible in that other things like slash that out or step over flow give this with more or less reputation karma on this kind of idea where you have some leeway to revert things until you're metamodernated to say that you've done it poorly several times so start taking away those risks for you and it seems like that's a natural way to go if you want it to be more automated more community focused and require less intervention from us but it's also kind of scary because it can be gained and if you automatically are a class in my head you're not doing it in an invisible way but some reverted edits are searchable medical in some arms so if you're a good faith editor but a model it's actually easy of mine and if you're bad because you're by the level your head is not easy to stumble across you can't discover that around your chest maybe sometime later you will become a good editor or edits will become searchable again the idea you have was a community of aspects of how to protect our new colors and I think the wording is very important as you said maybe just changing the tune or more coordinated messages for new colors also makes sense and adds itself with little sentences to the reviewers that says this guy is a new color just say that the way back to the new color you've just changed the theme instead of reversing because reversing is kind of a hard work for somebody who don't know what it means it's just been moved into the story of the page and now I suggest that you go to that person to discuss why it is doing reversing etc with suggestions if they want to take actions like that you don't even have to ask if they want for help you suggest them the help they can find it, they send an email it's a bit easier I think that the flow and the technological aspect as you said of being nice to them between the wording is almost more important and also I think wait I want to capture that so your suggestion has to do with making sure we reword a lot of the messaging that's the standard messaging well at least an English Wikipedia was written by the community and we studied that it turns out that if you make the messages nicer, by the way the current version of the messages are the nicer version of the messages we had a professor at Revver go through and try to experiment with different rhetorical strategies for being nice not calling the editor out and making it explicit that the message was coming from a real person that sort of stuff it didn't work for making newcomers stick around longer however it did make mammals stop analyzing faster and so it's sort of like you know if somebody cuts you off the track if you flip them off or do you smile and wait if you flip them off they feel justified they just cut off the dirt but if you smile and wait well maybe they feel bad because they cut off a nice person who would smile and wait and it seems like it works that way they don't quit well they stop analyzing and you leave but the good faith newcomers who would have this did not stick around longer it's going to just relate not just make it nicer but make it clear what the next step is if you work if there were a good faith at it what should they do next so I really sort of hear that the current version of the next version that's really like I think there's a lot more experimentation you can probably do in that direction because when you hit it on something like Google Maps it's like thanks you're a good person like you've contributed to a broader knowledge base and it doesn't it doesn't cost anything to do to send that to a man but it doesn't cost anything to send that to somebody who's mature but the reward is so much higher they also use like fully formed HTML it looks nice it's kind I think there's actually a lot of room here I'm curious to hear so there's studies on that too automated responses well go ahead automated responses generally haven't worked I don't think that we've exhausted these potential positive automated responses but responses from humans that say hey I appreciate what you did that has substantial positive effects on retention and future activity that sort of stuff so I can get references we really need to convince about rewriting messages because I've tried to do that actually we did have nicer messages but people were like ok people are not acting on those messages so we are going to build big and red and like that messages saying you are found out you have removed the typo or something like that so yeah it's frightening for people to see things like that and we need to educate the communities to change I would think about that because as you mentioned some people use recent change tools just to shoot at movies because you need to go fast because there's a lot of bundles in front of the wolves you need to definitely because we are night ok that's not the red in my opinion that's not the red yeah that's not the red we call them cowboys cowboys and they don't like it because you know they shoot first and ask questions after which is not that cool so your point is we need to get the research out there that shows that better messaging works but we also already have researches, the question is how can we change minds in order to have people who are friendlier with new viewers messages which are better and there's also a question which is we don't have a very good interface and I think some people don't read the messages because they don't know how to read the messages I agree with the thing that we can make the difference between newcomers or we should make the difference between newcomers and experienced people and broader studies across software usage show that there are at least three levels of experience around software and the message you expect from the software has different level of experience can be different, it may be should be different it might be an interesting idea to suggest messages based on the expected level of experience for newcomers and the message that is that way, or term that way because it's important for very experienced people who know what university means you know, you probably already know why and you don't mean to have the same same kind of things No, it's true because I'm sure it's easy to reference things and messages that do more to me like your talk page how are you going to go on the other side of that? I feel like that siege mentality if you felt like you were provided with 500 new edits a day you had to get through them every day and feel a lot of pressure if someone says, hey slow down, be nice to these people you're like, I got 500 edits, I got a lot of things to do and so one of the benefits of some of these tools might be to segment that stream and so if really obvious vanillism gets dealt with by the 500 tenants per day people like, that's great, those are the front line defenders are great, but maybe we can segment it so those guys never have to see the small subset of these so viewing good faith in that case is not in terms of of actual faithness, as much as volume rate if we have a 10 edit a day stream and a 500 edit a stream we could probably spend a lot more time on the 10 edits and maybe we should have different tools different views for those and make sure that the people who are doing it I mean that's one of the things we thought about I think one of the things we'll offer to the community would be the ability to just filter out the good faith people and then we have this other question of who's going to come up because there are more you'll be happy to know there are many, many more good faith people many, many more and people are confused about that I know, it's funny Aaron, perspective so I just wanted to this is something that I measure a lot it takes about 267 hours of work every day to review the recent changes feed personality and so that's 33 people working 8 hours a day and assuming that some of them are working just English how many people do you think there are I think that there is probably something more like 10 that do better than any sort of regularity but they use automated tools that help filter this stuff out right now these tools only filter out things that are likely to be mangles and things that aren't likely to be mangles and I think that that dichotomy is super problematic so I appreciate that you brought up two different interfaces we need the models to also follow more than just the dichotomy of the interfaces show more than just the dichotomy that's it I'm a member of the community I'm not going to sign up to spend more than 8 hours a day reviewing it it seems like other people are doing it but you know I'll sign up if you want to give me like 1 hour review test today do like I can do that I feel like the next thing I should be looking at hours and speaking of signing up and being given some amount of tasks one of the things that Python calls its backer too is that the review interface is crap like if there isn't you can't really what we have we can't really call user interface for reviewing contributions typically on every review to do there is I haven't tried to read the documentation about it but I'm sure it's not very good and not very findable to some sort of a point and as much as we do need documentation about how others need to behave we also don't have either good documentation or good infrastructure for reviewers and what they should do and what they would expect to happen this is kind of a Twitter problem this is why Twitter causes people stress is that they feel like I've got to start here I've got to read it all right and that's kind of stressful and Twitter at least is experimenting with ways that we've also chosen so I don't think there's a solving but I need to read it all because they want people to have to read it they want people to spend more time on Twitter but for us we probably have a different thing like it would be nice if we gave people an interface that let them feel like they were done after a period of time and that's absolutely true that came up in a lot of our research conversations is back loads or stress loads for people what do you think? a little over here and a little over here I'm guessing a big number of the good paychecks that we've still never heard are things that don't necessarily be wrong on media I'm wondering if there is any thought about suggesting other projects that might be more suitable for the content like it's like a geographic place tag and wiki data and having some canned response that says hey try putting this stuff on wikiboyage so that was a question about Wikipedia right so this is I don't want to derail it but this is where I said if you make people actually be able to own long-lived revisions and work on them for a long time eventually you get to a point where they can have a fork of their pay which is long-lived which is edits which have never been added and will be added because people think that's what their popular culture or their not user point of view or whatever so at some point you get to this like it would be it seems like it would be nice if you could actually port these edits over in a more nice way so like I have this long thing I've worked on for a long time which is it's better to block some wiki source or wikiboyage and you can just move it without copying it but that also means I've got this thing which I worked on for a long time which is really block the conservative pedia or liberal pedia and do people are more or less afraid of forking Wikipedia that way whatever the fact and I'm interested in talking about I think it might be an off topic but I should the suggestion is to try and diagnose the problem and suggest the answers automatically or at least have it available in the reviewer interface if something has like a more possible review so I see it's boom you already saw it suggesting it so do we have numbers around like edits that were recorded in that state like where's the percentage of these and how would we use these because if I'm someone who sort of medically am like would it make a difference if you tell me hey this is the dead end fact that you work for a contractions this time like you're saying Erin do you want me to just set the level so I didn't catch the she was that word so so this is this is a topic that I've heard brought up before I haven't figured out a good way to detect these and quantify it I would be really interested in this would totally take me into volunteer hours if that was required to build an interface where we can have better operation of rewards and start finding out how to do bad rewards happen what do they look like who's doing that sort of stuff we don't even really have a good way of thinking about what they are and how often they happen what sort of effects they have but I think we can get there but edits to bad faith happen less than 1% edits to bad faith are about 2.5% of all edits edits that are damaging are about 10% and I think part of that is that we don't have a way to have a conversation around a regroup if we made it out of more first classes and had it flow underneath that or something like that then people could and people got notifications when their edit was recorded they could then point other people to where they go to the conversation and sort of like moderate it this goes in the direction of what was suggested Mr. I think a branching system more than an actual tool that goes between river to river to river but branches for rivers we come back at the end it's all one page but the view of this makes it easier for people to understand I was actually asking about the numbers because I wonder what you know our minds do we expect some process to be like 100% successful or everyone's happy some ever been happy but I think the development was like right people would actually are at sea hopefully another question about the numbers Aaron you just said 2.5% is damaging the rest is probably good phase maybe not ok but the rest of it is good shouldn't be remembered but is that more fine grained like how many of those require further revision I guess like someone makes a change when it contains a title someone says well it would be nice if I was better but I'll just do it with confidence and revert it that's a fair question essentially the way that I ask people to think about this when they evaluate it I ask like it's damaging if you think that it should not continue to exist in the article so you would revert it you would make a change immediately to fix it and so I believe that 90% is stuff that doesn't even need to have a follow-up it's not problematic enough to need a change but this all depends on what Wikipedia is thinking about as they were doing this they labeled consistently I gave those instructions but I don't know what happened in their heads when they were labeling so if you adopted my program and all things like had a different name and the team is bad and converted you might actually get more reverse immediately as people fix them themselves or refine them so there were a couple points that I wanted to add to the discussion from earlier with regards to doing work we don't like there's nothing that I found that suggests that certain amounts of work is important so like doing 8 hours or doing 1 hour but you can really frustrate people with a chunk of work that takes longer than 7 minutes and try to read a blog article that takes you longer than 7 minutes and find yourself getting frustrated so I don't need to read 3 search papers but this is like a really strong regularity in human behavior you want to make the work basically 5 minutes 7 minutes is a strong put off and then with putting a conversation 5 minutes over and over again until you feel like you're done you want to make people consistently like that there was another thread that I wanted to get into which was about putting like flow conversations or something like that on every revert on English Wikipedia a lot of the other wikis there is the old revert discuss cycle which is the idea you should boldly do which you think is right, don't read the rules just do what you think is right if somebody reverts you you should have a discussion we have good research that shows that this breaks down especially when people use automated tools for applications and flow to try and make that actually happen there's a reason why people advocate this is a best practice thing to do there's also reasons why people don't do it which we'll probably address that so just to follow up on that the reason why they're not having a discussion when the automated tools are used is because they don't know with whom to discuss it it's not clear that somebody would leave a message or something like that just boom it happened there's no recourse is that typically that kind of a controller is the reason to discuss it either is that what it's about so in other words one description then would be to make sure that there's somebody to trunk to or to encourage somehow the reviewer to let themselves be available to discuss but I think it's designed for more for the owners to be onto the reviewer to start a discussion is that revert discuss cycle that when you revert something you at the same time if you perceive it as something at all controversial if you perceive it as something that is not included in the garbage then I think in many cases the reviewer should be encouraged to start a discussion that's probably the only thing but you gotta aim not to do it that's a problem so in my perspective do you have a cue there are I was going to say I'm the first one, two, three just an experience on the French Wikipedia all automated messages are ending by if you want something you can go to the database so when we have done that we have reduced the increase of people who are asking for help so I think that was great but again that's not real still but that's an interesting thing and we also have some photographers who don't want their name in the messages they ultimately left because they need to discuss with people what they have done and sometimes they don't want to justify their senses which is I think probably right I was thinking about the first point to sort of surface the impact if we didn't make first class page kind of thing then we could close statistics or specifically personalize statistics out from that and this person stopped after this happened and so we lost one person for quite a bit of the case can I maybe after you yeah I think part of the issue is that we have a social expectation for how you handle controversial items like if I edit a critical article and it gets reverted then it understood that I appeal follow up or re-discuss pattern and there's a social prevention about that there's no way of social prevention for how you handle a good feed from a problematic edit regardless of whether it's good feed I don't know how to follow up so I think we need to create a sort of prevention for how that is to operate and I agree having it be like a first class page would probably help with that so a convention where if it's weighted ban it wasn't like someone's blinked a page you're not expected to start a discussion but if it's like oh they added like a blinked a vlog and that's not really positive you do have to start a discussion about that so forcing actually reviewers into to choose a path whether they've gone down no this is just vandalism or whether they've gone down in faith but this needs more good faith follow up I was encouraged by that I was going to suggest something in between which is making explicit this sort of triage in the doctorate division that maybe it's okay if the frontline infantry doesn't have their name on the reverse as long as there is a responsible doctor behind it who's willing to actually talk to people when these things happen and then the other thing was I think the side effect of the bold revert discuss cycle is that in many cases people aren't bold ever people start out with good faith discussing but then they feel like there's no one who is willing to take responsibility for it so it never gets done so like there's a bunch of articles that he title changes because like the thing that they're describing changes name because the company changes name but like shouldn't change name and then the talk page is actually just curious about that and then the last thing was I can't remember it so I'll talk about it wait just as a response to that point I don't agree about not having frontline infantry having their name on it because once you start disassociating responsibility of actions like when you're reverting it is your responsibility for what you did, why you're reverting them if I'm reverting something it it would be unfair to expect somebody else to explain my actions but you could explain it to that somebody else like I think these patrollers feel comfortable being responsible within the patroller community it's just they don't want to deal with every single troll individually that's social work that's harder for life that's a complicated process I mean someone saying what someone else has said because we have an institution the first thing I do in this process is to actually use the tools like I'm responsible for the things I do and this person I send it to somebody else because I'm not a troll manager yeah I think yeah this is probably a very naive and I don't know if patrollers would appreciate it but one thing I noticed about the revert workflow is that you have no opportunity to explain yourself so I think it would be really great to force people to choose why they're reverting something when they do it it would also be very useful information for us later to understand just a little bit about the mindset of what's going on when people are full reverting there's supposed to be a message there's no way of positioning no when you click rollback there's a standard message but you can customize it no no no when you click on do yes although it's free when you click rollback there's a lot of operations there are many just for the reason yes but yeah I was going to say I do kind of like the idea of a simple triage doctor I do think some people want to do those roles but I like the idea because the people are like feeling like they have to do 500 heads a day and they may not feel like they have time to communicate individually with people and even if they want to they may not actually be good at that they might be good at determining what needs to be heard but not good at discussing it and I think if you like tag it properly this is a reliable source problem in theory you can let someone else discuss that in a friendly way after they hear why you're reverting there should be a accountability I'll just say I think if you made some requirement that the vandalism community had to be in communication with all of them some people like it but some would completely revolt they would well you can already using some interfaces you can already specify why you're reverting or why you're tagging it I have these concepts on my page Trio there does need to be consequences if you're reverting something that's actually fine and using something else to communicate your problems but the consequences for that would be that you would lose your patrolling rights if you mess up too many times there's a better moderation so with the Snowball interface which isn't really about edit reviewing we still found this really strong result that about 50% of the people using the interface did not want to have interactions with other people even though it was explicitly a newcomer socialization tool but they really wanted to help by triaging things and the other half really wanted to actually have interactions with people so I think that review tool, socialization tool we're going to see that everywhere some people just want to help in triage but they don't want to talk I think it's related to being under siege if you've got 33 hours per day of stuff to get through you can't be distracted by everyone who is a guest but once you've gone down past this triage now we've got a few people the other thing I wanted to say it's just not some people's arrangement well yeah, but right here some people like to be in a job or some people like to be in a doctor but the one thing I should say is that the person who's actually is quite possibly neither the doctor nor the triage doctor like a lot of pages have informal maintenance who are the people who do most of the constructive edits to it and so if I wanted to have a discussion about whether or not this possibly corner case was in fact on topic or not I don't want to talk to either the first ingredient or the other person, I want to start a discussion with the community who maintains the page just to keep it up so we have 10 minutes left did our IRC person ever get back to us about their question about we are 10 minutes left so it may be interesting to have the point of view of people who don't have spoke yet so if you can take the opportunity please take the opportunity I'm seconded I'm quite curious I'm sorry so I'm like I've never been in the course of being a little in the picture or anything like that and I'd be quite interested in seeing getting into it reviewing changes so is documentation about or is there a way of getting more involved in that I guess oh sure there's endless information about yeah I just wanted to talk about like newcomers in just editing and being more familiar with that I'd be quite interested in seeing what would be the process of becoming an active contributor in terms of visually developing changes I'm not an expert in that but I can show you a couple places to start if that's interesting that makes me think maybe the reason for the season mentality and all that is maybe we don't have enough people being already the patrollers right so if we could increase that number maybe that table goes away and hopefully it gets more relaxed by itself I would say I really like Adam's idea especially if it was possible to maybe like have only a pre-populated list of options that people could select to say like this is why I'm burning other perspectives and have that information be able to say like very easily 7% of reverts are going to do this and the person who's getting their reverts is like C it's like an easy formula to fill out as we're doing it and they can say oh I didn't do that because it was I don't know if I'm doing it but then so in general it's like it's done new page control you have to make sure that the options are limited because otherwise I'm just going to pick the easiest option every single time there's a lot of options now right the number of different versions and like either you provide some sort of searchable interface that I can take a word and it will predict for me otherwise I'm just going to go general submit next and I'll do that unless it's a specific case like generally speaking I've probably said like 50% of my reverts maybe more I don't know but I would do like a general thing if it was a specific case particularly if it was like this person has done something very specific that is a major issue then I would then look through and find a specific one I'm curious if y'all think this is too complicated but maybe like the percentage that or as a science student this is likely being a symbolism makes it so you have a simpler interface then you just revert and the less likely it is that symbolism the more it makes it required then you can have one of the reasons and give a sort of justification interesting hi hi I think or as can do I think a lot of people are wishing it's not great correct me if I say this wrong here in diagnosing exactly what's wrong with something that it says is probably damaging it's not going to necessarily tell you you know we have abuse filter that can tell you certain obvious things that you did wrong and then that can trip out of the wire very easily for these other things it's its own brain and it's not always obvious why it says what it says so it's not always obvious what the prescription is but that's not what you're saying interesting I think that if we take like the upper probabilities that something is vandalism you're going to find things that are pretty and reach us there most of the time when something gets applied there somebody knows what they did in the description it might be nice to be nice to them so they feel bad for what they did so I think the network I think we can definitely do that we already sort of have levels built into the filtering system that we're working on so it might fit very well into those levels too we have 5 minutes Julia wants to wrap up alright I don't know what I'm going to wrap up with except that this is something that as I said we've been focusing so far on helping to identify the struggling good faith new users and enabling viewers to identify them and not finish with that work yet we will shortly release some new faith so I will use the changes if you want me to demo it I'm proud to do you find me around here this week and we're working on the feed as I said it will be something that has all this data has data about newcomer levels it has the standardized data about your 4S levels and we're going to be releasing that feed this quarter probably next quarter will be the quarter where we're really working with various communities around some of these most popular tools to get those features that are now in that feed into the community so during the next quarter we'll really be trying to do some thinking about this research I can put some links that may already be in this note pow has made some interesting explorations already into what what for example yeah hugo and rtrc would look like if they had some ways of bringing some of this stuff forward and you're very interested in all of your comments and as I said if you give us your username and let us know if you're interested in being about this further or following up or being a test subject reading that stuff we'd love to follow up with all of you and thank you so much for coming for a very interesting discussion you've shared with us a lot what's that very good people have seen them you're very kind I'm impressed that things go on live at home what's the point of being a manager if you don't even know what your people are doing I mean those are those are designs that are meant to help us when we say to someone in the public we're thinking of bringing these features to your tool we'd like you to work with you to do that we're talking about we can at least show us a picture so those designs are they're very political thanks everybody oh I'll just say just because I feel like everyone's room there is a session today about reader and writer engagement on board or support so it is now happening today it is actually in this room I don't know if it's in this room oh is it? okay cool so yeah oh and if you look at the links that are in the fabricator document you'll find the page for this one the our features for special changes and there's kind of a nice if I say so myself video demo of the the changes we're bringing to recent changes on there so you might want to watch the video no we don't have any time okay it's lunchtime so we're going to discuss about all the other things that can match together yo don't listen you I'm pretty I'm pretty I'm pretty you guys are having a flowy discussion that would be really cool we already don't have structured data about so it would be wonderful to bring structured data and explicit discussion so it's two characters that's the most interesting I think I'm going to do this it's something to see that it's actually one of the main characters I'm proud of comment especially anything we're we'll we'll look at threads the the flow the It's not a problem. I didn't get a job. I didn't get a job. Oh, so yeah, these are... Yeah. Yeah. That's what I wanted to say. Yeah. Yeah. Yeah. Yeah. Yeah. Ah, they mixed that in there. That's crazy. Thank you.