 Okay, we're live now. Thanks. And we are getting just a little bit of echo from the room right now. Okay, I'll write the mute button. Thanks. Welcome, everybody, to credit. It looks like Baja is up first. Baja, are you ready? I am. Hello. I will share my screen now. Can you see my screen? I can't hear you. Yes? Yes. Okay, great. Today I'm going to talk about the Wiki Labels extension. It's this one. And so right now we have the Wiki Labels service that runs the back end and the front end of it, which is a labeling service for Media Weekly. This service has a gadget code in it and the back end code that gadget talks to. And what I'm doing with the extension is I'm extracting the gadget code so that it's not a gadget anymore and it's an extension. So currently you need to have this JavaScript code in your globals.js file to load the gadget and the gadget looks like something like this where you can see campaigns and you can mark some texts as valid and valid or good faith, bad faith and other types of markings you can do. What the extension does is it enables itself as a beta feature here. As you can see Wiki Labels, once it's enabled, it runs under special column Wiki Labels and it's the same thing as the gadget, but it's an extension now. The code hasn't been merged to production yet, but it has been reviewed multiple times and thanks for reviewers. And there are many more things to do such as moving all the gadget code to the extension and making the extension stand alone and not depend on the gadget at all or the back end service at all, except for sending data and receiving some responses from the back end. So this was my little side project. It has been my project for a long time and it's coming to an end finally. I'm excited to see it moving forward. So that's my presentation. Okay, thanks Baha. It looks like some people are having a hard time joining the Hangout. We might be hitting some kind of limit. So once you're done doing your demo, you may want to leave the Hangout. Baha, in your case, I think you're first. Thanks. Danny, I think you're up next. That's exactly how I thought you were doing. What I'm going to talk about right now is numerical sorting, which was on the Community Wishlist Survey. It was number five for the year. So this was the thing that a lot of people really wanted, although the interesting thing is, now that I'm telling people about it, I can't get them to be interested. So the problem with numerical sorting was that previously. One second, Danny. Danny, can you hear me? Yes. We're getting some feedback on the Wikimedia Office channel that the audio is broken. I can hear you, but apparently others cannot at the moment. It's very quiet to me. It's probably just a little quiet. Is there anything I can do? I'm monitoring the YouTube stream, and it sounds fine to me, but I'll boost the level a little bit. Sorry about that, Danny. Are we equal to go? Seems like people are saying it's very quiet because I'm on the YouTube stream. Matt Flashin says you can hear it now. So maybe I can try again. Testing. How's this? We're good? Yep. All right. So the problem with numerical sorting is that the old default collation was treating numbers as if they were letters, sort of alphabetizing them. So it would say, for example, like 1, and then 10, and then 100, and then 2. So for example, like 101 dalmatians would be higher in the list than 24. That is silly and doesn't work. So what we found was another collation script, UCA collation, that we could use. And we ran that script first on a test wiki and then on Swedish Wikipedia. And it worked there, and it worked in less than a day. We just ran it on English Wikipedia last week. We thought it was going to take about a day, and in fact it took either seven or eight days. So that was a long time. But now, Ray, it is now actually treating numbers like numbers. So here in Wikipedia in English language films, we've got, you know, the ninth life of Louis Drax and then 10 film, 10 film field thing, which is, if you're into categories, really exciting. So there are some other languages that we're already using UCA collation, French and Russian and Polish and Portuguese. There's about 10 languages in the top 25 that don't have this yet. And so now that we've done English and kind of gotten that out of the way, we need to have some conversations with folks on like German and Spanish and Korean and some of the other big languages to see how they want to use it. And there's one weird thing that came up that we're going to see if we can fix, which is hyphenated surname. So it's something that really we haven't thought about before this. What's happening right now with the new collation is that it's counting a hyphen as higher than a comma. So for example, in category-living people, Jackie, joiner, cursey, which is sorted as joiner, cursey, comma, Jackie comes up higher than al-joiner, which is sorted as joiner, comma, al. And so we're figuring out if there's either something we can do with the collation or an adjustment that we can make to the default script. We are going to see how that goes. But it's nice to have, at least for English, at least like another thing checked off on the wishlist serving. That is it for me. Thank you very much. Thanks, Danny. Okay, it looks like BDA is up, friend. Awesome. So I'm going to show off an application that we got released into production about, oh, just about two weeks ago now. It's called toolsadmin.wikimedia.org or the internal code name for it when I was building it as Striker. And this is basically a new web application to help tool labs, tool maintainers get things done. The name Striker comes from a blacksmiths Striker who is an assistant that helps blacksmiths build their tools. So it's got several features on the roadmap, but the main thing for the initial launch is that it allows tool developers to create new diffusion repos. Diffusion is the Git repository hosting system that is built into Fabricator. And we'll log in here as a demo user and show that you can go over to an existing tool that you're a maintainer of and hit this shiny button that says create new repository, pick a name for your repository, hit the create button, and wait. Oh, wow, live demos work. So we've now created a new repository on Fabricator and we can go over and see that it's actually live now. So we would be able to now publish the source code for our tool pretty easily and share it with others. Got a lot of things coming up on the roadmap to add more features to this over the next couple of quarters. The next major thing that I'm going to work on is making a nicer workflow for actually creating your initial tool labs user account, WikiTech user account. And yeah, excited to see this finally in production. Cool, thanks. Okay, Dmitri, you're up. Hey, everyone see my screen here? Yes. Yes, all right. I'll just show something I mocked together. I'm going to go into the Android app and I'm going to log into it. Oh, I am logged in under my WMF account. That's good. I'm going to go to this sandbox page that I have. Where is it? There we go. I'm going to make an edit to it. And I'm going to make an ungenuinely edit. Made an edit. And now I'm going to leave the app. Close out of it. In fact, I'll just lock my phone screen here. Now I'm going to need a little bit of audience participation. I'm going to paste a link to that sandbox page in IRC, in Office Chat. And I'm going to ask someone to revert that edit that I just made. Just anybody, the first person to do it, will win. I just want to see what happens. Preferably someone who's logged in. Oh, Dan Gary is the winner. But what's this? I see a notification on my device that someone reverted my edit. And if I unlock the device, it's up there. Under my notifications, someone reverted my edit on the sandbox page. And I have options to view the page or to go to the talk page of the user who reverted it. Basic native notifications. How about we try something else? I'm going to go to that page again. Make another edit. And can someone thank me for that edit? Thanks from Matt Flashin. Thank you. Anyway, basic notifications. A couple of big caveats here. First of all, these are not push notifications. This is just the app polling the API and checking every X number of seconds for new notifications. But it's a step forward. I think to have true push notifications, we would need some major architectural requirements. I'm not sure. But I think this could potentially open up a new use case for the app. Like where the app serves as a notification post on your device. Whether or not you want to browse articles using the app, like the view page could lead to the mobile web version of the page. It could just serve as notifications for you. Anyway, that's where I'm at. That's all I have. Thanks, Dmitry. Okay, Stas, I think you're up. Okay, so I'll be talking about some new thing that we are developing in Wikidata query service GUI. Namely, building graphs that are usable on Wiki from queries. So let's take a query that we can see how many musicians dies at some age. So we have this query that gives us the results. And now we want to build the graph. So we have this Polestar tool. And we can see that we can build a graph like this. So something... We have all kinds of options here. When we got the graph that we like, we get, basically, we can get this barely comprehensive text. So next interesting part, when we can go to the Wiki, and we can just paste it instead inside the graph tag, and we don't even need to understand what this thing is. We just copy-paste it. And we got a graph. So, basically, we can generate graphs for Wikis from anything. So, for example, we can... Let's say we want, instead of this, we want a graph of population of London. So we go London, and we want also these dates. So we want... And we also want dates. So this is population and date. Let's see what we get. So we got some populations and dates. So now we can put a date here and population here. And we got some... So let's say we want it nicer here, and we can also choose... We have years here. And now if we do export, paste it here. And that's all you need to get this graph on Wiki. And this graph is driven by the data. So if somebody adds population figure for London in the past or in the future, it's automatically updated on your Wiki page. So that's basically all for this demo. Yuri. Hello, everyone. I'm going to demo some other interesting things. Some related to Wiki data and maps. First, we have enabled Wiki data query service to get geoshapes. This map is actually... It's fully interactive. You can expand it. It shows France and Italy. But if you look at the source of this graph, you see that it basically just retrieves external data, geoshapes for Wiki data ID 142 and Q38. Those are France and Italy. But sometimes you don't know what those IDs are, so you get a more interesting thing, something like that. This is the query that Stas kindly provided. It retrieves data from Wiki data about governors and their party affiliation. And if you look at it, all this is is just a very long and complicated query from Wiki data that gets this data and also some of the columns specify the color of the graph. Lastly, we this week launched map link everywhere. So that same graph that you just saw previously, it's a link. It can be done as a link. Almost the same code. You basically just have map link here instead of map frame and you do not need width and height. That was a mistake. And it creates a graph. This is available on all WikiPDS now, not WikiPDS, WikiBoard, Wiki, basically every Wiki project. And that concludes my demo for today. Thanks, Jerry. Okay, Julian, cartographer for developers. I think you're muted right now, Julian. Yes. Okay, I hear you. I hope you can see my screen right now. And on the same topic, cartographer, I wanted to show you the developer guide. So it's a work in progress, especially with the code examples, but there are some I just, I did an example for creating a map frame and soon I will add to create a map link. You also have the links to the API documentation. So you can just go to the JS doc generated documentation and it should be quite well documented. And you also have some explanation on the Wiki Voyage customization and also the list of cartographer modules. So it will be easier for you to figure out what is what and what does what. As part of that, I wanted to first to show you what could look the cartographer for developers. So I'm going to first take this piece of code. This is basically loading the cartographer box. And once you have the cartographer box, you have a constructor, you can pass a container. So here I'm getting body content removing everything inside and just putting a height of 400 pixels. I pass a center and zoom. And I also have a parameter if I want to add a full screen mode on it. So as soon as I do that I have a map and I can open it in full screen mode and close it. It's that easy. And if I want to add a little data layer, I can pass a group name and then some georgison. If I pass some georgison then I get a point on the map and that's it. And this is not actually the, this is only creating for this map and we have a patch. So within the constructor you can pass data and the georgison directly and that's the actual best way to do this. I also wanted to add some integration that I've done that we have the JavaScript API that is usable. So here is a global script on not only my WMF account and an account that I created and here you can see that it's loading some script for search it's loading from script from commands and for wiki data. This is something that you can already add in your command.js file for those specific projects or in global.js if you want it everywhere. So what this does I'm going to show you an example of wikipedia and so here I'm on wikipedia and I'm going to use the recently I did a couple of months ago the near title keyword for search and I can add for example San Francisco and if I add San Francisco you can see that it's going to represent those search results on a map above the list and you can open it full screen and you can zoom the map is interactive and you can for example see that this is Twitter, electronic container foundation like the internet so this is the these are the search results so I can try something else what am I going to try San Diego maybe and this is really the San Diego that I was expecting I think oh yes it is and so the same thing little integration so this is an example of how the API can be used by any developer to add integration pretty much on every projects I made another example on commands so if you go to commands for example if I click on it sometimes those pictures have a camera location you can see that above the coordinates it just added a little map and again you can like zoom out and look this was taken and if you click on the marker you have the object location and the coordinates and this is working for pretty much any media file right now if I go to if I go for example here and you can see again it's showing a little map above the coordinates same thing for wiki data a little integration currently I mean Montpellier and so if I go to Montpellier there should be some coordinates so it's really hard to find where this there it is so if there is the coordinate location property again same thing it takes the coordinates and adds a little map right above the coordinates and so this is working for again any entity that we have the coordinate location property so where is it it's right here so this is something that developers can start using we might also just enable it in production if users want to and if some people want to but these are examples of using the JavaScript API directly to add maps on many wiki projects I'm going to end the demo with an example of a draft that I'm working on and I'm going to go to the San Francisco page and we would like to offer a replacement for GeoHack some people might not know what is GeoHack here on the San Francisco article for example you have in the top right corner the coordinates and if you click on the coordinates it will move you away from the media wiki and we will send you to that GeoHack page this GeoHack page has a lot more details and there's also some global services so from here you can retrieve the most common map services and you can for example open all big maps and look where is on that service this is not really mobile friendly and I wanted to go there so this is not really mobile friendly this is how it looks on the Galaxy S5 for example and it makes it a bit hard to use this service so it's moving the users away from wikipedia which is not really enjoyable for the user experience so I was working on a draft where if you open the map you will have sorry if you open the map you will have more details button and once which is a toggle button once you click on that button it opens a side pane on that side pane you can retrieve the latitude longitude and your eye and as well as the global services and this is actually a media wiki template that is rendered here so again this is still a draft but for now users would actually be able to edit that list pretty much the same way GeoHack was working because it was a template as well you have the link on the left column to that template and so if you click on satellite it will toggle the columns which makes it a lot more mobile friendly and so this is an example of the iPhone 5 and so on the iPhone 5 same thing you would have more details on the bottom and when you click on it well this page is a lot more usable than the other one and last on an iPad same thing it would render pretty much like this so this is a very early draft this is actually the very first draft and I'm going to open a discussion on Fabricator and soon I invite anybody who is interested to shine in and give their opinion and their feedback so that's it, I hope you can check out this developer guide I'm planning on improving the developer guide to enable more developers to play with maps and I hope we can see more maps on Wiki projects soon that's it thank you Excellent Okay, Andrew Otto Hey, cool, yeah so I am going to be demoing something that's currently called Kazaki as part of a larger project we're calling public event streams which is meant to supersede and replace the back end RC Stream server and also sort of like make it easier for different formats and types of streams of public events to be exposed to the internet so RC Stream is cool, you guys all know it it's based on a Redis Pub sub back end and is built specifically for the recent changes feed and media wiki type data and also uses an older version of Sakurai O, so it's been some desire for a while to re-implement it using a newer version but as part of the you know, event bus project that the analytics team and the services team have been working on together to make it easier to publish types of streams of events and also like to make it possible to make those events that can be made public possible for the internet at large to consume we handle streams now using Kafka which you guys can look up if you don't know too much about rather than Redis so Kazaki is just a Kafka consumer to Sakurai O library so it listens for incoming Sakurai O connections and registers Kafka consumers and then lets you subscribe to certain topics this is still work in progress, we're trying to figure out exactly how this is going to work but RC Stream supports some server side based filtering, we want to support that as well but we also want to be able to do more than that so it's still pretty heavy work in progress lots of things are moving around but I thought I'd give you guys a little demo real quick I'll show real quick so the demo we don't have RC Stream in Kafka yet but Eventbus is producing a lot of similar type of events in much stricter schemas so there's a MediaWiki event schema repository that contains all of these some of them are deprecated but the one that we'll be looking at right now is this one called MediaWiki Revision Create and it's similar to RC Stream except for or sorry recent changes except for that it only has this particular topic with this schema will only have revision create events so a lot of the fields will be familiar Rev ID etc. things like that performer here has information about the user that's making the change, things like this and this is all being produced from MediaWiki into Eventbus into Kafka now so I'll be running a Sakadayo server that uses Kassaki so this will listen on a certain point for Sakadayo connections and create a new Kassaki instance which will hook up that Sakadayo instance to a Kafka consumer and I have a few demos over here that I'll just show so I'll run here so I've got that running here so this one's pretty simple I'm just going to consume a single message just whatever latest message happens to be on the stream right now so if I run that you can see that it's subscribing to this topic MediaWiki Revision Create offset minus one means start at the latest at the end of the stream rather than earlier and here I get a result so this is a particular revision that was just made very recently on English Wikipedia you can see all this stuff pretty similar to RC stream Andrew could I pause for just one second it looks like people would like it if you could increase the font size yep how's that looking I'll do this here too is that better just a moment while we wait for IRC feedback I mean I can make it bigger I think it's probably because it's on the larger monitor that it's hard to see looks like the crowd is satisfied hi I think I froze for a second PD here you know is this better yep great so I just consumed the message many of you can see that conforms to that schema the revision create schema this was a Wikipedia administer page or something anyway that's just the one that we consumed this particular consume mode is a pull based it will only consume one at a time there's also a more streaming mode where you can register an on message handler and it will try to do its consume as fast as possible so we'll just do that real quick you're connecting I'm not sure if I did that right so lost my so this should start yeah okay so now we're consuming a full stream of revision creates as they come in so that's very similar to RC stream as well so next one we'll do is stream with filter now RC stream also had the availability to filter but I think it only filtered if I remember correctly on like wiki database name I'll do something a little more complicated this filtering feature is really sort of like we're not exactly sure what we'll keep what we can actually scale server side how complicated we can get what it will look like for now this is what it's doing so here after I subscribe to a topic at the end I'm going to tell my particular connection to filter for where records where database matches media wiki wiki and where page title matches this reg X so this reg X is my user pages so we'll go ahead and set up that so we're going to stream with filter cool get we subscribe we filtered so now it's waiting to like find records in RC stream that actually match that so real quick I'll make an edit on just my user page I'm going to save this real quick here's cross we actually get something in here fingers crossed we actually get something in here demo effect I'm just going to try it again does that definitely work earlier okay I don't know what's going on there but you know demo effect so there we go so we filtered for something that was media wiki wiki and on a regular expression where the page title matched this regular expression here and again that filtering will that's very like under discussion we're not sure exactly how that's going to work and finally this is a feature that RC stream definitely does not have which is starting from the past now the how far in the past we can go and what this interface will look like depends a lot on what we can keep in Kafka Kafka is by default just a distributed log buffer so it's not meant to contain records going off and to belong to the far past by default we keep seven days and a lot of our clusters depending on how we deploy this in production we might keep a lot more time a lot more history future versions of Kafka will also allow consumption based on time stamp but that is not supported quite yet so anyway so this is similar to what I was doing before except for instead of specifying just the topic name which was that echad media wiki revision create I'm actually specifying that and I'm specifying an offset in the past offset is a particular Kafka offset in the past so it's very Kafka specific but you see that in each of the events that I'm returning I'm also returning the offset of the particular message so this pushes thing offset management off the client side so if you as long as you keep track of these offsets and you're not starting restarting at a time when this offset is no longer available you can start from the past so I believe that this offset when I was writing this demo was with yesterday or last night or something so if I instead now start streaming from this demo starting from this offset oh in this demo I'm also emitting a filter on Polish Wikipedia so we'll just yep so I'm going to run this demo and so this should start streaming from that offset but also filter for Polish Wikipedia so now we're starting to get a lot of records I'll just stop there real quick and we can look and see that yeah what is this in 03 what is it right now this is many hours ago that I'm consuming from so that's kind of cool um yeah and when we get when Kafka actually supports and we figure out how we're going to do time stamp based consumption ideally be able to specify you know something like this type of time stamp in the past of like when this revision was created the real ideal would be to like somehow be able to you know store the entire history of changes to be able to start streaming from any time in the past but we're not sure if we win or if we will be able to actually support that um yeah that is that's it nice okay Eric I think you're up next yeah awesome let me screen share hopefully that's coming across media wiki completion demo so one thing we've been working on with the completion which is that thing in the top right hand corner of most wikis is being able to complete from other parts of the page title so historically this has always been a prefix search you have to start typing from the very first part of the paid title and then you only find things so like right now I have issues you can see the normal one that's currently deployed you only find issue tracker we have two different ones we're working on we're not sure which one we're going to use sub page only splits on the slashes sub phrase actually lets you choose search from any word that's within that's within the page title this is something that will be deploying not everywhere but specifically to wikis that have particularly long titles or ones that heavily use sub pages the reason for that is mostly resource based we're using something like 70 gigs of cluster memory to support what it is now and that would go up to 280 gigs if we were to support this everywhere and that's just not something that we currently have the resources for but we'll be seeing some things like media wiki.org on wikitech probably on the wiki source and maybe a few other places we'll have to evaluate but you can see from various examples there are things like if you were to search voting on media wiki.org currently you'd get nothing with this you find some sub pages other things like if you were to search for XSS similarly the normal one in the sub page you're only going to find the raw XSS page but with sub phrase you'll also get completes for pages that just have XSS in the title like DOM based XSS or preventing XSS attacks through CSS Wikelisting there's really all there is to it for this we're still working on the scoring for this for which things are going to be at the top of which things are going to be in what order but yeah that's basically the demo it's just we're working on increasing the recall dramatically for some wikis for the completion for the autocomplete suggestor on the top right of all the wikis okay if you'd like to post questions for people who did demos there is an ether pad available the name of the pad is credit C-R-E-D-I-T and that's in uppercase people are also available on the wikimedia office IRC channel thanks everybody those were awesome demos and I think that's it we'll give it a few seconds in case anybody wants to chime in but I think we are probably done thanks again everybody have a great day have a great day thanks Adam