 Okay, we're live. Hi everybody. Welcome to Credit for December 2016. This is our last credit of the calendar year. I want to thank everybody for the great demos that they've been showing throughout the year. Just as a couple of notes, if you have questions for the presenters, please enter them in the EtherPad. You can enter them at etherpad.bookamedia.org slash P slash credit and credit is all of our case. Also, we're planning to issue a survey about credit to make sure that we're reaching the right audiences and to get some feedback on how it's been going for people. So stay tuned for more information on the mailing lists about that. All right, I think we're ready to go. David, can you hear us? Yes, can you hear me okay? Yeah, and could you share your screen please? Can you see that okay? Yeah. All right. And not too much echo? No, seems to be working. Okay, so we've been doing a little bit of work in our spare time adding multi-user real-time collaborative editing to visual editor. So this is based on visual editor core. The part of visual editor that doesn't depend on media Wiki, so there's no media Wiki dependencies in here. And we found just by adding a few classes and a little bit of code, we can create something that's fairly similar to a sort of markup version of Etherpad. So I can create a new document or go into an existing document by name. Can you see that okay? Yes. So here I have an editing session in my browser. And you can see the cursor of add, I think that is. So I can type stuff at the same time as I type stuff. And it's that Rowan in green now, I think maybe. And because it's visual editor, we can do all the formatting kind of things that we normally can do in visual editor. And I can make a link, say, or insert a table, that kind of stuff. So there we are. So as I say, this is based on visual editor core. It's a very simple, minimal wrapper that we've put around it. And there's three main classes. And the core logic to do this is about something under 100 lines of code, say. Visual editor already had a store of your edit history as a list of transactions. So each time you press a key more or less, it's adding a transaction to a list that says that you added a piece of text. Or deleted, or what have you. And basically all we needed to do to make this possible was to synchronize this list of transactions between two different instances of visual editor on two different computers. So when visual editor was, when the code was started, it was already envisaged that this might be a possible use case. And so throughout its history, it's always been deliberately considered whether a decision will prevent this use case. And we've always been able to work around this. So this synchronization goes via a central server. And basically the central server stores a canonical list of the transactions that have taken place building the document. And things can arrive in a different order from different users. And the server has to do a bit of maths and rearrange the order in which things have happened to make some canonical history in the order it arrived on the server. We've optimized to be as minimal as possible here. So the conflict resolution we do is very, very simple. If you have an edit, if I have an edit that affects one selection and at the same time somebody else is working on that selection, then whichever one of those edits hits the server first will be rejected. Fortunately that as you can see as the editing is going along, that's actually a fairly rare occurrence. Most of what you're doing is adding single letters to a document or you'd be unlucky for two people to be changing formatting at the same time. Because of this, the conflict resolution here is dead simple. And so because of that we didn't have to do too much coding around this. Yeah, it's not a finished product. It's a simple minimal wrapper as I say. And it's got things that it doesn't do, but there's a solid core of design that works pretty well most of the time. Okay, I have a feeling this is going to generate a fair amount of interest. Very cool. Yeah, thanks for the demo. It sounds like we might be having some problems with people trying to join the Hangout on Air, like trying to join the presenters list here. So we might need to take a minute or two to troubleshoot and just verify that that is in fact a reproducible problem. As far as I know, the YouTube stream is still working over eight for some people to join. It looks like Brian is giving it a shot. Of course, we can take some questions from the room if they're already. Yeah, I was just able to join the Hangout, Adam. So Ryan, I would just try and, I guess he can't hear me, he's on on the Hangout. But if we could just relaunch Chrome, probably be able to get in. Okay, it looks like he's saying he's going to try reloading. Oh, he's talking about the YouTube stream, I guess. Not the Hangout, so... We have some more questions coming in. Should I answer this? Is that okay? Sure. I guess we can check out the YouTube stream while you answer it, unfortunately. I'm monitoring the stream the whole time, Adam. It's fine. Oh, okay. So yeah, on the first question, yeah, that would be a very simple sort of extension of this would be to have it. This is media-wiki compatible. Obviously, we could drop this whole product into media-wiki, and then we'd have, you know, you could insert images and other media-wiki-specific features. Obviously, there's a bunch of sort of little features in Etherpad that... This is really a technical demo. Simple stuff to add like would be choosing your own color, choosing your own username, all sorts of stuff like that. But this is still in the sort of very early technical demo phase. You can actually visit this URL and play around with it. But don't guarantee your doctorate will be there tomorrow, or even, you know, in ten minutes' time. You pretty much guarantee it won't be there. Yeah. So, yeah, obviously, all of the features of Etherpad we're aware of, and we'd probably have to be in there before we make this public-public. Right. Well, David, I think you can stop sharing your screen, and Ed, are you ready to do your demo? Yes, I am. Okay. Okay, can everybody see that? Okay, and Amy? Okay. Apologies if I break into coughing fits as well. So, as I've written in the introduction, we're getting to the point where more and more people are starting to use VE, and one of the main sort of points of contentious respite, both with existing users and new users who've only ever used VE, is that whenever they do need to switch to source mode, which either because they're an experienced editor and they want to do something, they just prefer it for that section, or because there's a specific piece of complex syntax that's not yet supported in VE, they're currently presented with a huge change in user interface and sort of six-year-old editor that's missing all the tools that they used to having on there. So, what we really have always planned on having with VE, and we've got sort of, we have designs of this going back five years, is the VE would have its own source mode. So, you wouldn't have to leave your editing environment just to start changing the source text. You wouldn't have to reload the page just because you want to start editing the VE text. So, actually, this is going to go straight into source mode. So, let me just do this. Here you are in your familiar VE environment, which is taking a bit slower on this test server. And so, at this point, I want to switch into source mode. Now, usually, this would reload the page and take me into the odd WikiText editor. And here, we just take that text and load it into a slightly tweaked version of the visual editor service that's designed to handle just plain text so we can throw away a lot of stuff, make it a bit faster. And then what we've done as well with the toolbar is basically make all of the tools work in WikiText mode as well by basically throwing the content into a hidden visual editor surface and then converting the generative content to WikiText using a password call. So, for example, if I want to insert a citation, I'm now in WikiText mode, but I have the automatic citation tool. I don't have a link, so let me just put up a BBC article to link to the first one. And provided the citation server is working, there you go. We have some WikiText. Tools can also be rewritten to just work on the client side. So, sort of, I can just control B, control I. And unlike the old WikiText editor, you can actually use undo all the way through, which is some of the basic features aren't even available on the old WikiText editor. The link inspector with autocomplete, which isn't going to work on this server because there are no other costs to autocomplete. But, oh yeah, that's an autocomplete test. And all the other tools, headings, you can change my heading level. You can make stuff into lists. And you can use the media dialogue, for example, to insert a new picture of a Wombat, and a generation of syntax for me. So, this is a beta feature on our test install here, which you can go to and enable it in beta features. It'll also hopefully be a beta feature on a Wiki near you very shortly. And that's the end of the demo. Any questions? And if you do have a question that comes to mind, please add it to the etherpad. If you've been presenting, it might be a good idea to check the etherpad for the next 30, 40 minutes, that kind of thing. Wow, that's really cool too. Nice job. All right, Eric. Yeah, sure. I'm going to get this screen share. Hopefully that's coming across. Perhaps not the most readable thing. So, I want to talk a little bit about the static analysis of PHP. Within PHP, there are a number of tools that people are using, kind of ad hoc for static analysis. Various IDEs that do some things. But there's been some recent developments from Etsy, a project called fan, that does static analysis from the command line as something we can integrate into our continuous integration system. We've been using this on serious search for probably six plus months now. And it's caught a number of bugs that can be useful. Recently, I've put together a configuration that runs this on core. And so, it's not in CI yet, but you can run the static analysis directly. There's a lot of options for changing how strict it is. We use a very not strict profile on core because there are thousands of things that are just old. But I don't want to demonstrate some of the things that it can show you. So, in serious search here, there's a number of... I grabbed an older copy where there's a number of errors. But so, we get warnings for deprecated classes. Brad Yorche recently just deprecated USIX exception to start using something different. And so, this tells us that this is still being used in one place and you can go ahead and clean that up and rerun the analysis. The analysis is relatively quick on my laptop here. Running in serious search takes about a minute and a half, two minutes. This can also tell you a variety of other things, like where your annotations don't agree with what you've been with what you said. So, for example, there are... If you say that the thing only takes a string and you pass it an object, it'll tell you about that. If it says that it only returns a string and you return something different, it'll tell you about that. This can just help a whole lot in making sure your code is clean and doing what you think it's doing, especially when you're refactoring things. Test help, the static analysis helps too. Not a whole lot more to say, not a huge demo of it. But if you ping me on IRC, I can show you how to get it running on core with the configuration that's now in there. I would love to get more people starting to use this. If there's interest, hopefully we can get it into the CI, probably initially as non-voting and start having a little higher quality of code available. That's about all I have to say there. Thanks. Steven. Hey, everyone. Hopefully you can hear me fine. Today, I just wanted to talk about an experimental way of testing that we're doing on Android. I don't know that we'll do this long term. I hope we will. It's been interesting to consider a lot of different configurations. What I'm talking about is automating unit tests for views. If you're a Java developer, you're probably familiar with JUnit tests, wouldn't it be great if you were also testing whatever view you're showing to a user to make sure that no changes in the style have creeped in, or if it does expose an API that you're also testing that API? I wanted to talk just a little bit about some of the different configurations we're testing for and also just kind of show what some of those tests look like currently. A lot of this is very specific to Android, but the same ideas can be applied to other platforms as well, and probably are. The first thing that I think would be a good way to jump into this is just kind of show what a view test is. A view test generates screenshots of a specific view. So, for example, these are all the different views that different engineers on the Android team have generated, and then they took a screenshot of it using a view test under a very specific configuration and said that this looks good. So, here's an example. So this is a view that is used for writing a description for a WikiData description. These strings are kind of placeholder, and I'll get to that in a little bit, but basically you'll define a configuration for a view and then generate a screenshot for that and then manually verify it and then future tests just look at this reference screenshot and make sure that there isn't a difference. So we do a difference between what's currently generated by today's version of the code and what was vetted by an engineer and committed to the repository previously. So this is one example. Let's look at maybe some... Here's a more interesting looking one. So that's kind of the idea there. Now let's jump into the code really quickly. I won't get into too much detail, but I just want to point out a couple things. So we have a view test class that all our tests... all our view tests subclass, and the view test class allows us to parameterize a single test. I made a test for a fundraising card. The new code that I wrote to test that fundraising card under all sorts of different widths and things like font scales might just be these few lines of code here. But because we've parameterized the test, you'll generate quite a few screenshots from that and so you can quickly see all the different corner cases of how your view shows to a user. This is really important for things like right-to-left languages if you are a left-to-right native. Make sure that it looks good in those languages. Or for example, if you have good eyesight, you should probably test on some larger fonts and make sure that your view doesn't fall apart once a user has bumped up their font size a little bit larger. Or if you're a dark mode user, you want to make sure that your view also looks good in light mode. So those are some of the things that we test for. And all of those configurations are kind of in this superclass and you can add more like... if we had a third theme for like a matrix mode, you're using the app and you want a matrix theme so you feel really cool like a hacker. When you're using it, we could add a theme for that and make sure that our views look great and very elite. Let's look again at one of the tests really quickly. So these are some of the things that you might find in a typical view test. You want to test under different screen configurations. So this would be like... I want to know that it looks good on small screens and large screens. You want to test the font size. So I want to make sure it looks good at 1x. I want to make sure it looks good at 150% as well. Test the layout direction. So I want to make sure that it looks good in left to right and right to left. Test the focus state of a button. So if I open up a new view and that view has focus and the user presses Enter, what will be the action of that view and will they actually be able to tell if they're changing focus to different sub-views within that view what view is currently selected? So these are kind of like the high-level view ideas we're starting to explore now. This is still really experimental for us. But just because it's a view test doesn't mean that you can't still do your typical API test that you might do on a standard JUnit test. So here you'll see like, well, this is all the all our code to exercise, to generate screenshots and exercise like the view portion of how it renders. But we also, and you'll see that different string lengths are being fed in here as well. You can parameterize it to no end. It's a problem if you over parameterize it because somebody still has to, and that's somebody's you by the way, somebody still has to review all those manual, all those screenshots initially. We don't have an automation to kind of make sure that something looks good. So here's some, here's some API tests. So here we have a, an API called set callback, and it allows you to put a callback in there that is non-null or null. So we test those two cases. You still, you still exercise this the same way you would a standard JUnit test. The last thing I want to point out is what does this look like when there is regression. So we have a script called diff screenshots that just generates diffs between what was committed to the repository as known good, like a known good screenshot. This looks, somebody looked at this at one point and said this looks great. This is how I want it to look. And anytime you run the tests you'll be comparing against what that person has said looks great in the past to make sure that the current code hasn't introduced any new regressions. And when a regression does happen this is kind of what it looks like. So here's an example that came up I think yesterday where we had changed the casing of today to always be capitalized and it kind of showed up in the view and also I don't remember the reasoning but the title for my reading list has moved down a little bit there. So you can quickly identify not that just that there is a problem but what the problem is at a very fine level. I think that's about it. I put a link in the chat or in the etherpad about how this is integrated into CI. You can see our job the job is still pretty flaky when it's executing in CI we're slowly figuring that out but when you're running locally it should work pretty well. So check it out. Thanks. All right. Very cool. Static analysis and visual unit tests are very elite. Nice. And it looks like we have at least one more demo Eric. Search click models. Yeah. So something I've been working on with relation to search recently so one of the big problems we have with search is that whatever you turn one knob 100,000 queries get different results and you have to evaluate are those results better or worse? It's very easy to look at one query and make that one query better but you can make all kinds of other queries worse and it's hard to know whether you're doing it or not. So you may remember a few months ago I presented a project called the Cernatron which collects explicit user feedback and explicit meaning we are explicitly asking people are these results any good and they give us feedback and that's actually turning to one of our most powerful ways to evaluate changes to search. This is investigating a different way of doing that which is called implicit feedback where we take behavior that users are doing such as clicking on search results and turn that into a measure of how good our search results and turn them into lists of things that we can then evaluate against when we change our search results. One of the big benefits of this way of doing it versus Cernatron is we have lots of data. I can easily pull 10,000 queries that have been seen by a few dozens of users and see what they thought they were. So this particular one is called a DBN dynamic Bayesian model doesn't really matter what that is but basically it tries to use probabilities to estimate two values one of which is how attractive is a search result which is how likely someone is to click it and then how much satisfaction do users get from a search result which is basically a measure of was this the last thing you clicked in your session. It's basically making the assumption that if you read a search page from top to bottom that maybe you click on the second result but you don't like it so you go back to the search and then maybe you click the fourth result and then you like the fourth one so in that case the satisfaction of the second one would be low because you read it and then didn't like it and kept searching. But so anyways we run it through some thing it takes a few hours to calculate this against a few thousand queries but we get some interesting results out of it so like in this case the query on screen hopefully you guys can read that these are queries that normalized to the 2016 MLB post season. We can see that on average the one that everybody liked the most according to this model was the major league baseball post season. This was in position four to five this is the average position averaged across all the sessions that saw this but we can see that most of the people actually clicked on this one and probably ended their session there because our relevance we have a 90% chance that this is a relevant result for this document. We can also see some of the ones that end up further down in the list you have things like the 2016 national league division series one person clicked this but they didn't end their session there so it didn't actually end up being counted as a very popular thing. But anyways this works for a variety of queries we have some other examples here this one is the night manager this is a mini series you can see that you have more clicks to the second to the first position here versus the second position you have the night manager the exact match got 174 clicks versus the one at second but because we make assumptions that users click the first one they don't like it as much they click the second one and they quit there so they really like it even though it got less clicks we know that this is that the mini series one is more relevant for what people are looking for today. This kind of stuff is really powerful and we're hoping to use more things like this to help evaluate changes to search relevance. This is basically a method of machine learning we're also hoping to start feeding this data into a machine learning model that can learn these rankings for us rather than us hand tuning things but yeah so that's my quick presentation on quick models. Alright does anybody else have a demo they'd like to show? Okay I think that's it thanks everybody those were really cool demos and again we'll be getting a survey ready for people to reflect on credit from 2016 to hear your feedback Cool thanks Adam