 Hello everybody, our next speaker is Stephen Finooken with the mailing list, welcome CI. Please give him a warm welcome. Good morning everyone, everyone can hear me okay? Still? Perfect. So, I'm Stephen Finooken, developer from Ireland working on OpenSec predominantly. I worked on OpenV switch previously, my last employer Intel, and when I was working on OpenV switch, we saw a kind of a pretty common problem with some of the work that we planned on doing. So when we were in Intel, Intel has this DPTK data path as part of OpenV switch. And the DPTK data path isn't really the, it's not the main way of using OpenV switch, so Intel has a lot of focus on it and it's a lot of use in telco and NFV applications. But because it wasn't the main data path in there, we saw a lot of problems where people were submitting patches to OpenV switch and those patches were killing functionality with the DPTK data path. And this is a pretty common problem with free open source projects. Open source projects really, really like using mailing lists. The Linux kernel is based on a mailing list, patchwork itself, which I'm going to talk about in a minute, is based on a mailing list and in this case OpenV switch was based on a mailing list. And the problem is that mailing lists don't work very well with tools like Jenkins and Travis and BuildButt and insert random build automation tool here. So we set out about 18 months or two years ago to try and remedy that problem. We spotted that there was this tool called patchwork, which actually does work very well with a mailing list. I'll go into a bit of detail about what patchwork is for anyone that's not familiar with it in the moment. And what we wanted to do was we wanted to take patchwork and we wanted to be able to hook it into Jenkins and provide a way to use Jenkins or BuildButt or whatever to automatically test the patches that come into our mailing list and continually test them and make sure that they don't regress anything that's already in the code base and basically do all the good stuff that we're able to get using GitHub and Garrett and other tools like that. So patchwork is this tool that takes a mailing list and puts it into a nice, pretty web UI that lets you do a couple of cool things with that mailing list, with patches on that mailing list. So you can do stuff, you can track reviewers or delegates in patchwork terminology. You can bundle patches up and download the inbox files of an entire series of patches. You can keep track of the state of the patches. It works with whatever your internal workflow is as you review it, as you test it, whatever you need to do with your patches. And it works pretty well. So we had a look at this and we spotted there was a couple of small changes we could make that might actually allow this to hook us into Jenkins. So there's four main features that we've been working on. In the original summary for this presentation, I said that these were all going to be included in patchwork 2.0. Unfortunately, life has a way of getting in between you and your development time. So three of these things emerged upstream. The final one, the last one at the end, is still in development. I hope to get that finished in the next month or so and that should make the 2.0 release complete. The first of these features is the best API. So patchwork came with this XML RPC API, which has been around since patchwork was first developed in about 2008. It works pretty well. XML RPC has kind of gone by the wayside, REST, REST for APIs in the new black, as far as web apps go. So we said, right, we'll keep the XML RPC API, but we'll expose this new REST for API. And everything that patchwork knows, we're going to expose that through this REST API. Patches, that's bundles, that's users, people, projects, the whole thing. And that's built on Django REST framework, which is possibly one of the nicest libraries I've ever had the pleasure of working with. The next thing, the thing that's probably taken the most time to work on has been a series support. So for anyone here that's familiar with the mailing list workflow, a series is a collection of patches that, exactly like a series of commits in Git, you need to apply the patches in order to build a logical feature or bug fix or whatever you need to do. So historically, patchwork hasn't supported series. It's supported bundles, which kind of equate to the same thing, but bundles have to be done manually. So someone has to go in and take a collection of patches and throw them into bundle. We wanted that to be automatic. So series support is part of the upstream patchwork at the moment. It exposes patches. It also extends patchwork's ability to track other types of mails, in this case cover letters. So a cover letter is a message that comes before a series that gives you information about what the series is intended to do, gives you a big picture view. So this will also track cover letters for you, and it will keep track of things like having received all of the patches that have come enough that should have been submitted to the mailing list. So if the mailing list has dropped two patches out of a 10-patch series, you're able to see this both through the web UI and through the REST API. As an extension to this, you're able to download patches and patch inboxes. We wanted to make it possible to download the inbox for an entire series. And for the sake of testing, we wanted to be able to download a patch, regardless of where it was in the series, with all its dependencies. So if you had a 10-patch series, we wanted to test patch five, which meant you need to download the first four patches, so you could apply those and make sure that the fifth one applied cleanly. This is the first of the things that still work in progress. There's patches up on the patchwork mailing list at the moment, going through review. I hope to get that merged in the next few weeks. The next thing, this is a thing that's currently being used by a couple of projects. I know DPDK are using it, and I'm hoping to start using it for OpenVee switch in the near future. Checks are simply a way of keeping track of the test results from a given test or build that you run. So we wanted a way of exposing this information to Jenkins so that we could actually go and test something, but we also wanted a way of reporting that information back to Patchwork so someone could go in and get a one-stop snapshot view of the current state of a patch. So not only are you able to see who's currently reviewing your patch and what state it is through the review pipeline, but you're also able to see is that patch testing so many tests. So if you have three different guys running tests, are all those guys happy with the test patch? This is up there at the moment, and it's exposed through the rest API, and it's also exposed to the legacy XML RPC API, which I'm pretty sure is how DPDK are using this functionality at the moment. The final one, the one that's taken, that's kept me up late at night for the last couple of weeks, has been events. So events, this is based on actually how GitHub do events in their API. I figured that why reinvent the wheel. Events will keep track of life cycle events and different elements to patchwork tracks. So if there's a new bundle created, if a patch is created, crucially if a patch is created and then all of its dependencies are met, i.e. patch number five in a series, if the first four patches have been received, events will be raised for this, and they provide a quick and easy way of downloading the patch and then your series for those kind of events. This is pretty crucial when it comes to CI because it gives you an automated way of figuring out what it is that is available for testing. So how do you actually go about using all this functionality? I put together a little bit of a demo. I'm not going to go into all the detail on it here. I have a blog post which I should be publishing this evening on my website where you can find the actual specific commands you could run. How would you go about tying patchwork into, in this case, Jenkins? So it's not that difficult. So start enough anyway, patchwork receives patches from a main list and it fires events. So remember now events is still in development so that any of the examples I show might change in the next week or two. But it'll fire a patch complete or a patch dependencies met event. Which you have some little script running somewhere. It's able to pull in those patches and it's able to build your job request which is going to submit to Jenkins. Jenkins takes that job request, an important step here. It submits a check back to patchwork say in the pending state saying that I'm about to kick off a test. The test is in the pending state which provides users with a visibility about whether their CI system is running and then what kind of test are actually expected to run against that patch. Jenkins in this case pulls in the patch, runs all of its tests and gets its end result whether that's a success or a failure or a warning. And then it pushes that result straight back up to patchwork. That information then is visible for users. You provide a URL where they can go and get more information. So if you have build artifacts, provide a URL where users can find all those build artifacts, give a brief description and of course the actual test status. So I'm not more aware of there's a couple of tools that are already using variants of that API. Snowpatch is one that I'm aware of. The demo that I built used plain old bash scripts because they were nice and hacky and I was able to do it in a couple of hours. But hopefully we'd build upon that in time. How we're actually planning on building upon that, the first thing is to merge all of the patches that are currently sitting in the main list. So finish events, finish the ability to download series inboxes and document everything to the eyeballs if we can. In addition, there isn't really any reason in that workflow to keep a script around. It's possible for Jenkins to actually do this itself by way of a Jenkins plugin or something similar. And there's also no reason to actually use Jenkins. I use that because it was the lowest common denominator. But coming from OpenStack, Zool is pretty much like the bee's knees of testing automation tools. If I can find a way of hooking Zool into patchwork, then that's something that I definitely like to do to bring in all the different advantages that Zool offers. So that's a very brief overview of the kind of stuff that's coming down the pipeline for patchwork 2.0. The first place you're probably going to find this would be the AusLab's instance, which is used for OpenV switch and a good few other Linux kernel projects. And I'm hoping to get the data plane development kit guys on board with this as well and get automated testing going there. More information available on the website and the Git repo. And of course, on the main list, it's probably the best place to find all of this. So I've been Stephen Finucan. I'm happy to take any questions that anyone might have. Thanks for listening. Thank you, Stephen. So we have two minutes. If there's any questions, don't be shy. First of all, thanks for the talk. This is something I was thinking about for the last couple of weeks because I was going to write some sort of integration between Jenkins and patchwork. And this helps to make this idea about this more clear because I was reading the documentation for patchwork and some things were missing. And you basically described those things. I think I will contribute some patches for the documentation so that this bit is covered. The one question, are you planning to write a Jenkins plugin for this? Or is it something that's? So I started a couple of weeks back. I spent about two days trying to install NetBeans, which I hadn't used since college and unfortunately, isn't packaged on Fedora. And that pretty much took all the energy that I had to go and write Java code again. So at some point in the next few months, a Jenkins plugin would be my ideal target. If I can't get a Jenkins plugin go, I might try BuildBus or something because that's Python based. One question, are these REST API, does it set any relevant HTTP headers so that ETAG or modified time, something like this, last modified so that a script? No. No, not yet. So there's two big optimizations needed for the REST API. The first one is ETAGs. So you don't need to make requests to the server every time. The second one is go back to the actual REST API body. You'll see, for example, there's a patch in a series there. I want the ability to expand that patch in series inline. So again, I don't need to make two additional requests to the server. But I need to get 2.0 out. After 2.0 is out, then maybe that's an optimization for 2.1. I'm afraid the time is up. You have to continue talking outside. Thanks a lot, Stephen. Good.