 Hello everybody. Our next speaker is Anton Marcukov. Marcukov. Sorry, Marcukov, with yet another reprimand. Please give him a warm welcome. Hello everybody, welcome to talk about reprimand. So, important thing to understand, the reprimand is like a superman, but that works with packages. So, a few words about me. So, I'm a software engineer working on Red Hat, and I'm a member of Overt community info team. So, Overt is an open source virtualization platform based on the QVM hypervisor. And we are info team, so we are responsible for all the infrastructures that this project uses, for builds, releases, and we're also doing continuous integration and recently we're also doing continuous delivery. And this tool was developed by us to make this possible. So, what is the challenge, why we did it? If you Google, you find some other reprimand, not necessarily dealing with packages. But the thing is we have the problem with that Overt is quite large application. It consists of multiple parts, there are packages in RPMs, and we really have to deal with a lot of RPMs and also ISOs by the way. So, at some times we process around thousands of RPM a day and we somehow need to make usable composites out of this so users can install them, testers can test them, and all the stuff. And obviously we are not having infinite resources and we need to do it efficiently. So, that's why this tool was created. And as you see it has nothing evil inside. So, the reason is that we made this tool for CI. So, you supposed to run it as a standalone application on command lane or you use it from Jenkins for example. And if you do it like that, that's why it's important that this tool doesn't use database. There is no database. Also, there is no cloud inside, it doesn't have any rest interfaces, it doesn't open network sockets that you supposed to deal with or interface with. And also there are no demons inside, so it's not evil and also no demons. So, it's not some demons and runs in the background and you need to manage it. So, it's completely have no state inside. So, if you run it, it's always run from sketch because if you have Jenkins we already have a way to run jobs, define jobs. Jenkins stores their configuration and you can store their artifacts. That's why this tool doesn't have anything inside. And, yeah, you pretty much guessed it. It's sweet old console application. So, once you install, you just use it like this from command line or obviously Jenkins is able to run command line application so you can run it from Jenkins. It's distributed currently as RPMs. It's Python application but the problem is that it uses some command line application like create repo. It's from yam and RPM sign so they're not available as Python models so we can't just satisfy it from pi, pi dependencies. So, that's why it's in RPM but if you have everything installed then you can access it as a Python model because it's actual Python model. Okay, so what it does? So, the idea behind the tool is that you have to define some sources then you can define some filters and then it store in the output store. It's made pretty abstract in code. It uses classes, of course you can override it. So, currently as input sources it supports the disk, the local disk file system and that's primarily how we use it. It tells Able to get packages from the internet just by using HTML pages with HTTP links. Anything that has links to artifacts like RPMs or ISOs on the web page will do it. There is also a way to recursively crawl the pages and look for artifacts but it's configuration settings. It tells Able to understand some particular HTTP applications like Jenkins. If you pass a link to Jenkins artifact page, it understands that it's Jenkins and will just download artifacts. It will not be parsing full HTML of Jenkins. And we also support Fedora build system such as Koji. That's primary Fedora build system and also we support copper links. So, if you know what is copper, it's like Fedora shared party repository then it will understand it. And for Koji, there is particular support. If you have Koji tech, you can pass the Koji tech and it will download stuff from Koji techs. Then for the filters, so basically to the inputs you apply the filters, to filter the RPM, I think, mostly we use this, only mission one. It allows you to, for example, take two input sources and use one as a primary and from second just feel what is mission because it understands the versions, it understands the names of RPM. So, it's able just to take what is mission but you can also leave for latest 10 last versions or you can filter by regress. And then it just stores it on a disk. So, for store, it's currently only support the disk file system and it's able to split it. I will show you a bit later on RPM synisers. Again, so it's pretty abstract. If you have other types of artifacts, you can define some classes inside and probably send us a patch because it really was meant to be a general artifact tool. And this is a real use case. This is how we use it for a weird experimental repo. So, a weird experimental repo is a weird repository where you can install a weird from it but the thing is that it's basically updated on commits. So, here at the left you have some jobs from our Jenkins. It's, yeah, names are not important but the thing is that there are a lot of them because the weird consists of these sub-projects and each of them is maintained by their own teams so they all build artifacts. But we as a CI team, we need to make one a total repository of all of these. So, essentially when something builds it from code commits we need to take it in place into this large repository. And this is when we use the repo man. And as you see, here's a structure because the repo man understands the distributions and understands the versions and artifacts so you see that it's created sub-directory RPM, sub-directory ISO depending on what content you give it and then it's also created sub-directory for the distrust it saw in packages. So, it happens automatic by repo man. That's why it's useful. And then from here the latest, it's basically latest continuous delivery to Overt that has just top-latest based on commits. And as I said about efficiency and this is one particular thing that repo man does is that hard links are essentially included. It will create a hard link to the file if it's not changed. That's why we can really produce hundreds of Overt repositories daily because if we will be copying all the files we won't run out of space pretty soon. But here, for example, we have the previous repository with Overt and you see that basically two new packages version were built. Like for artifact A we have version two and same for B. And then when we use this only mission filter and ask repo man to take this and just add this new stuff it will basically hard link all the artifacts that are the same so they won't really occupy any extra space. And since it's hard links then basically if you delete this this will stay because it's like separate file names but no extra disk space is occupied. So that's very useful for us just because those repos take a lot of space and we only mutate part of them. But it also produces problem if you will store it on different file system. It can't really see link right now. And that's pretty much it so just to describe where you can find additional information so this is my Twitter feel free to just ask questions I will reply them on devant and this is our info team list so if you have any questions regarding Overt how we do CI whatever or repo man just use this mail and currently the primary repository is our get it this is because we really do also continuous integration for repo man we test it on different fedoras and OSs and we build RPMs as we use our infrastructure but there is github mirror it's basically mirrors from get it so feel free to use it we as a python project support they read the docs and it's pretty standard python application and if you want current RPMs they are now Overt RPM repository that's basically it so if you have any questions yeah just copyright slide because I took pictures from the internet so I will be also available on Overt booth we have Overt booth here so as soon as I'm not on the second talk then feel free to come Thank you very much Anton if you have questions I come around with the mic thanks so if there is no demon no it's not demon yeah if there is no demon question is how frequently you run repo man to create repo so we run it I think several hundred it is Jenkins job so it triggered by commit to the code we have like maybe 50 repositories where people make commit and then each of them build the artifacts yeah that's all this Jenkins job and so it happens several hundred times of day and then for each of them the repo man runs so it's hundred of runs of it per day and it's yeah it doesn't really need a lot to load it just reads RPMs, parses the metadata and then composes the final repository and when possible it creates the hard links does it also reads the RPM contents or only the final? It reads only headers so we use RPM library to read only beginning of RPM where the metadata is okay thanks yeah Hello my question is what's the reason of using Jenkins for producing RPMs I mean I know that Red Hat has a lot of different tools for doing that but this is not really Red Hat product we have REF that is product by Red Hat based on but over this community project and community infrastructure uses Jenkins yeah we are reviewing other tools but Jenkins works pretty good for us and it really has anything we need right now so we are at Jenkins shop Any more questions? Well then thanks a lot give a warm welcome, applause to Anton Thank you