 OK, we're going to get started with the next section. This is the Cloud Containers and DevOps track. Our next speaker is Hector Martinez. He's a senior engineer at Collabora and also a Debian developer. And he's going to be talking to us today about orchestrating continuous integration through containers. So without further ado, Hector. All right. Hello, everyone. Thanks for coming. Hope you enjoy the talk. If you have any talk, feel free to interrupt me. And we can discuss whatever questions you have. I also need to apologize, because I was sick this morning, and I couldn't prepare properly all I wanted to do. But let's see what we can get to. So as the presenter said, I'm Hector, Oron Martinez. I'm from Spain. And I contribute to the Debian project. It's a community project run by volunteers. So if you have free time, feel free to join. We're also having a Debian conference in Taiwan this summer. So if you are nearby, feel free to register. The registration is open, and then you can come to DebConf in Taiwan and leave the Debian developer experience. I'm also working for a paid job at Colabora, which is a consultancy company on open source. We do graphics. We are upstream for Wayland, which is a graphic stack to replace Xorg and Gstreamer, WebKit. We also build Debian derivatives. And I'm going to present here today a pipeline that you can easily start and you can use it to build your own projects, your applications, derivatives, or whatever projects, or whatever is suitable for you. So we'll discuss what's continuous integration. Everyone familiar with this term? And what are the containers? So you all know what containers is, and how can we build a pipeline with a lot of containers and get something out of that? So that'll be the topic for this talk. So continuous integration, it's having one tree that a lot of developers committing to this tree, wanting to merge into a shared mainline tree, all the source. And then, suddenly, one developer adds a feature but breaks another one. And people just projects run into chaos because the mainline tree is broken because too many things at the same time. There is not enough time to test and validate all the changes. So continuous integration tries to fix these kind of problems. So what are the best practices for continuous integration? Is you maintain a code repository and automate a build of the project, of your application. And then you need to write some self-testing, some tests that you can run after your code is built to validate it so everyone can commit to it every day. And then every commit that goes into a mainline needs to be built and validated, and it needs to be fast. So then, if it's possible, you want to test this on a production environment. Like you have a new application that runs on some specific hardware, and then you want to deploy this application in this specific hardware and test if it's working or not and see if there are any regressions. And then it needs to be, to developers, needs to be available or deliverable. So all the native builds, so people can have the latest and test work with the latest. And everyone should be able to have access to this latest build and then automate the deployment of the application to the hardware. So it sounds like too many things to consider, but maybe thanks to containers and technologies available, we can do this really, really fast. Easier than that we might expect. And we'll see later what's the suggested pipeline. Let's talk first about what are containers? Everyone familiar with the term, probably? So it's a system-level virtualization which depends on namespaces and c-groups. So virtualization, KVM, it's on kernel, and you need an iPadvisor. And then it takes some seconds to minutes to get a machine up. But with this technology, with some milliseconds, you just run a process and have a container process space. And you can run many more containers in a system than KVMs or virtual machines. So for some cases, it's very, very useful. So here is a suggested continuous integration pipeline, the components or the blocks that you can put together. So first, you need a code review system. So everyone developing the code can commit into this code. And then you need an application build system. And after you build the application, it's nice to package this as an RPM or the Debian package or whatever OS you use. So it's easy to integrate this application into your OS. And that gets into repository with many other packages. And with all these packages, you can build an image for your target device. And then with this image, you can automate the test system. You can run unit tests, a system test, a smoke test. There are many different kind of tests that you can run. And also, I think it's very important. But I added an item about license checks, because many people is using, is building, is working on free open source. But they are building on top of a proprietary or closed source components. And they don't notice. They think they are. Because they have, they depend on a library. I wanted to use this library, which is very good. But then this library depends on something else, which is proprietary. So at the end, your code is depending on something which is closed source. And you think you have a free open source application, but it's really not, because there's a dependency up in the chain that is proprietary. So if you check for that, that would be nice. For example, a GitHub recently opened license set. So you can check for Go packages, and I think Ruby, and some others, it just checks the chain of the dependencies. So you know what your license is if they are all clear. So you can build this into a product, and you are fine to ship it or distribute it. So here are some suggested technologies that we can use to implement these features I was talking about. So for code review, there are many different technologies like Garret, Fabricator, GitLab. For the application build system, you can use Jenkins. You can use Buildbot. For package build system, to integrate with ANOS, you can use OpenBuildService, called OBS. We did a presentation yesterday, so if you were not there, you might refer to it in a video or the slides for the conference. We had also workshop, so we explained a little bit how to set up OBS, so you can build a package against many different distributions with only one source. And then you can build your image. So Jenkins probably is very useful for this, because everyone builds images creating their own scripts, and it's quite a mess. There is no one single tool. So we created yet another tool to build images with what we call DevOps that you can find in the Deviant repositories. And this way, you describe what your image, you like to be like with a recipe, we call it with a Jamel description. And then with this Jamel description, you are able to build this image. So if you're thinking on building an image, and you don't yet have craft all your scripts together, maybe it's worth that you have a look to this tool. And then for test system, you can run Jenkins as well, or you write some test cases and have a virtual machine or a container, and you can test. Or if you want to deploy on real hardware, there's a linear automated validation tool, which is called Lava. I don't know if you're familiar with Lava, but it's quite interesting as well. And then for the license check, Linux Foundation has a maintenance tool. It's called Phosology, which is quite nice as well. It keeps a database and keeps all the license that you have. So let's talk a little bit about the architecture. So what is Jenkins? I don't know if you're familiar with all these technologies. So once I put in bold letters, I'm going to comment a little bit. So Jenkins is a master server. And for me, it's like a cron job. So it's cron job on Asteroids. You can configure and do many tasks. And then that's why the logo is like a serving master. And then you can have nodes, which are the slaves in many different OS, like Windows, or Apple, or Linux. So you can run your application and deploy on many different operating systems, depending on the slaves you want to support. So the Open Build Service, it's also a command line tool, an API that you can drive with this command line tool. And there's a backend server, which just creates the environment to build packages, and build a package, and creates repositories, and publish these packages into repositories. So it makes it very easy to manage Linux distributions. Phosology. Phosology, you upload your upstream, your target with sources. And then you can run agents, which scan these star balls and gives you the, and then checks for the licenses of the software. And then you can check these licenses afterwards. And it can provide you with a report on SPDX format, or Depth 5. It's a Debian format as well. And then you can validate these reports and see that all the licenses are correct. And then Lava is for testing on real hardware. It's a scheduler, it's also like a Jenkins thing, that you can send to dispatchers, that you can have in many different places. Like, for example, we are contributing to Kernel CI. You don't know if you heard about kernelci.org, but many different companies have labs on their offices or people at their homes, and they are connected to a single master, which is kernelci.org. And they hook all these little ARM devices or hardware that you care about. So because ARM development boards is quite a big spectrum that we have, Raspberry Pi's, or Sabre Lite's, Exynos boards, Dragon Balls. I mean, there are a lot of boards. So this project, Kernel CI, wants to smoke test, boot a kernel on every target, on every change that happens in the kernel, so you verify and validate that the kernel boot will boot on your next board. So this is a community project as well, and it's quite useful for the kernel validation. But you can run it for your own purposes as well and try to deploy this on your device. So all these technologies sound very complex and many different things to do. But thanks to containers, it's very easy to deliver this. So Docker Hub has Jenkins. I recommend you need to define a Jenkins job. And I recommend to look up the pipelines in Jenkins. You look up the documentation, and it's if you're going to do a complex job for it. So you just define a job, and then you can have a Jenkins file where you can describe your pipeline, and then you can have different steps, like building the image, or you can check the image. Anyway, you can have a look to that. I won't talk about this in this talk. And then Open Build Service, we collaborate. We have the scripts, the Docker file to create the Docker files. I plan to upload to Docker, but I didn't have access yet to the collaborative space. But it should be a Docker OBS in Docker Hub. And for Sology, also, they have Docker image. You can run. And also Lava, and probably GitLab as well. So I wanted to do a demo. I don't know if it's going to work to see how we can set up all these pipelines in the minutes we have left here. So I want to explain here that you have Jenkins, the pipeline, basically, you keep your sources in GitLab, or you get packages from Debian. And you can combine these two. And get through Jenkins can pull these sources and packages and feed those into the OS build machine, which is Open Build Service. And then you can build an image with Jenkins and deploy on Lava and test that. And then you can test your license of the repositories with a for Sology tool. So let's try to do a minute. I think I have five minutes or so. So let's try to see if I'm able to get this started at least. So here I show there is no Docker containers running. And then I was stopped in OK. And then I have a Composer file. This is the Docker OBS thing. And then this is going to start the OBS server and the OBS API and the Worker. And then you get the whole system starting. It's yesterday we had a talk and explained how to set up OBS from packages and the configuration. And it took quite a while. But with Docker, you just start it and it should be running. Let me see because this setup is a bit here. So I don't know if it says the IP address somewhere here. You can see 17223023. So we go to the web browser and one. See, I don't get this here. So 1722.2303. And then let's try on the certificate. Of course, we didn't set up the proper certificate. And let's accept this because it's running on local host. OK. There's an error because probably I loaded a volume that I had because it's a demo. It doesn't work. But if you check out the three, build the Docker images. And at least you get the system started and the application is running. So that's one of the things. And then because now this is running on this port, it's taking on the other Docker containers. So all this part, I wasn't able to prepare. So it worked all together. But essentially, you can start your lava container. As well. So here is lava in running on local host. And then you have here I didn't start the lava. But here is the lava Docker. But if I say I have these containers running now, which is the OBS. And then if I run lava, it cannot connect because the port is already taken by OBS. So you need to align everything. This is the part I was not able to do. But I mean, it's OK. I think you get the idea, right? So I could go and stop the Docker, the OBS. And then once it stops, then probably we can run the lava container. It's all in these websites, well-documented. So it's very easy to get it started. So here it is started now. So you get lava running on this port, right? And then the hard bit now is that you need to understand how to configure this software, how to write the test cases. You might find examples online. Let's see, this is a scheduler. You can connect all your devices, all your jobs. Then you might want to have seen it. It's already defined some jobs here with a QMU to boot a kernel on QMU and test it. And all the devices. So you have a lava slave, which is a virtual machine based on QMU. And then I go, OK. Here it's going to start Jenkins container, but it's also the ports are taken. OK, we're running out of time. But you can run a phosology. Let me show you a list and make file here. So you have a here. This is to build. You can build the OBS images. But here to run the Jenkins, there's one command line. And you set up the ports accordingly. And it should just work. OBS and then phosology. Also, there's one command line. And you get phosology. And then you can start playing on how to configure this and how to make your configuration suitable for your needs. So let's go back to the presentation. So I don't know if you have any questions, but thank you for attending. And I hope you enjoy this minutes. Thank you for taking your time and listening to this presentation. So thanks. You have quick questions. Maybe you can ask me around. I'll be around. Or if we have some time, maybe you can talk. OK, so if there's no time, so feel free to.