 Hello, now Remy will present us Lava Federated Testing. Hello everyone, thanks for coming. So as I said, I will today present Lava Federated Testing. So the first thing to explain is what Lava is because I guess you don't know what it is. So Lava stands for Linear Automated Validation Architecture. It's a CI system, so the goal of it is to test your software on specific hardware, like Raspberry Pi, Juno, Panda Board, things like that. So you take, for instance, you compile Linux and you want to test it, or your software that you want to test on real hardware. So Lava will help you to deploy the software on the hardware, boot the hardware, and then run the test. It's more or less becoming the de facto standard for CI system on real hardware. So just to explain a bit more what it will do for you, if you want to test on a Raspberry Pi, but without Lava, what you will usually do is to, you first power on the board, so using the power controller, then you connect to the serial. If you have a serial relay, you will just do telnet something. And then you will see your boot starting with, and you will wait for it any care to stop at the boot. You just type enter, and then you type comments, like the HTTP, set server IP. You load kernel DTB from TFTP into the RAM. And then you set the boot target and your boot. So you will look roughly for the kernel output to see if everything is going correctly. And you go to prompt, you log in, and then you run the test, and you power off the board. That's all the thing that you have to do manually if you want to run tests on a physical hardware. If you want to use Lava to do that, then instead of typing everything manually, you describe what you want to do in a job definition. I will explain a bit more later on what you put inside it. You put the information, like the kernel, the TB, rootFS, you want to test inside the job definition. You give that to Lava Dispatcher. So Lava Dispatcher is the part of Lava, which is responsible for booting, and deploying booting the boards directly, really connecting to the board and handling the board things. You give the job configuration to Lava, and it will just do everything as I explained before, but automatically. You can grab a coffee and do something else, and go back and just look at the results. Lava will do everything for you. Obviously, you can do that at a large scale, not only with one board on Dispatcher, but five, ten boards on one Dispatcher, having many, many Dispatchers all connected to the same server. So you can have 100 boards in the same system. All the users will talk directly to the server, to Lava server, and they will submit jobs, get results, and see what's happening. So with that, you can have many, many jobs running in parallel. You don't have to care about it. Everything will be done automatically for you. So for instance, you can want to test your Linux kernel on Panda board, Bigelbone, Raspberry Pi, or at the same time. You just submit three different jobs, and Lava will do that for you. As I said, for a user, on a user perspective, when you use Lava, you write down your job definition, and then you submit it. Usually just with the common line, you use Lava CLI, job submit, job definition, and that's done. You don't have to do anything else. You can do some more interesting work. In the job definition itself, what you will put usually, just some more important things that you can use. There's more things to describe justly, but it's too large for the screen. So usually you will describe the device type you want to address. So you will not say, I want this specific device which is in the lab, not Raspberry Pi of 1 or 05. I want a Raspberry Pi 3B, and Lava will pick up the board which is matching this device type, and available at the same time. You will point to the kernel DTB root FS you want to deploy. It's a URL, so it has to be available somewhere, and Lava will fetch it for you. Some interesting features that are not mandatory, but you can use it, is autologing, for instance. So if you have to type username, password, Lava will do it for you. You just have to provide the username and password, obviously. And then the tests, just you want to run. Usually it's inside a git repository, so you have versions of your tests that Lava will record the version along with the results. And we already provide, as I've addressed, many wrappers around well-known test suite like LTP, KSF test, Android test suite, and things like that. For the admin perspective, the server will do all the web UI. So you will submit jobs manually, or you'll use through the API. You will see the results, the logs that have been passed. It will do the access control for you. So who is able to access a board? Who is not? Some boards might be private. If there is NDAs and it's a public server, you might want to hide some boards from public access. We'll do the scaling with priorities and things like that. We can even do multi-node jobs, which is one job definition, sorry, that is using more than one devices. Once you want to test a streaming program like VLC, you will have two boards. One will be the server and one will be the client, and it will be all in the same job definition. On the dispatcher side, the dispatcher, which is connected directly to a board, will regularly check that the board is having a good health. So it will run some specific jobs to see if the board is working correctly or not. If it's not the case, the board will be put in maintenance, and admin will be notified that they have to look at it. When the board is booting, we will pass automatically the kernel output to see if there is a kernel panic. In this case, we will report it and stop the jobs or boot the errors, and we'll try to classify why there was an error. If it's an infrastructure error, like in this case, it's a TFTP, which is not responded correctly. This is not the test which has an issue. It's an infra error directly. So it has to be reported to the users. Same if it's a bug, we will try to report it. So a user can just say, it's not my problem. Slava issue, not mine. We do support a lot of different methods for deploying, booting, and testing. I will not explain everything. And we do support a lot of different boards. This is the actual list of supported boards from before Friday, because Friday we did one more, something is still here. So that's a lot of different boards. And as Slava is a testing software, we have to prove that it's working. So we have to test it. It's a bit different, strange if we provided a test software which is not tested correctly. We have to do it, to do that correctly. So ever we are really lazy and rich, and we just buy 142 different boards and build a really big lab, or we are not, and we have to be smart. Obviously, if I'm here, that's because I prefer to be smart and to do some fun stuff. That's why we created LavaFed. So LavaFed for Lava Federated Testing. So the goal of LavaFed is to prove to people for the community that the next version of Lava will still work the same way it was working before for them. So the goal is to take the docker build that we do for every commit. Every time, sometimes, push a commit on Lava, I will do two docker containers, one for Lava Server and one for Lava Dispatcher. And I want to prove that these containers are working correctly. I want to prove that it's working correctly on the board that the community cares about. I'm already testing on the board that my employer is caring about. I want, for the same reason, the use case you care about because my use cases are already tested. And I want LavaFed to be almost only LavaJobs. It's better. I'm developing Lava. It's a testing software. I should be able to test myself with Lava and with some glue around it which is building notifications and APIs. Which means that I will also test notifications and APIs at the same time. So two features that you have to understand before I explain really how LavaFed is working. The first thing is that we have a lot of different device types in Lava and we have one specific device type which is docker. We can just create a job definition in which we start a docker container on the dispatcher itself. It allows to do some testing quite easily. So instead of connecting to a Raspberry Pi serial connection, we'll just run docker run and take the output of docker run and command in the shell that we get. The second thing that we use a lot is notification. You can ask Lava to give to post to a specific URL so it's a callback when a job is finished or canceled so we can get results, logs, everything quite easily. So let's finally speak about LavaFed. So we use GitLab for our own for our CI. So every morning, around four in the morning if I remember correctly, GitLab will start a container. It will keep the last version of LavaServer container which is available. So this is from the beginning of this week. You will see the full version here. It will start it. This will be the LavaServer and it will be the LavaFed server which is available from everywhere. In blue you will see a community lab somewhere in the community is okay to share a lab with us and the server is available from everywhere and the dispatcher is in their private lab so it's not accessible from outside and they do have a Raspberry Pi connected to it. So the first thing LavaFed is doing is asking for the version of LavaFed server so we are testing the same containers, same versions and it will submit LavaJob. As I said, we can submit to a specific device type which is a Docker container. So we are submitting to a LavaServer with submitting a Docker job which is in fact a LavaDispatcher. So in their dispatcher they will run my own dispatcher in the Docker container and this dispatcher is configured to connect to my server. So I'm kind of hacking a bit their own dispatcher so I'm sitting in their network, in their own dispatcher connecting back to my server. And then I'm asking for the configuration I'm asking the lab one server can I have your configuration for the Raspberry Pi and I'm copying everything on my own server so I know how to configure to use their boards with my dispatcher connected in their network and I ask the lab server to say now the board is offline for you you should not use it and my LavaFed server will now see the board available and will use it. So I submit jobs on my server and it will run using my dispatcher on their board and I will get, when it's finished I will get results in my own servers so I will be able to do some graphics and obviously when it's finished I will do the reverse and give them back their own board and cancel my jobs. So what's next? Currently we start a small we have only three labs and five devices which is not much and only running five jobs a day it's not much but we are adding new device types as you can see when I am adding a new device type it's just failing usually so I'm fixing it and then adding another device type a bit later on and we want to have new jobs all function tests to run inside LavaFed so obviously the next step it's LavaFed already and the other goal is to share the board with the community to test together so I need people that are already using Lava or want to use Lava to just come to talk to us and try to work together to be able to test the board you care about and the job you care about so please send me a mail if you are interested or just go to the website to see what we are doing with LavaFed Thanks