 Hello everyone, welcome and thanks for joining me today. My name is Houloud Twill and I'm going to present for you Let's test with Kernel CI. To start, I'm going to introduce myself. My name is Houloud, I'm a software engineer. I'm based in the South of France and I am a Kernel CI contributor. Well, I work for a company named Pay Libre. We are an embedded Linux consultancy company and we do a lot of open source projects. We are a top 20 Linux kernel contributor. We keep contributing to Uboot, Yocto this year and we do a lot of automated testing. Okay, so to start, let's talk a little bit about the Kernel CI project. Well, it's a free service for testing Linux on a huge variety of hardware platforms. It's an open source project and a collaboration hub for hardware focused open source testing and automation. Well, the purpose of this project is to gather all together to test the Linux kernel to get a better quality of the software. These are the founding members of the Kernel CI project. Okay, to begin in here we have it could be a kernel developer or a kernel maintainer or just a simple user with several thoughts in the mind. It could be I don't have the same test platform as Kernel CI is even possible to join the project. I really want to use Kernel CI but I don't know how they taste. Also, I use the same platform as you can join your project and different other thoughts. So to start, I will do a quick recap about the Kernel CI module pipeline. In here we have the source from we are going to fetch the code. So we have a new patch to come and we want to test this patch. So from there we are going to start a build with the configs that you want to do. And then you will pass to the run section where we do the testing part and creating the test plan definitions. After finishing the tests we will submit the results to be stored in the database and then analyzed. And finally we will have to report the results either via emails or dashboard. So this is from a talk I made it back in 2020. You'll see with my colleague Kevin Hellman in the link you can find this presentation along with the video. So today the talk will be focusing on the run section, how we do the testing, how we do submitting the tests, and how we see the results. So let's test with Kernel CI. Okay so let's test with Kernel CI. We'll have three major points. The first one will be how to prepare the test suite or case including creating the test plan or customizing the root of s image in the run or even adding a new root of s image to be used. After finishing with creating our test suite or case we'll be running the test being created and then submitting test results. Well for submitting test results we have two different types. First one for lava users and this second one for lava users. So it's never too late to join the project and you are welcome every time. Okay so to start by preparing the test suite or case well Kernel CI provides for you pre-built root of s images you can check the list by using the case root of s tool exactly doing case root of s list variants and as you can see in here the different root of s images that we to provide. So the default root of s image based on the build buster and we do provide several support several architectures including for example RM64 MIPS RISC 5. So to create these root of s images we are using debuess which will use the bootstrap for creating these images. In here you can find the debuess project and this is the link for this project. Okay so before to get actually started you have firstly to clone the Kernel CI core repo so this is the link for it. Well when you are going to clone this repo this is what are you going to have. Well in this talk we'll be modifying these files and directories. We have here this test configs.yaml file root of s configs.yaml file labs configs.yaml file and the two directories templates and Jenkins. As here there is different tools but in this talk we will be using the case root of s tool and the case 8 s tool. So in the previous slide we did use the case root of s list variants to list the existing root of s images. Okay now we are starting so I don't want to get things complicated for you in the beginning so we'll be starting to test a simple test case which is echo hello world. First step you have to create the relevant directory and the re-templates directory. In here we are doing a hello world so it will be named as hello world. Then the next step which is the second will be to create the test plan definition. We have two ginger vise this is the first one you have to create your test plan definition in this format and as you can see we have here the name the description this is in the lava style format that can be used in the test. In here you can see this is lava this case which is again lava this in format this is the name of your test case hello world and in here you are you are running your test case echo hello world. The second file it will be it will include the first file which is hello world ginger 2 and it will extend another from another file which is under put generic uboot tpt ramdisk boot template ginger 2 file. To know more about this you can go to templates and see the different templates that we have in our project. The third step would be to modify the test config xaml file we go to the test plan section and we add our new test plan which name it hello world and as we are using a pre-built root of s image in here you insert the one you want to use root of s this is the beyond buster ramdisk one of the basics root of s images. Then the fourth step which is going to update the test config section in the same file test config xaml to add to use the test plan you added earlier so assuming that you are going to use a raspberry pi 4 as a device to run the software and the test plan to be run would be hello world. So this is the first point which was adding a simple test plan definition and this would be the second point which is customize the root of s image in the run. Okay so more complicated test than hello world you want to keep using a pre-built root of s image but you need it to be customized you can do that in a simple easy way. First you have to create a test script for me I use bash but you can use python perl any script language you know and in here as you see we are customizing our root of s in the test script so when this will be launched in the run section you will start by installing these three packages wget succs lip succs format mp3 so this test would be again hello world but the hello world won't be in the shell but in an audio file in here you are going to create your test directory you are going to download the audio file that says hello world then you are going to play it and this is the test case in the lava test format. So we're using a test script instead of going directly to gingery2 file we have to pass through a yaml file that have the run steps and that will call this file which is the test script the hello world audio.sh after creating that we will be creating again our gingery2 file test plan definitions but this time it will be getting the yaml file from this path templates hello world audio hello world audio.chaml and this would be from the repo on github with this exact brush. Again you will update the test plan section in test config yaml file this time we are going to name its hello world audio and we are going to use the same root of s deviant Buster RAM disk. Again we will modify the test config section we are using the same board raspberry pi 4 and in here we are updating with the test plans we are going to run which is hello world audio. Okay so till now we've seen two points the first one is to add simple test plan definition and the second one is to customize your root of s image in the run section and doing it in your test script before really testing the things you want to now we pass through adding a new root of s image well we have several tests to do more sophisticated tests there is a need for a new root of s image so let's do it okay so I will be keep using the same example of hello world audio but this time instead of customizing the root of s image in the run section we will do it before and we will create a new root of s image that already have these packages installed so the first step would be to modify the root of s config YAML to add the root of s definition in the root of s config section so here we are going to be to name it WM Buster hello world audio you have to precise the root of s type which is WS WM release Buster the architecture list you will be using example the R64 and in here we will put insert the packages the needs packages for the root of s images image which are wget stocks ellipsox format mp3 then we move to the second step which is to build your root of s image so in here we are going to use the kernel CI to cast CI root of s build using the table s project for building the root of s image so these are the tools I mentioned and the files I mentioned beginning of the talk when I just cloned my repo kernel CI call okay so the third step would be to modify the test config YAML and this time it's going to be the file system section here we are adding our new root of s image this is the name to be ambassador hello world audio the type it's Debian and the RAM disk and as for the fourth step it's going to be the same you add your test plan in the test plan section in test config YAML but this time we have the same name of the test plan hello world audio but we are changing the root of s to be used so this is in here we are inserting the the root of s that we had created previously which is Debian buster hello world audio and that's it for this point so until now we have seen three points the first one needs to add a simple test plan the second one is to customize your root of s image in the RAM section and the third point which is to create a whole new root of s image which is already customized we move now to add a new test to root of s image okay things getting complicated uh tests are more complicated than just a hello world let's build our root of s with our test okay so i'm sorry this time i'm run off i'm run out of simple examples so i'm gonna need to use one of the kernel ci uh existing tests so in here uh i'm presenting the IG test which is the which tests the kernel uh graphical so uh we start as always by adding our new root of s in root of s configs the model file the root of s type the u s Debian release buster in here we have the architecture list the extra packages you want to add to your root of s image also you may want to remove some packages like this in here or even remove some files voila and then uh you want to add a config script it would be script buster IGT.sh for us and we mention it in here so what what this script does exactly is to add non-debian packages tools to to build the test part in the script and the script will be exactly run in the root of s creation time also you need to add a test overlays in here you have overlays IGT so under trinkets the bnw s overlays IGT user been you will adding your IGT parser.sh so this would be uh the script to run the test and then to parse the test results in lava style okay so again after uh adding our uh test root of s image we are going to create the test plan to finish in we are using the same as uh previously this is the gg2 file and in here as you can see you will launch the IGT parser.sh which had been created already in the root of s image okay let's move now so now uh let's uh run the test well not yet uh let's generate the test first okay so uh to generate tests you have to to there is lava requirements and setups should be done firstly so uh you have to set up your lab either physically or using uh kmu vmware and also you should create a user uh and a lab token then you will need to add uh your lab in the lab configs yaml file which have the lists of uh labs that we uh we have so in here i'm showing you the example of lab day labor uh lab lab type which is lava in here this is the URL of the lab and as you can see in here the planned test plans that we are going to be uh run in this particular particular lab so after this you will need to um contact the kernel cim team that would generate for you a lab token that permits you to submit your test our results after the test is done okay so uh this is gasi test tool uh i mentioned previously with uh this tool we are going to generate our job it will be used this way gasi i test generate with these different arguments in here we have the storage from where you are going to to um to get your build artifacts these are the pmeter jason um and the dds jason files which are uh two files uh from the build uh artifacts in here you have the lab config the lab name the test plan that you are going to be uh to run in here we have the uh output where you are going to generate the job yaml file in here you have to mention uh your lab user and your laptop and finally we are running the test well this time for sure we are running and using the cassier test submit it's so simple you have just to mention uh the lab and the lab user the lab token and in here you have to to to answer the lab the the job yaml file okay so this was the part where we are uh running generating and running uh tests and now we'll move to how we do submit test results well for lava users it's going to be uh done automatically uh with uh lava callback directly to the kernel ci backend and then we'll be meeting in kernel ci dot or which is uh kernel ci uh front end so this is our dashboard in here we have uh the available test results this is from the different trees the branches uh in here we have test plans uh different test plans been uh run in uh our different labs for kernel ci and in here i'm showing you an exact uh results for baseline tests and this is all the test cases uh in the baseline test suite well if you are too lazy to come to us never uh it's okay it's okay we'll meet you in your email box we do also provide the service of email reporting as you can see here it's the same example of uh this results but this is uh an email that you will uh get in your email box still want to join kernel ci and you are not a lava user well we do provide for you a kci db tool let's know what's that exactly well kci db tool it's a tool for submitting and querying linux kernel ci reports and this is the bigger picture well uh i use this from giam uh tacker's talk in the linux plumber uh in here you find the link for his talk he's quite um explaining uh the kci db tool so i advise you uh to check this if you want to know more uh about the uh this tool so this is the part i've been talking about we are uh testing here and then we are sending a callback to the kernel ci backend and it's going to be in the dashboard of kernel ci dot org but if you are a non lava user in here we are working already we've read how to uh this kci project intel zero day google size but they also are submitting their results to the kernel ci project through this exact tool which is kci db this is an example that as you can see um also the kernel ci uh labs uh the tests in lava could be sent to kci db so this is shared uh dashboard between the different labs even lava labs so in here i'm showing an example of test results from the kernel ci lava labs in here i'm showing an example for uh red hat labs which are known uh lava labs okay so quick recap uh well this project aims really to get everyone together to have the our maximum efforts to test the linux kernel um we do provide kernel ci backend kci db tool and it's all to get everyone together and get a better quality of the linux kernel well today uh my talk was about uh how to test uh with kernel ci and explaining a little bit more how to join kernel ci even if you are not a lava user how to get involved in our project we have here uh this list of blog twitter middle english urc kernel ci um freenote we have weekly technical calls and we you can reach us also in link it in okay so thanks for joining and now we move to questions