 OK, I think we should start. So welcome, everybody. And we're going to talk about exploring the Linux kernel source code with Eclipse and QT Creator, actually. So my name is Marcin Bies, and I'm running a small, very small engineering company in Poland. And the first thing we do, or I do, are projects, like debugging some driver problems. Well, the video is playing too slow, for example. If we lick a finger and put on PCB, it's playing even slower and so on. So this kind of task needs to just jump into the driver and fix something, analyze something, and so on. And then I'm providing the trainings, and particularly the Linux trainer development. So this is the training the engineers come to learn about how to debug and try the device drivers. And actually, we need a proper tool for this. So the problem is that the Linux kernel contains the vast amount of codes, like the millions lines of codes. You can easily find how much is it. Well, it consists of, actually, the kernel mechanism, which are, well, very fine-is and very complicated code. There were some topics discussed in the conference, like the interrupts and so on. So it's complicated. There are functions that are or are not called. Yeah, you need a way to see this. So all the virtual frame systems, IPCs, RCU, and all the frameworks are here. And we have the device drivers code. Well, we do a lot of programming here. Well, device drivers is mostly the boilerplate code, like put the value into the register. There's no place for fancy algorithms. Actually, I have to know the frameworks to use the frameworks. Well, so the first problem is I have to have a quick look into the kernel source code, into the frameworks, into the drivers. And actually, what I see depends on the kernel configuration, kernel version, and sometimes the compiler version. So that's the first problem. And the second problem I have is to introduce newcomers. So I've got the engineers who were probably before working on some embedded systems regarding microcontrollers. And they are used to the idea that they have the IDE, and they configure the project, then then upload the project and write it to the microcontroller. So it's hard to tell them it's not possible in Linux by default. So we have the problem. So the solution, of course, is to use VI or Emacs, which are both great IDEs or editors. They can be configured using script. Actually, Excubran CTAX supported in the kernel. There's also a C-scope text-based tool to view, to get into the source code. Well, actually, I used both of these. But, well, people nowadays are used to using mouse, clicking, and having multiple windows. So yeah, that's nice. But well, it doesn't get them the idea of the newcomers. There are some tools like AXR, Linux cross-reference. We have a site. Works. So you can see all the source code here. The problem is the source code is not configured. It's just a source code taken from kernel.org, put here, and everything in indexed. So if I click on the symbol, and the symbol is defined in multiple architectures, I will just see the disambiguation window instead of the proper definition. And sometimes I do not know which one is it. It also requires the access to the internet. Of course, you can configure your local copy, but well, it's quite complicated. So we will cover the two solutions, the Eclipse and Qt Creator, starting with the Eclipse. There's also KDevelop, which actually is the great idea and there's a plug-in for the Linux kernel. But well, Eclipse and QDevelop has the marketing is rolling. So Eclipse is the base of almost every IDE now. So I get people that were working with WindRiver, DS5, times this time storm, everything is built on the Eclipse. So this is the last solution. So we can have a commercial IDE, pay a dollars, usually per seat. And here's the example. It's not advertisement. I'm not affiliated with the company, but well, it's Eclipse-based. I used it in the project. It includes proprietary plugins, which Eclipse license allows, and it's licensed per seat. So you can just pay dollars and have a one-click install of the Linux kernel project. Yeah, so that's all for today. Thank you. No, it's a joke. We are going to do the same using the open source tool and it will take something like half an hour. Every time you have to configure the project, it takes half an hour or five minutes if you know what to do. Okay, so yeah, something like this is our goal. It's similar with various and other commercial IDs. Okay, the first thing, the Eclipse. Eclipse has multiple versions. Actually, I've got one in Ubuntu. I used the one where I was using the Ubuntu 14.04. This is Eclipse-free eight. Now I have Ubuntu 16.04, and it's also Eclipse-free eight. They do not update this. Probably Eclipse is more Red Hat-affiliated. I don't know. So yeah, we can use the newer version. There's a Kepler, Luna, Mars, and Neon. Well, this is important because actually for basic kernel and external models configuration, you can use any version of Eclipse. But some fancy features begin from, for example, Luna, which has tracing tools integrated and Mars and Neon. And there are some additional problems also. So the first thing is to download the Eclipse. Set the proper Java version. In case of Java, you may have in your distribution multiple revisions installed. So there's updates, alternatives, minus, minus config Java to select the one. Then just untar it and run it. Actually, there are two hints. First of all, if you set the path or include all the environments created by Yocto, build system, for example, it will be easier to configure because all the environmental variables will be here for use. Then, well, unfortunately, in my Subuntu Eclipse is looking awful, so it's totally crap how it looks, how the interface is shown because it's using the SWT in Java and they're using the GTK3. Well, we have to force it to GTK2. It looks better. I have tested another distribution since the problem is not here. Okay, so the basic idea. The kernel sources has to be configured before we will start to load it into the IDE. At least initial steps of build process has to be executed. Well, actually, there are lots of build step depending by the kernel version make modules prepare would be the good one. And this is because include generated is, well, generated inside of kernel module and most important thing is it contains the auto.h, which is generated from .config. Depending on the kernel version, there are another files spread through the kernel source code also generated. Like before using device three for ARM, there used to be the unique machine version for each board and well, the file containing the machine version was also generated during the build step from the text file. So the auto.h, this one is most important. Well, then we create the new C project. Easy to the point. What are the problems here? I have to set the name. This is the name of the directory that will be created in the workspace. Indexes will be stored here. Then I already have the source code. So I have to point it to a location when I put the source code. And all the project configuration will be stored in the kernel source code. So yeah, make sure you are using, not using any temporary file like if you use eclipse and to remove all your work after building. It's not a good idea to point it to the director that may be removed. If the eclipse will last file, well, the project will be gone. Yeah, kernel is built using make files. So we choose the make file project. This will make eclipse not generate its own make file. Eclipse is quite good at generating own make files or using auto.conf tools. In this case, we use the make file and set the cross GCC. And well, it doesn't allow us to set what the cross tools will be used. So what's next? Well, project is created. The indexer starts for the newest 4.8 kernel. It's some like 44,000 files. Yeah, which are being indexed. Well, make sure you are using 46 bit version of the distribution because in case of 42 bit, you will exhaust your process address space very soon. And also make sure you have more than two gigabytes of RAM because yeah, you will exhaust it also. Well, fortunately, if it hangs here, you can rerun it and all the indexer will start from the beginning. So that's nice. The project is created but well, it's barely usable. The only thing I have, I have the colored syntax. The macros are not understood. Yeah, this is just a warning. And well, if I will try to control click on this platform driver, I will see nothing. Yeah, it won't be resolved. Yeah, so that's the idea. Yeah, I try to control click and it's resolved. So at this point, a little disclaimer. Eclipse shows the seven errors here. And actually, that's because, well, first, we did not configure it yet. Second, well, the Eclipse indexer is actually a bit different from what the GCC is doing. So it will never be perfect. So the goal is to have either configured to be good enough to develop the external kernel modules, for example. Yeah, so I just modified the project settings. And the first thing we have to do is, well, everything of course depends on the kernel version we are using here. This configuration is for the newest one. So I just force include some files. So the first one is the autoconf h. You can see, without autoconf h, the top one, config of is great. We don't see it, actually it is set in the kernel configuration, so I didn't see it. After including autoconf h, the situation is better. Yeah, so we have to include generated autoconf h for all the versions of Eclipse and all the versions of kernel. The file has to be set as a preprocessor macros file, which changes the way it is seen by Eclipse. Well, there are options like include file or macros file. So this one has to be in the macros file. For newer versions of kernel, there is include config h, which actually includes the generated file, plus it defines some macros to operate on the variables, like instead of if defined, the code also can check if isModule, for example. So the isModule macro is actually generated here. Yeah, so before and after. Next thing, I also include the printk, sometimes another files. It will depend on the output. It's like the control loop. I just get the output. I can see that while Eclipse says this line is error, so I can see if it's error leads to, for example, lacking definition of VA list from the compiler, so I add file here. So those two is good enough, like 99% of the code will be readable. Next thing, Eclipse tries to be smart. So being smart means it calls the compiler, asking the compiler about the default includes, and actually it's asking the compiler just a GCC usually. It's using the compiler variable, not even the CC variable. So even under Eclipse environment, it's not fixed. So we have to turn off the provider or there's a possibility to change the name of the compiler to our cross version. Well, it doesn't matter a lot. Okay, the next thing is we have just removed from Eclipse, it's default include directories. It will not ask the compiler, so we have to provide something else. So I just run the kernel compilation in the verbose mode, v equals one, I get something like this. It's just like for every file it produces the line like this. Actually the line is, well, like two lines more than you can see on the slide. And what's important, you can see in bold, all the includes, arc, arm, include, the architecture is set in the external variable. The cross compiler is also set here. The symbol is defined, the kernel, and another symbol is defined, the Linux arm arc equals seven. So this is a configuration for the Raspberry Pi kernel right now. Yeah, you can see the compiler options and there are some more variables defined like the K build mode name, but this is file specific. So those at the top are global. Yeah, so I have to set the paths and symbols. So set proper include directories according to this and well, the good enough configuration would be to include, include, and architecturally include. Actually we have ARM here. I tested in on MIPS, on PowerPC, and also X86, some Intel Edison boards. It's more or less the same. Sometimes for some, for quite older boards like OMAPs, for example, from Texas Instruments, you also have ARC-ARMAC include and PLAT include. So this also may be include. So this is the good enough configuration for external driver development. Well, the better configuration will be to add everything from here. And also you can notice there is an iSystem directory and the iSystem directory is actually taken from the compiler, but it is not compiler sysroot. It is like, it is internal compiler or libGCC headers and well, the C standard specified something like VA ARCs definitions, VA ARCs macros and there are actually functions to process them and the variable types like VA list are defined in a compiler specific files. So other than this, may allow me to, well, see something like print case better depending on the configuration because well, the print case can be, for example, configured as dynamic print case and then it's, well, heavy use of VA list and VA macros. Okay, I set the symbols, all the versions of Eclipse that do not allow Juno, I think, do not allow me to set the symbol without value so I can just put any value for the kernel symbol. In case of Linux ARC ARM, it's actually important. It's pointing to the variant of the architecture. So I've got ARM V7 here, well, it's Raspberry Pi 3. So for Raspberry Pi 1, it probably will be ARM V6. So it's quite important because the variables are used actually in code to split the code for a different definition. So you can see that ARM V7 specifies the DSB assembler instructions, the data synchronization barrier to just make sure the variable gets to the memory and in case of ARM version six, we have the MCR, which is calling the cache controlling processor to just set it some comments. So probably the DSB is faster, but yeah, I want to see them, how actually it's implemented here. Yeah, the next thing, I said some functions are defined architecture specific. So yeah, I have to set the filter to actually remove some directors from indexer. I do not want it to go to any directory under ARC instead of ARC ARM, ARC ARM only. I also remove tools, script, and documentation. Tools and script has various tools and for example, there are some common data structures like the list, age, or something like this thing here. So yeah, that's quite trivial. Yeah, this is a nice hack if I would like to make it faster. I will just remove the index file, remove from the indexed files that are not built. All the Eclipse has the output parser. So it actually, it's somewhere here in the providers. Yeah, we have the first one, the GCC build output parser. It can parse the compiler output and well, it can see which files are actually being built which are not. Probably, I do not want to index files. I do not want to build. Well, the last step is to have a make, the proper make command to actually be able to compile the kernel. Well, this will depend on your configuration. If you have an environment included from Yachter generated SDK, probably setting arc arm and cross compile is not needed here because well, it will use this from the environment. Some environment variables can be set here. Yeah, we can actually decide if Eclipse is resetting the default environment or is adding some variables here. So yeah, the patch for example can be generated. And well, the index is to be rebuilt. And after rebuilding the index, we have something like a while rebuilding the index, we have something like 13,000 files instead of 44,000. So that's the improvement. Well, and the index will take something like 800 to 1.1 gigabyte depending on actually the kernel configuration to kernel version. So it's huge. It runs very slow in virtual machines for example. Also, it's a lot of IO here done from the Java. Yeah, so yeah, we can build the kernel here. Well, actually this one can be done from the terminal and it actually doesn't get us anything new than this. I have the console output. I will be also seeing problems probably. So the configuration is good enough. Not our problems will be removed. The good idea is to create additional make targets which can be run from Eclipse just by clicking on them. Like buildings that image building the TB or a specific DTB or every DTBS, DTB is compatible. Yeah, so actually usually I'm using the kernel project like this. I've got it configured here for the for the board I'm running. You can see there's a Linux RPI. The documentation scripts tools are not indexed and we can look at some files like, let's see, drivers. I square C, buses. For example, I square C BCM to 708. So this is the driver for this board. Actually, well, the kernel was built. I have all the fancy things like if I hover the platform driver, I will get the definition. If I control click the platform driver, I will go to the definition. Yeah, so that's very nice for learning and exploring the kernel. There are some problems. It doesn't like module in it. It's unused declaration. Well, I can just turn off showing warnings. But well, I would like to have a warning in my source code. There are also some additional things like the false. It's not defined. And actually, what's else? Okay, the false, false and doesn't like module param syntax error but control clicking on module param gets me to the definition. Yeah, so it's nice for the point and it allows me to develop the code. Well, actually, this false is intentional. We can see the project configuration here and I've got paths and symbols. I didn't add the compiler internal definitions. So the false and the VA arcs are defined inside of the compiler of GCC, actually. So I didn't add this. So it's depending on the compiler and on the version of the configuration. Yes, so for out of three kernel modules, well, the configuration is basically the same. And I have a project created here, it's not this one, LDD, this one, which is built using the make file. So if I include my Yocto environment, I can just make the project and well, the first .co module was built. It was built against the kernel placed in Linux Raspberry Pi. This is the same that the clip shows. Well, so I create this new C project. It's a make file project. I include the autoconf page, print kh. I turn off the default tool chain includes. I set the includes, include our arm include. I set the symbols. I do not define the filter here. The filter is per project and here I'm using all the includes and other things I'm referring to the project, another project in the same workspace. Okay, so I've got the project here. So it's called LDD. And you can see it will build. And so console shows everything is okay. The problems are still here. So the quick hint is do not look at the problems. Yeah, it's eclipse problem, not yours actually. So that's the idea. Eclipse has the feature here. I can define for the LDD project another targets like clean or deploy. And actually clean and deploy have to be defined in my make file. So deploy will SCP the module to the remote system actually. Yes, so the result is quite nice. Ida configured to able me to configure them to work on the kernel project. I can hover it, the mouse button over the variable. I can just control click the variable. Yeah, so actually that's the demo. So yeah, if I deploy, I just didn't make deploy. It was copied to the target. So yeah, I'm loaded here and I can just in smot first KO and Dmask tail shows that loading first is actually done. So this is for developing the kernel driver. Eclipse has also another nice feature. Well, my Raspberry Pi system here was built using using Yachto and for a quick demo, I just set this called the extra image features to include tools profile. And you can see here. So the kernel configurations allows it. If the config trace points is enabled, I will get the ability to use the LT TNG. So let's LT TNG create a configuration. LT TNG is the kernel profiler which is built well out of kernel. And that's actually one flag in Yachto here. So running LT TNG create, let me load the models for me. It's using the K probes. It's using the F trace to the various traces. I can LT TNG enable event kernel all. Oops, sorry, too much LT TNG, yes, this one. Our kernel events, LT TNG start. Yeah, so system is saving the trace now. So the two is Raspberry Pi. Let's ping fluid for a moment. And now I got LT TNG stop and LT TNG destroy which will actually save all the data and unload the modules. The session is created here, LT TNG traces. So let's copy this traces, sorry. And I've got it here. And actually Eclipse already has a trace compass included. So we can change the perspective here. It's open perspective or window open perspective. I have the LT TNG kernel perspective. And I'm creating the new project, new tracing project here in this perspective. Let's give it to whatever name, just a test. And actually, yeah, in the workspace the project is created here. So I've got traces and I will open the trace. And actually I just copied the file to my home directory. So let's see this LT TNG traces. There is an auto here with today's date and I'm opening the metadata. Okay, so Eclipse should open it. It's 7.5 megabytes. Well, it became quite huge in the moment. And you can actually see an arrows here. What is calling what? And well, the problem with such traces is, well, they're good if you are debugging some problem with some deadlocks or some processes or some read time applications. They're not so good if I just randomly grab it and show it to you. So actually for an example I have something like this. Yeah, we can switch to the resources here and see the interrupts and what the CPUs are doing. And actually this is an example of, well, why the soft ear cues are good. And so you can see the CPU is running the IRQ here and the IRQ line is locked. And then it's offloading the processing to the bottom half. So this amount of time the CPU is processing the data but only this amount of time the interrupt line is locked and it doesn't accept an interrupt. So yeah, so you can see how the code from the top is running. It's from IMX6 ultralight, I think. So the interrupt processing. So this is also good for profiling and it's a good learning tool. Okay, another thing the Eclipse can do is being the GDP front end. So you can use the JTAG to actually trigger the CPU and connect to board. Or you can use the KGDB. KGDB has to be compiled. KGDB is using the serial port in polling mode. So GDB is connecting to the kernel here and this kernel only starts and hangs on KGDB. We need a serial console to do this. Actually, there is one exception. Up to the point, we did everything just using Eclipse without installing any additional plugins. Now we have to install plugin but well, this plugin is usually the part of any commercial ID based on Eclipse and it also is from the standard repository. So it's a GCC, GDB hardware debugging. For connecting the debugging well, perspective to external process, external device. So well, the thing we would like to put for the debugger's application is a VM Linux. So actually, what we are running in Linux Raspberry Pi is Arc Arm Boot Z image which is compressed image and in VM Linux, I have the L file which is linked during the build process and then well, the parts of this L file are cut, compressed and here the Z image is built. Actually, the good idea is to configure the kernel with debugging symbols, well, I can see the size for the VM Linux does not. It would be like 10 times more for the VM Linux. Well, Z image would be the same. So I'm including the VM Linux. I'm setting the remote debugger as TTY USB zero and make sure the user running the Eclipse has access to the TTY USB zero and actually it's quite spectacular but it works. But well, this climber, the KGDB is very good for presentation. It's not so good if you are trying to do any actual work with it. Well, usually something hangs. Usually you have to use the $30 USB to TTL Serial dongle instead of 50 cent one and so on. So it's very fragile. Actually, it's like one each five times I'm able to get to the screen here if doing it's live. So actually, you can see it's waiting for KGDB breakpoint all the time Well, in JTAG I have the stop line so I can just stop the CPU or break the CPU and see what it's doing. In case of KGDB, well, everything is pulled. So kernel has to pull me, pull the GDB and GDB has to send the command. If kernel doesn't pull, well, I don't see anything. So things like step into, step over here are actually working like setting the breakpoint after the next line and then running the code. It should fall into breakpoint or sometimes not. Yeah, so it works better with JTAG, but well, the more expensive the JTAG the better the idea. So this is basically what can be done using the clips and well, I actually use it for my project and for the training and well, people, especially those used to Microsoft tools are very happy, like it's five, after five days of running terminal commands, they have, well, we can work like this. Yeah, so that's the one. So in case of Qt Creator, well, the background is the same. You also have to get the kernel configured, building in verbose mode to get the includes. It's just the configuration is a bit different. So you do not have to click so much. So I get the Qt Creator for a Qt open source bundle, of course, from the installer. And we have something like projects. Mostly there are Qt-based project. This tool is designed for Qt actually. And this is quite counterintuitive. I have to import existing project here. So I'm importing existing project, putting the name and location where my kernel sources are. And then I have the file selection. So this is more or less the Eclipse filter. Well, it's nicer here. So I just, well, turned off every architecture from Arc. I'm not using. I also turned off every Mach I'm not using. This configuration example is for Bigel Bone Black. So documentation tool scripts also. So this one, yeah, so it asks me if I would like to add it to Git or any other build system. Then the parsing cc++ files starts. It's actually doing it every time I start it. And well, it takes like the 10th of the time that Eclipse uses it. And well, the amount of data is not so big also. Then I have to do the more or less same steps. Define includes, define symbols, and well, exclude the directors. There's no clickable interface here. I have three files created. The project includes project files and project config. Actually, I do not touch project files. It was set here. So I just type the include paths. I just type into the config the symbols. I missed Linux ARM Arc here, as you can see. It's also needed. I can force include some files instead of including the directories. And well, that's it. The project is configured. To compile it, I also have to, in the project setting, define the cross compiler. So that's the idea. And well, it's more or less the same as in Eclipse. Well, in Qt Creator, you have to control click on each symbol while the hovering doesn't allow you to do this. Okay, so yeah, that's all. I fit it in time. So yeah, so for the end, both Eclipse and Qt Creator can be used. Well, Qt Creator is lacking things like trace compass. You have to use the external application. Both remote debugging can be configured, but it's well, KGB is unstable with the setup like this. And well, it gets the people, the boost in learning how to develop the kernel drivers and do they work? So yeah, the tools are very nice and please use them. So do you have any questions, comments? Yes? Yes, I have auto completion for the code. So let's see, for the code and for the modules. It's like the same, yeah? So let me get the modules so I can just, yeah, loading first and let's do something like well static struct timeval and let's save the time the kernel was. So it will be just no get, do get time. Yeah, that's a good do, thank you. Do get, actually it doesn't work. Get time of day, yeah, it works. Timeval and let's, oh yeah, static, there's an error here, so static students start time. So we have a do get time of day start time. It shows me this, there's also a feature that if I just do the control shift N for the symbol, it automatically added the Linux timekeeping H. So that's a great feature. There's like tons of well shortcuts in here. Yeah, so let's do get time of day here and let's struct end time and let's do get time of day of the end time after a percent LD seconds and it should be end time. Time dot TV sec, sorry TV sec minus start time TV sec. The small problem it doesn't save before build so I have to type control S, I did it automatically actually. It can be configured. So let's build, let's deploy and now we have a module that, yeah, so ints mod first KO and after a while if I run mod this, it's, yeah, it's three seconds. Yeah, so this is fast kernel development. And you can do it on the modules and you can also do it on the kernel source itself. Depending on the version it, well the Eclipse may have some problems. It has the problems with module init, module exit. It also may have some problems with print K, for example, which is not so fine. And well, depending on the kernel configured, depending on the version. If you said the dynamic print case to be enabled, it doesn't like print case. Once I have also the problem of container off, for example, which is not so nice. Let's see this version. Oh, sorry, this is the user space code. Let's see this example. This example is using the container off and actually, yeah, I can see all the container of magic. Yeah, the container of allows you to get the pointer to the structure containing the structure you have the pointer to. So it's used widely. Okay, so, any questions? Yes? Well, it's required to build the kernel, but actually in case of Eclipse, the kernel is built and we just want to see this. So Eclipse has to do the initial steps of building actually, yeah, so to parse the source code. So we include the outer conf h to make it see the parts of files that are, well, depending on the defines and not see any part. So if we, for example, see any task struct, this is control shift error also, a quick search. So we can see that actually the task struct is the structure representing the process. It has something great out. So this is the config preempt notifiers and actually the preempt notifiers is the configuration option. So I got this one because I have included the outer conf h and the kconfig.h is in the newer kernels. If it is, it should be included because, well, this one is actually including the outer conf h plus defining some macros. Yes? Yes, I actually did do this because, well, Yocta is using a lot of variables. So actually I have a script. I'm running, let's lapse Eclipse and I have something like Eclipse run, yeah, for this RPI sh. And this one is doing a configuration for me. I can also set the variables in the project configuration, but, well, with all the settings that Yocta is doing, it's a mess. If I'm using the linear tool chain, the only the path is important, yeah? And in case of Yocta, actually the script is, let's see, yeah, it's setting all the variables, yeah? I don't know. The question is, is there any way of estimating the memory and time for the requirements for indexer? I do not know. It's like, run it and, try it and if it fails, just rerun it. Well, actually it's, the Java allows me to set something like Eclipse ini inside of Eclipse and I can set the memory, yeah, the two lines at the end. So XMS, XMX, the one is, yeah, when the garbage collector starts, another one is another amount of shared memory. So if it fails, the thing I do is just increase the values. You can find it in Java reference. And well, yes, please. Yeah, it's a good question. So the question is how it integrates with menu config. So the idea is that if I do the menu config, so yes, so this kernel sources are extracted from Yocta. Yeah, so I'm running the Yocta build system but I take away the kernel and configuration and patches from Yocta. Yes, so it's separated. So if I make arc arm menu config, well, this is the Yocta tool chain, sorry. And the car says it's not here. Yeah, so if I just run it in the shell, let's paste it here. If I change some option, let's device drivers something. Well, the dot config was changed. There's a dot config, dot config, dot alt. The value is set here, but while it will be set in auto conf h, synced by Eclipse, why I rebuild the kernel? So configure it and build it. This is the idea. You can also read them, make her to extract the steps needed for just getting the auto conf age generated. Well, it depends on your workflow. Like I'm covering the workflow when you have the Yocta recipe, you are building on the build server and you have the copy of the kernel on your local machine you are developing. So I just created the configuration so I upload the configuration to the recipe it will rebuild next time. For KGB, well, I used two main vendors here. This is the prolific, I think, and FTDI. And, well, the FTDI chip seems to work better. But, well, the best idea is the investment in JTAC, if you have to. Yes. Well, usually I leave it to the hardware guys. I'm a software guy. Actually, in case of OpenOCD, you have another program that may fail. So I personally was working with the FTDI chips with OpenOCD. They're more stable than KGB, but less stable than dedicated JTACs for a set of box development kit, for example. So it's, well, hard to estimate like this. Okay, thank you. We are running out of time, I'm afraid. So, yeah, you can get me, I'm here at the conference. Yeah, so thank you very much.