 So I think we can start now. So good morning everyone. Quentin and today my talk will be an introduction to Uboot and Linux Sporting on new ARM platforms. So this is basically a step-by-step guide. You will not have a lot of code involved, nothing to deal with drivers writing or anything really code involved. So a bit about myself. So Quentin Schultz, like I said. I'm a limited Linux and kernel engineers at 3 electrons from a year now. I started to code on the kernel a year ago. So my, I would say, the reason why I'm doing this talk is that I have working on, I've been working on adding Uboot and Linux support for an AMX6 custom board. So this will be my feedback to this journey to have Uboot and Linux work on this specific board. So this is an AMX6-based board. This is really a well-supported SOC. And there are a lot of already well-supported IPs. That's why I'm choosing to use this board as an example, so you don't have a lot of coding skills involved. So as I said in the title, there is Uboot and Linux parts. But of course, if it was that easy to explain how to pour the Linux to a board, there wouldn't be any ELCE. So it will really focus on Uboot and then a small part on Linux as well. So a few good rules when you want to add support to a board. You first really, really wish to have the sources of your BSP. And you have, and if you have them, you want to compile and run the BSP code on your platform. So that one, you can validate the IPs actually working with some code, even if it's ugly and maintainable or whatever. Two, have a reference code so that you can use those registers, the workflow, the probing process, everything from the BSP. And have a code that you can use to debug. And this is really, really, really important. And that will really help you to add support to your board. Then you want to focus only on RAM initialization and UART. So only that. And once you have UART, you can start to debug and add new IPs and new features, et cetera. So stop there. Only RAM, UART, commit. It's okay. Then you can have one IP at a time and then commit. Because sometimes you will break things. When you add support for new IP and break another IP, then you can bisect with Git. So these are the golden rules. So a little presentation on the custom board I used. And I put a UBOOT and Linux on. So it's on IMX6. So all quad, we have two boards. Module with an extension board. And on this product, you have Ethernet support, I2C, Spy, NAND, EMMC, SD card reader, USB device from on I2C, GPIO, UART, audio on I2S, HMI, LVDS, PCIe, USB host, RTC, and PMIC. So everything is supported in Linux. And some are supported in UBOOT. The client didn't want all the features. So I'll present how I added support and how easy it was. So first part, the UBOOT porting, which is the biggest part of this talk. So you first have to know that UBOOT is kind of a middle of a transition in two aspects. The first one is the UBOOT had to use to have board header files only. So you had to define in this board header file constants and defines. And to say, I want to probe this device, I want to configure it this way or that way, and the register buzz address, et cetera. And some features, so which common, for example, I want in UBOOT, so NAND, or UBI command, or feed image commands, et cetera. And now it's slowly migrating to K-config options only. And so you can access them via menu config, which is really, really helpful. The second migration is from major driver probing to driver model. So now the drivers are registering in the class, and you can basically, it's basically better, I would say, but it's still ongoing. So there is a little bit of code that is using still the board header file a lot and not K-config, and the same for driver model. So when you take inspiration from other boards, be careful because they might be, I would say, obsolete in their kind of way to use UBOOT. So a little bit of the presentation of UBOOT architecture. So first you have the Arc directory with everything Arc or platform related. So the DTS, the Divid3Source file, the CPU init sequence, the pin mix controller, the drum controller, the clocks, whatever. And then you have the board directory. So it's code board specific. So it's the init sequence of your board. So for example, if you have to, let's say, in our case, we had to set a set of GPIO high and then low in this init sequence in a given timing so that the board could boot. This is where you put this init sequence. You have also pin mixing configuration as well. Some K-config file to say where you find the board header file, when you find the board file, and make file a lot of things. And we'll explain each of them. Then you have the configs directory where you have all the configs, so the def config. Then you have the drivers, the include, you have all headers. And especially in include configs, you have all boards header files. So that's where you define the constants and defines you need for your board. So et cetera. Of course, there is a lot more, but it's not really helpful in this talk. So next, you have to have a workflow. So you don't get lost and it's easy to do exactly the same each time you want to add support for your board. So first, you want to create the board file. So the board file is the file where you say my board needs this pin mix configuration. It needs this init sequence, et cetera. Then you create the board config file to say where you want to find the make file, the board header file, the board file, et cetera. Then you have the make file, the def config file for the board, the header file. And the last two, you have to source your board's config in the architecture's config so that it can be found and define the target config option in its CPU. I will explain all that. But you have to be careful. This talk is about an experience I had on AMX 6 and some platforms like old winner share command files. So you don't have to, for example, the three first points and a fifth you don't have to do them. So basically, you have only to create a def config for all winner boards. So first, you have to create the board file. So you create my vendor directory and my board directory in which you put your board file. In this file, you have all the includes you need and this weird declare global data pointer, which can be used before the RAM is initialized. So it's a, I will explain, it's a register you can use. And you need your RAM. Basically, you just say that the RAM size is equal to something AMX, which is AMX 6 dependent. And then you have the mandatory board init function, which is basically where you want to put all the pin mix configuration code and everything you both need to, well, boot. So about this declare global data pointer, it's usable with GD, global variable, like you see in the previous slide, on ARM, it equals to R9 on ARM 32 and X18 on ARM 64. So it's a general purpose register you can use. Basically, they use it in your boot to set a lot of flags before the RAM is initialized. So they know how to, for example, the size of the RAM. And you can also, for example, disable the console from there. And yeah, a lot of things. So, like I said, is store info, which is available very, very, very early in the boot before the RAM is initialized. And yeah, basically, you have all the info in include asm-generic global data.h to find what kind of info is sourced. There are a lot of variables. Next, we want to create a Kconfig file. So the Kconfig file, which is really important, is in the same directory. So my vendor, my board, another board directory. And you have first the if target my board. So all the Kconfig are sourced in your architecture Kconfig. This means that all are passed. And you of course want only your sysboard, your sysvendor and your sysconfig name for this specific board. So you have to have an if statement on your Kconfig option for your board. And then you set your config sysboard to a default sysvendor for the same and sysconfig name the same. Which are which? So what are they doing? sysvendor and sysboard are used to identify where the board file is. So you would have no idea where it is. It needs these Kconfig options. So first, if both are present, the board file will be under a board sysvendor sysboard file. Then if sysvendor is omitted, it's in board sysboard. And if sysboard is omitted, it's in sysboard slash sys underscore vendor underscore slash common. And the sysconfig name is used to identify the board header file. So for example here, you have include config sys underscore configs underscore names. So from the previous slide, you know that now you're... Sorry, okay. Yeah, it's not where to find the board file. It's where to find the files that make will use to compile. So from there, you will have basically myvendor, myboard, the makefile. And with sysconfig name, you will have include configs myboard.h. Then create the board makefile. So it's under board slash myvendor slash myboard makefile. And then you just say, I need to compile my board, which is the name of my board file. Fourth step, you need to create the board that's left config. So here we have the architecture, which is armed. Then the platform, kind of platform, you have imx6. And you want to have this target myboard. You remember the one on top of the kconfig file. So here, the same with config in front. And of course, we want uart driver, otherwise we wouldn't have any uart. And that's it. That's basically the first thing you need to boot an imx6 baseboard. Yeah, so basically here, so in this defconfig file, you put everything you can find, and which is selectable in menu config. So it's basically drivers, features, so commands, you would behaviors, lives like RSA or anything else that you can select in menu config. This is the minimal example for a board head of file. So it's the fifth step, create your board head of file. So a lot of different options. And you see here, the most important one include imx6 underscore common.h, which basically is the SOC board head of file, SOC head of file you want to include in all imx6 based board. So of course, we'll have different ones for different SOCs or architectures or whatever. At first, we want to be sure we don't compile it twice. Then you want to define, so this is all imx6 specific. And you just say where you want, where is the base address for your uart driver. And a lot of others define constants and how to compute things. And that's it. That's all you need for imx6 from the start to get uart working. Then you have the source boards kconfig file. So you have to source your boards kconfig file in your architecture. So I'm not really sure I've looked a bit in Uboot. There are two different ways to source your board kconfig file, either in directly the ARM architecture or whatever architecture you're working on kconfig. So like I show here, so you source your board slash my vendor slash my board kconfig file directly in ARM kconfig or in your platform kconfig file. So currently, for imx6, it's the last one, but for others platforms, it depends. So take a look at Uboot source code to find out which one you should use. And I think it's the last step is to add your target my board you used in your kconfig file. So kconfig option. And you just say, yeah, it's my awesome board and it's on imx6 solo. So imx6 solo, here you select all the options you cannot select in kconfig. So they are visible in module config, but you cannot select them. And this is the case for imx6s. And that's it. So what do you need to know now to add IPs, to set up your board, etc. Is that there is really specific Uboot unisequence. So first, you need to know there are two lists of functions that will be called. The first one that will be called before the relocation of the code into RAM and one afterwards. So the first one is called unisequence f and it's in common board f. So there is anything that is needed to initialize the RAM. And basically, it's really everything that is low, low, low level. And then you have the board. We need sequence r, which is done after the f, of course. And then you can use the RAM inside it and other things like the clocks are initialized, etc. You have to know that some functions are run only when a constant is defined. So for example, in unisequence f, we have a function called board early init f, which is run, I would say compiled, only when config board early init f is defined in your born header file. Yeah. Any function in this, in one of those two lists that returns anything else that a zero will fail Uboot. So it's stopped and then you can stop in middle of UART. So you know the start of Uboot, you have everything printed. So then I have so many gigabytes, it can stop in the middle, just for, so you know. And so to debug that, because it's really, really annoying, you can define debug. So it will add a lot more of code, but we'll have in this loop of init sequence, the one that is failing. And you have also to know that not all features are, I would say features are available in all functions. So for example, we had a problem with uDelay not being usable in board early init f, and it took us like two days to find out. Then you want to of course enable the drivers of your IPs. So you may want to take inspiration from boards with the same IP. So for example, I took inspiration from the Sabre SD, which is basically the same SOC. So this is a good start if you have been in trouble to find the correct drivers and the correct config options. You want to inspect the drivers to be sure it's really the one you want. So you go into the appropriate subsystem. For example, from NAND, you want to go in drivers slash mtd slash NAND, and you just go there, open the first one that looks a bit like the one you want or is named after your IP. You focus on the behavior first to see if it can match, then the registers, the bit offsets, etc. And you check for undefined macros. So the first and two items are to select which drivers you want to use. And the third is to find out which macros and constants you need to define to make these driver works or even compile. The fourth one is that you want to be careful. There are a lot of if-def blocks in your boot. So these also define constants you want to define in your bold header file. Then you want to look for the object file of this driver in the make file of the subsystem. So for example, for the NAND, we'll go to drivers slash mtd slash make file. I think I will just present it later. And then you have the base name of your driver, so mydriver.o, and you know that you need to enable this, to enable the driver, you have to set this define either in the kconfig of your bold, the def config of your bold, sorry, or in the bold header file. The best way to do it is to know where to put it, is to grab this config option. If it's visible symbol in some kconfig file, so if menu config, in menu config, you add it to the bold def config. If it's not visible, so it's in menu config, but you cannot select it, then or if it's defined elsewhere in bold header of others, bold header files, you put it in your bold header file. Make sure your bold, your driver is compiled. It's already happened a lot of time to me for me. So you want to look for mydriver.o file. This is a good indication. So a small example with the name driver, how I enabled it for our bold, so driver slash mtd slash name slash, then underscore mxcs.c is the driver we want to use for this bold. I found it the same way I explained the slide before. So there are a lot of three different defines we have to set. The first one is config none and mxcs that you need to compile the driver. So this one here, so in the make file of the subsystem, it's config none mxcs. And you need it in your depth def config. Then you have config sysmax none device and config sysnone base constant for configuring your device. So in your bold def config, you add none mxcs and say yes, I want it. Then you go to the next, which is the bold header file, and you define what is needed. So config sysmax none device is one. We want only one device and its base address is this. And of course, it's an IP. You will most likely have to set the bin maxing. So you go to your bold file in the bold init function and use this mxcs function, but you can directly write to the registers to set the correct bin maxing configuration. And that's it. You now have your driver, your nend driver in support for you would. A little note on device trees. So I know there are device trees in your wood now. It's slowly, well, it started in 2012. And the code is slowly migrated driver by driver subsystem by subsystem to support device trees. It's still an ongoing effort. You need driver model support for device tree to work. And this, the driver model is enabled with config dm. And most of subsystems and driver have a lot of big, big, big if def blocks. So you can't really choose which driver you want to use dm with and some you don't want to use dm for. So it's kind of all or nothing. And for us it was nothing because the nend framework isn't support, isn't migrated to the dm model now. And we had to support it. So I will not really go deep into that. I mean, it's all I can say. But so you know, there are device tree support in, there is device tree support in your boot. And it's still an ongoing effort. So please help if you can. So the effort needed to support, to add support, you would support for my board was basically, so everything that is working now is infinite e from on i2c, nend, emmc as the card reader, usb divide, gpu, uart, audio, and PMIC. And all I needed to write is 510 lines. And basically half of it because this 160 lines here are only for the RAM configuration and are given by the BSP or by the vendor. So to add one line in the kconfig file of the architecture to source my board kconfig. So here, which is 15 lines long, four lines in cpu arm v7 mxs kconfig to define the target underscore by underscore board. And actually there's only 100 lines of code of code only setting the pin maxing in the board init config, the board file, which is the board init function. No modification otherwise of you would source code. So that's really important. It was really easy for imx6. But of course we had some back. So we got a really, really weird bug. For this client we had to use signed filimage. And uboot is actually checking the filimage before booting it. And it's based on the lsa lib and it was just crashing in the middle of checking. So I had to look a bit inside the lib code and I couldn't find anything really relevant. So out of luck I just said why not update uboot, right? So from the 2017 version from March I went up to the one from July. It was quite easy to only take the board header file, the board file, basically whatever I just said before, and make sure options defined in board header files are not kconfig options now. And that's it. Then I compile it, run it, and it was working. So it took me like half an hour just to upgrade. And so that you know it's really easy once your board is supported upstream. You just have to update and not bend your head on your table because you don't understand why it doesn't work. First you update and then if you can't, if it doesn't work then you just look into the code. I think it's pretty decent technique. So there are also other problems encountered. So like I said we had a problem with udelay. So our board any sequence was basically toggling a few GPOs with a given timing. And all signals, this is very important, all signals even UART go through its FPGA. So basically if the FPGA is not powered, no UART. So failing any sequence, no FPGA, no UART, no hair on the head anymore. So it took us like I said almost two days to find out that you cannot use udelay in board early in its app. So the workaround for us was to use a folder with cpu-relax and then it worked. So I think we have to send a mail to the mailing list to ask why or what is the correct way to deal with delays in early boot sequence. Sorry, okay thank you. So the answer is to use timer in it and then you can use udelay. So once it's in it you can use it directly. So thank you. So yeah, forget this slide. So that was it for uboot. Now for the Linux kernel it will be really really quick and have to be. So your workflow, one create the device 3 of your board and add your device 3 blob to the make file and then create the dev config file. That's it. So the device 3 it's a file in a special format, so dts device resource purely describe the hardware of your board or should be at least and it matches an IP with a driver thanks to the compatible strings. You can find a documentation of these device 3 nodes in documentation device 3 bindings and of course I will not present it device 3 in itself here because it's a really long subject. So you can find here a really interesting talk given by Thomas Pettazzoni, one of my colleagues on device 3 four years ago and next one. So create the board device 3 you want to write really you want to write a map of your IP relationship to know which one is depending on which one. This can be really a good start to understand how your platform works to your board works. So first you want to find the SOC DTSI so everything that is all IPs that are inside the SOC they are already defined. So you just have to find the SOC DTSI for so for our body was IMX6 I think it was dl.dtsi. Then you look for IPs driver in the correct subsystem so like we did for Uboot you go to the correct subsystem and then you can try to grab the code name of your IP it's usually a good start or you can find IPs that are really close to your IP so for example I had to on another other subject but for PMICs it was not really exactly the same code name but it was working with something really close so it's not mandatory that your compatible will be exactly the name of your IP of course. Once found you look for the compatible swing in your driver and you find it in the DT binding documentation so you know what kind of properties you need to add to your device 3 and you follow the documentation you write the correct binding and some you have to know that some bindings are framework-wide so you need to go also to DT binding documentation of the framework so you are sure that all the properties in your device 3 are set. Yeah basically for IMX6 that was really easy because the SOC IPs are all defined in the SOC DTSI and I just had to add them so here is the example for my awesome board so we have the IMX6S so it's a IMX6 solo board DTS you include the correct DTSI so it's DL because it's dual light or solo or I mean Frisk and it's really sorry for my language but fucked up because you have IMX6 Squad so Q but it's almost the same as DL so dual light but yeah solo works also I don't know and for the PCIe I had only to add the reset GPIO so it just said the reset GPIO of PCIe is here it's active low and the power supply here is here this regulator and I want to enable it and you are also and that's it then you want to compile your device 3 with the correct when you enable when you choose the correct art platform so for example as I said Frisk it's IMX6Q there is no option for DL or S so you put everything that is IMX6Q DLS in this part of the code and then that's it when you have a DevConfig that is selecting config soc IMX6Q your DTS will be compiled into a DTB you create a DevConfig for your board so you start from your SOC DevConfig so here in my case it's IMX underscore v7 DevConfig and if you're not as lucky as me you have to start from the multi v7 DevConfig with all the SOC family and all the drivers and most of drivers enable so it's really the second options the second steps you have to do is to strip everything you don't need so drivers that are not relevant to your board features basically everything that is useless so SOC families you don't want to build anything admin related while you have a IMX6 SOC yeah then you add the config of your driver you want to build so as I said you grab the name the base name of the driver like we did for Uboot so where is it here so you have the dot o and then you just take take this one and add it to your DevConfig of course not the one from Uboot but yeah here we go yeah you have to know that most drivers depends also on subsystems of course so if you want to enable the driver uh the IMX6 NAND driver you have to enable the subsystem the NAND subsystem as well I mean it's logical problems encountered the PCIe driver was probing but not enumerating any device basically was working BSP so I knew it was my fault or mainstream something missing in upstream that was the case I was missing a regulator support so yeah I wrote a patch send it upstream and that's it it works same for Ethernet driver it was missing a post reset delay for the PHY so it was not initializing and same wrote a 20 line patch send it upstream and that's it so the effort needed to support Linux Ethernet I mean everything here is supported now for with 1000 lines and yeah most of it is the DTS so you're helped with the documentation from the DT binding and the DevConfigures have to select the correct subsystems and drivers so it was one line in the MAC file one more line in the MAC file 20 lines for the regulator supports etc no modification of Linux source or code otherwise thanks to the well-supported IMX6 SOC and the ports the the IP we had in the extension ball of course we had some problem so we got some weird bug with dual display so this display driver would crash completely crash when we had HDMI and LVDS DTS node enabled at the same time even if the both output was not connected at the same time and we found some quirks on the mailing list and decided yeah well it's better to go to 4.13 did really copy the DTB make sure nothing has changed in the bindings and yeah it worked so half an hour work and it just fixed my problem so basically I just wanted to say once your SOC is really well supported it's really easy really really easy and not time consuming to add support for your ball so that's it for me do you have any question I think we have two or three minutes tops yeah okay a couple of comments about the IMX6 so this is a bit of a legacy design that's why it looks like it looks a new boot so it's not super about the pinmux controller the ram clocks that actually goes into drivers now okay so because the IMX6 is a legacy design it's still in arc but on the newer ports it goes into drivers and actually if you are doing a new port use are supposed to use driver model always so avoid the legacy stuff yeah and if at all possible use device so that it can be shared with Linux again since the IMX6 is kind of legacy ish it's not there yeah and we had some problems with some drivers that when combined with DM would just not work at all so well patch it welcome yeah patch it yeah of course yeah the client who really wanted a solution right away so I don't otherwise the MX6 port is obviously very good so by the way on slide 34 you have a typo in the device tree yeah the root node is supposed to be forward slash thanks okay thank you anyone else yeah I will put it on on internet yeah on the website just after the the talk no it's okay hi great slides by the way but how do you know it's done I mean how do you know you should finish porting what's your test cas suit do you get it from the customer or do you have a standard set of tests for compatibility basically yes I use the use case of the the client so you have a set of commands in your boot so for example to test NAND you erase the NAND write to NAND boot whatever it's in NAND now or load it into RAM and then test it's the same as the file you don't load it from any other mean so basically yeah you have to set your own test I don't know if I think there are a test suit but it's I don't know nothing about it so okay yeah wait sorry yeah okay so there is a package named empty the test that you can use in you would thank you and yeah but if you want oh I think the question was in you would and Linux both right yeah okay so in you would you have nothing really defined you have to use yourself the NAND comment and in Linux it's empty the test attention okay thank you uh yeah uh thank you