 I'm going to give a quick overview of some of the stuff that we've done as part of the Fedora work to standardize booting systems, Linux arm systems, specifically MV-7 because that MV-8 evil nasty UFI crap. It just works apparently. Apparently, after a bazillion man hours of work, it just works. One day you'll get, also, you'll have this lot of mon hours done. Yeah. So, many, many years ago when we first started down the road of booting Fedora on ARM, we had this wonderful concept of things we needed. You needed to know things like memory address locations because the kernel had to be loaded to specific memory addresses and there's no way to report to them by anything except a hexadecimal memory location. You'd have to wrap the kernel in Macimage because that was the way it worked. And you'd have to wrap in it, RAM, FS. You would need to know different options for different forwards. So, you know, like the TI stuff would go to location 800000000. That would get 80,000 in. And then, you know, like Tigero is... I had a feeling he was just here. No, I promised not to. I hope I didn't. You had to know information about the particular system that you're booting on. You'd have to know that, oh, I need to pass this particular argument on the boot command line so that, you know, it doesn't try to bring a video because that doesn't work on this board or whatever. It was really, really messy. So you'd have like a boot.sql file. And this is a simple, very, very simple boot.sql file where you set the environment with some bootogs. Say the console is this, the root is here, root way. You know, panic if nothing happens and then you just, you know, load the kernel from MMC 0, MMC device 0 to this location, this file. And the same for the NMMFS. And then... You may get no output. It's no serial port, so... It literally doesn't work on the devices where you get telecoms being passed. Yeah. And sometimes, you know, what works on one board doesn't work on another because the one you've been working on had 512 micro RAM. This one has 256 and you're trying to work to a non-existent address. It was really, really messy. And as an end-user, you don't need to know that... You don't want to know. You shouldn't want to know and you shouldn't care. Could you summarize the situation and pre-word to that? Ah, sure. Cued embedded hacks, I'm trying to... No, no, no, no. Cued embedded nonsense hacks. Oh, nonsense. Cued embedded nonsense hacks were later. It was a mess. To get a system to boot in this old format was a complete not a mess. And then we wrote... Oh, Brennan Cowboy, who's not here, wrote a tool called Unboot Confidic, which attempted to do fancy things like detect... Oh, you're on a... It would spit out a boot SCR that was about a thousand lines long. And it attempted to detect... Oh, this is a TI system and it's this... And so it wooges these addresses... Double shell, like, it was just... It worked. It was a nice stop-gap fix. It gave you a menu, but it was really, really ugly and it was convoluted. And, you know, you'd have to run the image on everything, so you need to know, you know, the memory locations of where you're going to load your kernel because you need to tell the image, the kernel needs to go here. So you can't even, like, build time, you couldn't run the images as part of the build because different kernels needed to load the kernel... Different ports needed to load the kernel in different locations. So you have to do it at install time and we're getting lots and lots of just messy things trying to deal with... And then you try to integrate, like, telling them to come to... Oh, you need to be able to detect that you're running on a panda board so you need to set up these things or you're running on a high bank and you need to use these locations It was really bad. So we kind of went for a standard. And the standard we went with was syslinux... Pixi Linux, EXT Linux, the syslinux standard for booting. The main reason was when Hasada implemented Pixi support in Uboot, they stole... Well, not stole, but they borrowed all the formatting for Pixi Linux. The config file, so they told Uboot to read a config file, they told it to be able to boot the kernel and load the new RAMFS and parcel the options without actually having to know all that detail yourself. And so we were able to extend that quite easily to boot from things other than Pixi booting. So a snippet of an EXT Linux config file is a label, a kernel, we add it to the standard, it's like a pseudo thing, FTDDir, Uboot and the way Hasada implemented it, there's an FTD option which you can use, but it loads a specific the DTD file, the device tree file. Passing the FTDDir entry says, here's where you can go find the device tree files and otherwise on Uboot it's a no. I'm on this system and this system needs a particular you know, I need this file so it will look in that directory and load the file that it needs. And then you append your boot options, you're init.rd and that's it. From an user perspective it's much simpler and when you actually boot a system that has this defined, you get a menu and the menu at the moment is quite rudimentary, you get 1, 2, 3, 4 and by default it will boot 1 and if you want anything else you've got to hit 2 and enter or 3 and enter rather than having a hierarchy support which is something that Uboot has all that features it's just not wired up at some point we could make it even better by enabling, you know, push the hierarchy and hit enter on the kernel that you want. That's from one of my one boards boot entry, it's pretty simple. So to get all that implemented we use the existing sysboot and Vixy support and Uboot we standardize a bunch of config options so that one of the big issues was some Uboots would have fat support some would have EXT Linux they'd better read EXT file systems some would have no file systems they would you know, there would a lot of inconsistency between build to Uboot as to what it would work and what you had to do to get it to work we standardize the boot options and bingo, it worked so I'm going to boot one more Well, I've got a lot of them I just need to make it more visible I was just going to go through some of the way that it's implemented so that you can see how it's actually not really that hard I always use Venom maybe Kate sucks I normally use Venom I have everything pretty small most people look at my laptop and can't read so the first thing that I got into Uboot was this viable config destroy defaults all it does is it defines a bunch of default things we define that we want the pixie boot options and to be able to set the gateway, the host name DNS, pre-print, etc we define some pixie options we say that we want to have fdv support for the device tree because that's what everything uses today and it is awesome That's great, on 32 net So in here there's the command boot i which the AR64 uses the boot i command instead of boot z to boot for whatever reason they decided to do it slightly different we use boot z Define dhcpl the file systems we want basic network we have the pixie command we can do the command line so when a board decides to support the standard all they have to do is include this header and then they get all this stuff that we want to find there is no xms support in Uboot upstream if somebody's written it I'm not aware of it I had somebody internally to write it to write it and she was supposed to be but I need to follow up and find it out there she took a book out of it I think she just got distracted and did other stuff we would add xms if it existed on the other hand it's still pretty standard in every install I've seen that you have a separate boot I mean ultimately you have the xd boot and then run and in fact standard speed on ARM has a xfas root file system as per standard server complete by default yeah I did a pixie install in flora 22 of the server I got a xd4 boot partition xfas root you're still going to want that because you're going to have an encrypted volume you're still going to want that once the board does that we then have the distro boot command header which defines the default boot commands the way I actually ended up getting this into upstream Uboot was Steven Warren video developers somehow heard about the work I've been trying to do in standardizing it and he saw it and he's like this is awesome we need to get this everywhere and I had submitted patches and I wasn't getting feedback and he helped to push it through and get it accepted into upstream he ported all of the tag reports to user when all the Winner stuff landed upstream they supported it by default slowly the imx6 stuff is getting ported over to it it's in a bit of a mess because every vendor that puts a new imx6 board upstream in Uboot they do a lot of stuff just slightly different and so there's some ongoing work there to make things more consistent and standard I've submitted about a number of series that are quite for about 30 patches that are standardized a bunch of that stuff there's still a bit more work to do there but the imx maintainer the core maintainer is very happy to accept that those changes so we're getting there in that way it's getting there slowly and other people have taken stuff like there's an iftap coming from Sandbox that Uboot has this Sandbox feature where you can build it and run Uboot as an application on the imx6 system and it's like a testing thing they've implemented full support for it so that you can go through and test it in that environment it requires some slightly different config options to be set up if the device and a lot of this is like if the device has they said config command MMC config says we have an MMC device it then enables the ability to boot from the MMC and if it has you know SATA it enables the SATA stuff to be Ubooted and sent to SCSI there is I mean weirdly different vendors have done strange things over the years and it's taking like three or four years to try and unravel a lot of it some like Marvell's devices that have SATA they put it they implemented it as the IDE command in the high bank they implemented the SATA support as a SCSI command and some people implemented the SATA support as a SATA command they're all basically the same thing and they're mostly using the same code with some slight tweaks so that's why there's a command IDE I don't think you'd find an IDE device attached to an ARM system anyway today but it's not a V7 system V5 you will but all of those GPIO lines it's possible shush by default implemented pixie boot support so that any system that you get will at the least at the moment it's kind of a little not user friendly you can't redefine the boot order easily it's pretty much how coded you know we've done it like MMC, SATA and then pixie last and you can't really easily and again that's bit more dependent and it's a little bit more dependent ideally an ideal world would make it like a BIOS or UEFI you can quite easily redefine the boot order on the fly or you say I want to boot from USB just this one time to boot from USB you've got a right type of couple of commands it's not the most user friendly but it's getting there can you prove some of us wrong can we view UFI as a payload for probably there is a grub payload for you boot? it's been done someone's done it actually they did that in linaro originally it's like the hack to kind of figure some of the stuff out and there was a series of patches that was just recently that will now use you to build Uboot as a UFI application I would like reverse so you boot which has basic which has board support Uboot will load UFI as payload and then we have one standard that in theory has been done it's something that we as the Fedora project are doing Armwave 7 aren't really that interested Fedora's reminded me I destroyed that brain cell actually I have done this on V8 when we got the HP hardware I guess I can talk about it now we actually sort of convinced HP that they're just shipping it now they're V8 hardware I know it's V8 but they're shipping a new firmware that has UFI and then all Uboot can even switch whether you want to boot a Red Hat OS or not and that was built originally the hack was put together by me and a couple other people the point is you actually it could be wrong here because I thought this would never work out the whole trying to clean it up and just by being stubborn and pushing it I'm pretty stubborn and big headed when I want to be and ultimately having John Masters in a talk in front of a whole bunch of people admitted he was wrong and was worth it yeah it was wrong actually it's Uboot I mean it's we don't have the answer for pretty good damn quick ones it's not perfect and some of the issues that we hit we still have issues like putting Uboot onto the system and that's not anything we can control most of that is vendors are being lazy and cheap and rather than spending a dollar I'm putting a mega nice spy flash or whatever on the board we had this with the 964 consumer spec it's still not a requirement that you have I argued this for I helped write those specs I argued for weeks that there would be a requirement you have to have a spy flash hold this purpose well and there's a whole bunch of other issues like the consumer addition of the board like they don't have okay if you're using it as a consumer device to say develop a set top box or mobile phone or something like that yeah you probably don't want why necessarily on it and you don't need a serial console and stuff like that the development board aims to enable development and so the fact that they don't have pinged out and in fact the TTL is a non-standard 1.8 volt compared to every other device out there the fact that they don't have why an ethernet on board while it's a minor inconvenience when you use USB it actually makes the device a whole unusual but in this to come back to boot learning at least with the enterprise boards we made it a requirement for every system and I'm going to try to go back and do the same for the consumer stuff for this reason because we will see your boot on VA we will see this stuff happen on the other side and we do need to make sure people understand when they small, I don't care how nasty there's something that's persistent that you can keep the boot loader independent of having the OS shut down as long as they the vendor when they build a uboot they follow this it's really easy for us to support I don't think it's better if we didn't have to ship okay if we only had to ship one uboot binary ideally we don't ship any uboot binaries the vendors and it supports a standard and it just works but the reality is that most of them don't ship it one of the other interesting side effects of this when Dennis first proposed the patch just to upstream there was massive massive pushback from the upstream uboot community and it wasn't until Steven got behind it there was a bunch of factors and forwards and eventually when Steven picked it up it was finally reluctantly accepted as an optional feature now they're talking about it as a default well in the vast majority of cases when companies submit patches the general response from some of the core maintainers is why the hell are you doing that why don't you use distro support and so it's gone from something where over our dead body is that ever going to be accepted where people are going actually it generally works pretty well and Debian has adopted it as a distro well sitting on the fence we don't want to break anything that exists so we want to support the other stuff but in theory it's a really good idea and so like so Debian guys are back behind it Debian guys are full behind it they'll inherit that for Debian in theory but like I mean we pushed and were the first to actively use the multi multi SOC kernel support and that hurt us quite a bit in terms of the budget support and hanging and everything but now Debian is using that by Debian as well and so I mean was sort of led by example a whole bunch of pain as a result of that but now we're in a situation where it's amazing the number of devices where we just like all the organist stuff we just build all the organist for you guys and we have a number of people email boys or pop up on ISE and they're like hey I've got this random device in a lot of cases that I've never played or heard of and they've just gone we plug for Dorian so what would be good actually looking to the future I'm sorry to keep giving you that but I'm not that sorry when you when you if you had a I know you've got some documentation but if you have like a one reference when myself and others are going and meeting with some of these guys and we're talking mostly so but they're also saying hey we're building this embedded board in the future it's going to be really cheap but servers everyone knows where if you're building a embedded board make sure it is to this so do you have a single reference that we can find? there is a documentation file in the UBOOT sources that says how to implement it UBOOT is actually pretty good about that but realistically what you do is this is the Tegra common file which defines across the bottom so they've got at the bottom here it's not defined config SPL build so if they're not building the SPL include the config destroy defaults have it so when they're building the UBOOT.bin file include the config destroy defaults in the common post which is things that happen post loading and setting the system up here they're defining like setting up the USB keyboard and the Tegra keyboard so the AC100 all that kind of stuff some Chrome OS things so you need to they've got the memory layout which is defined on a per board basis if you're trying to standardize the memory map on VA right every vendor said not a chance of not being registered but you in here they're including one file which is the yeah so they're defining the boot device targets so they've got MMC1 MMC0 so this is defining the boot order USB so you can put a USB drive into the Tegra Pixi and then as a really last hold back the old DHCP method which we probably should just really remove that from all of this it's not so unnecessary they're including the config destroy boot command file and then so there's a boot and extra and bugs they're just taking the config extra and settings this boot end is a variable that's picked up from the destroy boot command and that's it, that's all that's needed in the board side to take it then in the specific board implementation they have a little section that says they're defining the memory settings to say the script ADD variable is this location, the Pixi file ADDIO and there's like four or five variables that have to be defined for its own work and as long as you define those variables it just works and it hides completely from the end user then the need to worry about memory addresses that was such a kind of argument about it at the start I think probably from your point of view there is section in the readme file and the uboot source and there is a number of commits where vendors are going to convert to generic uboot in a git and so they can actually go and see what various different SACs it does there's the readme.distro file and it goes through and tells you basically how the menu stuff works how to implement it what you need it's not a huge document but it's generated in my mind but it tells you most of what you need to do and all you need to do is look at one or two boards and say ok this is how they're done and it's really there's no reason not to tell people we don't build a server then go to this one if you want to support the way you do if you're not going to support UEFI and have an ACPI this should be the only other way that we will support you then I will modify the rhetoric I mean ultimately the fact of the matter is there are a number of very useful use cases in uboot whether you like it or not and so if they're going to use uboot if they can do this then it makes our life easier similar to UEFI there is very little that we need to support in this instance the ext1 is conflict file or you know like when you fix it but when you fix it is the API between the firmware and the system load and we're saying like anaconda is going to write out the conflict file as long as you read and pass that file it's going to boot and it's going to work I mean it could definitely be pretty useful I mean on the O and S stuff if you've got it plugged in you get uboot output on your screen and your screen plugged in and you get the output you can select it just like a desktop system the table of stuff is not quite there it's only outputting on the serial console but it wouldn't take much to get it there's a lot of improvements that could be done it could be made more user friendly so that you can just use arrow keys rather than typing 1, 2, or 3 depending on which boot option you've got we haven't seen this too well we want this build this card doesn't actually have anything to boot so it's going to go through and say oh we can't boot but it's trying to actually fix you boot at the moment that it's not plugged into the network so so I mean the environment is a little ugly looking here but a typical end user doesn't need to know any of that it still supports all the old but if you really want to use a boot.ser file and do all the ugly stuff we haven't removed that support we're just saying the default and the first thing it's going to try is EXT Linux and that's the way that it should be done like here we have the FTD file defined so that's what Uboot appends to the variable that's in the FTD and it loads that file so you get a tgb file loaded so like the Uboot itself the files are standardizing what they have in the name the device tree files Uboot itself knows more hardware than the file needs the user should never need to know that we tell it here we can go find them but it's really simple I probably should have put an OS on that NMC card rather than just Uboot but it's simple and straightforward and all this stuff is hidden in the years it's taken us about three years to get to the point where most things support it when we first did it I was carrying like half a dozen different boards or a dozen boards then like as old as I went and stuff got upstream I just supported it by default Tegra got ported upstream some of the I sent patches to port all the TI boards and they rejected them because some of the boards they didn't want it supporting so it better be a little more selective and that I think eventually I think I'll overcome that but something has gone from this is ugly and what are you doing and it's still from time to time so one of the big things is well you need to wrap in an upstream line you need to run the image you need to boot the wrapped kernel because then we can verify that everything is right so the average user that I'm carrying and having to actually wrap the kernel in a wrapped kernel is a confusing, convoluted extra step I know you can make the tools to it but you can't just then run the tracker I want to rebuild my NITMAPS and add a couple of extra modules because you've got to know I've got to wrap it and add these extra it adds something different from the people that use it PC or something else they're going to be like what am I doing here and ultimately we want it to be possible to the other architectures it's possible the growth there is actually a port of growth to Uboot as a Uboot application yes the problem is that the way that it is done it kind of goes into the growth binary at build time an address where it's going to load which does not work across the internal systems it could be and so we actually have the development we do have enabled the config API option which enables the growth application to work it may be relatively simple to get and maybe we could use growth but it's this was the part of the resistance this is a great standard to get to have the handle but you could then just long term look at having another video but it absolutely would look over that are still using this no use of it to load growth which then people that know even closer to what they're using amazing this is almost like a bias as opposed to growth this this this got us to the ability to run multiple kernel versions with very minimal effort without having to know the massive co-base that is growth one of the big things that's missing from what we have is command log editing so that you can you know if you need to enforce on equal zero because for some reason you need some SELinux issues that is preventing you from booting you have to put the sd card into a different system and edit the file and do it it doesn't currently have we'll boot the last video it doesn't look well when you add the fruit which was what I was about to say I mean if we had growth that problem just goes white I killed the well it doesn't just mean but I strongly encourage the demise of the global new boot project in Lenaro just from a resourcing point of view because we had that or some of them but there was still a lot of interest in and that kind of do exist probably wouldn't be a lot to get that yeah it's definitely an option yeah the good way and stuff that we support today is a million times better than what we were looking at two years ago we just say now is even possible then there's a slowly getting on board with it and upstream's getting on board with it and they kind of you know when I first submitted I submitted the first patches and had copyright Red Hat people were kind of like oh my god it's the end of the world and some people were like Red Hat's contributing to your boot and then and then now you know is the maintainer or when a tree so there's quite a lot of Red Hat contribution going on into your boot I mean ultimately even if we don't get growth support this is still needed it's not it's not you just maybe add something to it if anyone has any interest there's a bunch of boards that still need boarding and send patches upstream do you have a list of what you did Chromebook and the Chromebook's a special base of panel that's not a beginner's I didn't say it wasn't a panel we have most of we have all the bits in place to basically a whole recipe to enable us to get to help that sounds very interesting I'm going to ask another question I'm going to ask another question the new Chromebook was very stuck it was stuck on the back yeah so I thought you were saying the Yubu the Yubu comes upstream the hard bit is getting Yubu into the right location so it can be loaded and to make the best user experience you need to pop it open and take the screw and completely rewrite the spike latch but there is ways to do that the take-off hardware it sort of locks down with the ChromeOS bootloader but they had in the SPR flash there is an area that you can write a quote unquote legacy bootloader to but when you write that there you take your Yubu you wrap it in this stuff that they come into a I think it's a call boot file system and then you're essentially dv that call boot file system wrapped Yubu bootloader into the SPR flash it's flash address once you've got that you hit some control L um which then enables you to boot fedora and what you've done but you have to push the control L every second boot there's another setting for that and actually it's easy to do it's easy to do it's easy to do it's easy to do it's easy to do I think there is possibly those settings hidden somewhere it's not very well documented at all I mean it took us some time to get out of various different people that there was this memory area that was really right that we were able to write into and that you don't just write the Yubu into there but basically have to wrap the Yubu with this stuff so we know all the steps we don't necessarily know the exact details of the steps to make it easy so essentially the plan is to have a script or even ship a free package image that people can just flash in at which point you've got to show them a USB stick or an SD card and probably the internal NMC although I haven't even gone as far as looking at that and just with Fedora right and with all the Tegra K4 the Tegra K1 series and newer chipsets so there's basically two devices I think it's a Tegra K1 Tegra 124 there's just a few other numbers that I don't remember but I think if anything larger than the 124 shipped with the Mubo with the decent graphics or celery it's now fully supported in the Mubo driver and it's a firmware box that you need now up to 21X firmware and there is a patch series that's even the VRA next and that's it and that should give us fully accelerated known shell on a Tegra K1 yeah yeah and a lot of people when you're putting yourself up it's just now a matter of working out all that little bit of road to hell allowing you to call in engine stuff and getting that process to a stage where we know it works and we can do it reliably because hitting control in the short term while we provide people to tell us what if there's a command that we can run in and forget what Google calls it as a call boot or whatever they punish this as a compiler and how it works anyway it will get other than control our operation when we boot you should get an extremely usable really nice with our analysis and it's really fun to keep on finding out whether I'm on a Chromebox so really and the thing is the demo report which is a Tegra K132 which is a day out 64 is pretty compatible with the K1124 32 bit so in theory once like in theory it's possible to take a standard for a manufacturer to take their current shipping wire from Chromebox and talk out 32 bit ship for a 64 bit ship and that wouldn't be more awesome but I've not seen one of those on market yet I've not seen any announcement about but there's no reason why it couldn't happen yet one thing that just occurred to me is that these vendors aren't having some recovery tool and I haven't looked at that particular system but if there is some kind of downloaded recovery tool that's going to be the thing that might never want magic address code to reset for Google in the nose of the Tegra the recovery tool is Tegra RCM it's fully open source and packaged through Fedora but they also think you're just being recovering making some stuff like that they don't have anything else to make this some long the issue is using the what is it trust the platform trust the execution ground no trust in the technology anyway basically the thing that the vendors want to enforce the ERM stuff the trusted thermo yeah that's what I meant the trusted thermo is mainly on those boards and so they're essentially locked down a mobile platform yeah but and I mean like the the excellent way of dealing with that the unlocking of that lock down is the whole take out the board and doing some things to it and the video was nice enough that they enabled this area rival flash to boot the legacy it's exactly the same the boot loader stuff the table arm for our books is exactly the same as on the x86 ones it uses the same depth charge payload and it has the same rewrite legacy area on the x86 the rewrite legacy they put a C bias bias in there so that you can just go boot the next distro but they left that completely empty and I mean it's enabled quite sorry yes I have one more question are you comfortable or are you looking for other complex space there and just I'm sure in your analysis of memory references are there any other things other than I have to know the capability of personally doing that how do you change it from what we mean we can do we can land in my office for a couple hours that's all that's all nice work Dennis, thank you very much