 This part of the kernel track, but I'm actually it's actually a bit of a lie I'm not going to talk about the kernel at all The kernel is what comes afterwards so I Work for the Intel open source technology center But it this is not an Intel project so don't blame them for for that They also like me to put this up so Cislinix as of right now is a sweet the bootloaders It currently consists of four What I call derivatives This is a bit of a legacy term from when the differences between them was much bigger in terms of code It really comes down to different file system support the original Cislinix in all caps just to be extra confusing is for fat file systems MS-DOS file systems This was originally done for boot floppies Followed that came network booting in In PXC Linux, this is named after the PXC standard, which is the standard for network booting on x86 Followed that was ISO Linux boots from CD ROMs. This is of course from the ISO 9660 file system and most recently ext linux as A general-purpose bootloader loads from Linux ext 2 ext 3 file systems And I promised Ted that I'm gonna get ext 4 in there really soon now Right now this is only Supports the x86 BIOS platform and this dates back from when this was designed for boot floppies size Mattered tremendously A floppy isn't very big. So any overhead not being left for the kernel was a major problem So the solution was to make it as small as possible and that meant assembly language This has since become a liability in terms of support And maintainability, so we're I am very much trying to get as much code out of the assembly as possible We have several sophisticated menu systems some on Monday written by me and Perhaps most important This is Suslynx has been designed with a mod extents It's extensible via a module API It also contains a couple of additional pieces which are useful for some people Memdisk is a disc emulator It loads a chunk of memory As in a kernel and then allows the booting of any operating system that still uses the BIOS or For that matter another boot it can even be used to load another bootloader It's frequently used for diagnostics It contains gpixel Linux, which is a collaboration with the etherboot project whose flagship product is gpixi This is as you prob as you may be able to tell from the name an open source PXE stack with enhanced capabilities This lets us use a bunch of additional net ring protocol in addition to the legacy TFTP Finally and this is a relatively new feature. It contains a small tool called iso hybrid Iso hybrid post processes and iso image It only it only supports iso images that uses that uses iso Linux It it makes them bootable from from what BIOS thinks is a hard drive which includes a USB sticks now a lot of the Things about this Ultimately comes down to the fact that the x86 PC by now is an ancient platform. I Calculated out that the the the PC has existed for 45 percent of the entire Existence of electronic computers so almost half the original IBM 5150 Came out in 1981 and a lot of the BIOS definition was Dates back all the way to that one There were a few more things added in the IBM 80 and the ps2 that are still kind of considered Considered standard this pretty much dates back from when IBM had You know firm firm firm leadership in the in the area Kind of after that no After that We only had floppy and hard disk booting standard anything else required a custom ROM for everything This kind of led to a bit of an untenable situation where you know it seemed like no one could advance the platform and after a few kind of missteps There started to be Industry consensus standards and one of the first ones of those was the CD ROM booting standard called El Torito Supposedly named after the restaurant where the people who came up with the standard were had dinner Came out in 1993 El Torito contains two modes One is you have a disc image on the CD and it just boots as it's been a disc And the other one is so-called native mode where you can actually access the entire CD Unfortunately The native mode didn't work very well in many biases until the late 90s Similarly we had a network booting standard originally developed by Intel Came out in 1997 and revised in 1999 Some of the or again some of the early code were really really problematic there and even today We are seeing a fair number of bugs, but at the very least this brought standard standards based network booting to the X86 platform USB drives is the most recent collection of the to the PC bootable zoo. They just appear as Reg conventional disc drives at least when there aren't bugs and Unfortunately, there are still bugs in a lot of biases. There is however tricks that can help The syslinix wiki and the syslinix mailing list have collected a fair number of tricks that Works seems to work for a lot of people and it's a good place to Get some more information on that So the syslinix project itself Was came out of a overnight hacking binge that I had in 1994 I had just had a miserable experience Reinstalling the machine with Linux where I found out that I needed a special Carnal for for the driver that I had this was before modules were widely used in the Linux world and You know, I had to boot from this floppy which was a completely opaque object and You know, I didn't have another Linux machine handy and So basically I had to give up until I could go somewhere else where there was another Linux machine And I thought this is this really sucks Like this is just wrong, you know So I decided to go out and Write a bootloader that would actually Boot from a floppy that another operating system would recognize So that you could take your the your legacy operating system, you know, there was DOS primarily back then and Actually make your Linux boot floppy even if you have to do custom stuff and it should look like a DOS floppy, you know, and DOS should be able to read it. DOS should be able to write it and hence I You know, it needed to understand the file system that was underneath this was in contrast to Lilo Which was the dominant Linux bootloader at the time, which only which contain only did raw blocks You know didn't actually look at file systems at all so it was kind of okay a lot of Distributions adopted this for for their boot floppy's boot floppy's were pretty ubiquitous that back then out of some features that some people like like on You know to help the distros primarily added online help support in in the bootloader This is kind of a revolutionary idea at the time So kind of move on pics pixie comes along and Okay, we should have a you know We should be able to boot Linux off of pixie. This you know, this this network booting thing is good Well, so I You know look at the pixie spec And it's and the spec says that the network boot program, which is the first stage bootloader in the pixie scheme of things Should only be 32 kilobytes long Like wow, that's pretty restrictive I'm gonna need a pretty small bootloader Wait, I already have a pretty small bootloader. So rather than starting over from scratch Let's take this thing that I already have and beat on it until it does roughly what I need so This worked eventually and Users liked it because you know by this time syslinux had kind of matured enough that It got most of the features that people that people really depended upon in it already It's like suddenly you have another pro, you know, you have another medium. You can use all the same features. This is good So kind of on that general note Like okay, this is this is kind of cool Let's You know So decided to next tackle CD-ROMs and actually be able to boot CD-ROMs in native mode This was different from you know the disk image variety and The and the reason the motivator for it was that the disk images were getting too small They were getting too cramped People were already using syslinux on the disk images But so let's give them access to the whole to the whole disk. This became iso linux Few years later than that, you know a little bit more maturing and so on But you know there's I kind of gotten this whole thing about supporting multiple file systems down reasonably well, you know should actually and It should actually be able to make this so that I can use it as a general boot general purpose bootloader And that was ext linux In 2004 I did apparently did quite a lot of work because that's also where the Modular API and the menu system was first written 2006 a lot of people like yeah, I like this menu system But it's not flashy enough. So eventually like fine fine fine fancy graphics. You got it and in 2008 Gpixie linux and iso linux hybrid support, which is the USB key the iso image that also works in the USB key So what's good about syslinux? The number one thing that syslinux does different than other bootloaders or most other bootloaders Is that it was from the very beginning designed for dynamic systems? It was designed for floppies after all and on a floppy you can't assume that You know you're gonna boot on the same system that you originally created a floppy on But I've tried to keep true to that principle And do system discovery at boot time. I Do I try very very hard to keep syslinux such that You should make be able to make almost any arbitrary Reconfiguration of your system you should be able to pull out half your hard drives rearrange them, you know Put them in put them in another system. Whatever it should still work if at all possible The other the other basic principle I've tried to keep true to is place nice with others. I Tried to stick very closely to the established principles for booting a PC, which is somewhat. They're somewhat arcane and They're somewhat Restricting but for example, I don't support installing into the MBR As some other bootloaders do simply because it makes the job of dealing with other operating systems That do weird stuff harder By now we have a pretty sophisticated user interface and we have again the module or API which I'm gonna talk more about later What's this problems? Well, first of all large it has a large core of assembly code This was this was you this was useful for size, but it really has become a liability Again, we're only it's only able to build it on the x86 BIOS platform And this is pretty much because of the large assembly core Well, the whole thing about playing nice with others and Doing dynamic discovery does come at a price and that is In a lot of bootloaders you can do such things as You know the bootloaders install on one draw on one disc and it will Read a kernel from a file system on it on a different disc. I don't support that Mostly because it's not very clear what that even means in a dynamic system You know you then this drive may be you know this drive may be drive zero and and that drive may be drive two today But there is no guarantee that that's going to be the same the the case when the system is actually booted So There are ways one can work around But I haven't really I have found one in general when I've asked for people while I have this I need this feature And I generally want to ask them well what why do you need it? What what do you you know? Well, I'm doing this and and there's usually another Usually better way of doing what what they're needed to do now I'm fully willing to be convinced that this is you know, I'm have to narrow of view, but This has turned out to be the case so far I want to talk a little bit about gpixel Linux here because it's it's a It's a little bit of a segue, but it's kind of a it's kind of a cool thing So this is gpixie comes from the etherboot project It is gpixie and and we and we have then taken it and put it together with pixie Linux into one image What now gpixie? Provides an extended pixie interface which is specifically designed for pixie Linux So pixie Linux has some special bits for gpixie and gpixie has some special bits for pixie Linux You put these together you can boot over then, you know over a variety of network protocols Currently we're up to HTTP FTP NFS ATA over Ethernet iSCSI FSP and I think there are like two or three other protocols which are just too obscure for me to remember right now The goal is for this is that you should be able to The goal for this was that you should be able to swap in and out Pixie Linux in an existing configuration that have it just work We are about 95% there there is one semi-obscure feature of pixie Linux that is not supported in gpixie Linux yet But it works for the vast majority of users This lets you do things like well, I want to dynamically generate configuration files Well, I can just do it what I can just put it up on Apache and you know do a CGI script I know how to do that If you're doing it this way, you need you still need a TFTP server for the initial bootstrap This is because the the pick the the pixie stack that came on your on your card only supports TFTP Now if you are willing to reflash your Nick and put gpixie on your Nick Then you don't even need that so I wanted to show a little a little demo this and There's no safety net for this. I am going to boot a virtual machine over Over the internet from a server in California and the only thing that is kind of local on this is this This is the CD ROM image that has gpixie on it The actual gpixie image is about 64 kilobytes If you put it in a ROM, it would be a lot smaller than this. So this is gpixie We'll start up do usual discovery Well, it just downloaded pixie linux from California It's hunting for a configuration file This takes it this takes a little bit Yeah This is the actor. This is the syslinux menu system you can have sub menus and You know graphical backgrounds and all that sort of stuff In this case, let's try to boot Tom's boot disk Again, this is over this is from this is over HTTP from a web server in California No, it's just booting. I'm good. And there we are now I mentioned syslinux has this module API, which is designed to be very flexible this is actually sort of this is Kind of a good example on how interesting things come from unexpected places This is actually done because someone's because a former colleague of mine said can't you You know can't you support booting DOS style com Images and I'm like well, they're really easy to support. I I'm you know might as well You know, why do you need it? Well, you know this I have I have this This network bootloader called net boot which was the predecessor of the ether boot project That comes as a dot com file. Okay, fine This was kind of sitting in there for this is sitting in syslinux for quite a long time and not really being used at some point I said well, you know Yeah, this is 16 bit com stuff, but You know, we should really have 32 something 32 bit in there as well. So it came up with com 32 I Eventually kind of said well, let's make it easier to write one of these modules and maybe someone will actually use it So I had already written a small C library that was intended to be bundled with a Linux kernel called Caleb C So I ported it to this other system call interface com 32 So what you end up with is that you have a programming environment, which is basically standard C environment Most of the stuff you expect the standard C environment to have is there The biggest thing is that you is that when you open a file You can only do so read only and you can only read it sequentially. There's no support for seats this turns out to be a Pretty small problem for for most users you you can live with this and The interesting part of it is that these modules don't even need to know Whether or not they're booting from a you know, you're reading these files from a CD from you know A hard drive or from a network connections from across the world So the most common types of modules that people have written and this is sort of it You know, this is a rough taxonomy and doesn't fit everything, but It's a good example what people can and do do Now first of all, you obviously have user interfaces the menu system that you already saw is Implemented as an entirely as a module We have file format modules. This is ways to support loading new types of kernels of various sorts Policy modules, which are this is what I want to do in this situation so that the user doesn't have to do it for me and Diagnostic modules Hardware diagnostics mostly user interfaces Right now there are In in bundle with assist Linux distribution, there are there are two different menu systems The very first one was the complex menu system written by Morali Ganapati While he was at the University of Chicago this thing does everything I have some screenshots on it over on my website It will it does things like you can have cascading sub menus and dialogue boxes Where setting a dialogue box here changes the command line in a completely different menu And you have a little status bar at the at the bottom of the screen showing you what you're currently doing Trust me that yeah like for a large site that just needs this enormous flexibility as they apparently did at University of Chicago It's absolutely unbeatable The simple menu system Came later. This was after people was people were saying well Yeah, this menu system that you have is it's really cool, but it's way too much work to configure For most people it really is so the simple menu system just takes a simple file or set of files and You know pretty straightforward just low, you know presents a menu and Not without too much fanciness There's a graphics library in here Which is designed to make the same code work for for either a graphical console a text console or a zero console the code in the menu system that you saw is The same for all three types of consoles in fact if The graphics line if that menu system had booted and found that we didn't have a Graphics card that it could use it would just have defaulted back to text mode Similarly if it had found that it had a zero console it would just have used the zero console In fact, the serial console works at the same You know it works in parallel with the display console so that you can either be a human being touching the physical console or you can be a You know on a remote terminal server somewhere else Now obviously if you're using graphics features like you're using little icons for example well and You're bypassing this sort of console library. Well, there's not much I can do about that But this is of course allowed if you want it file format modules are a way to you know It's a module that you that implements a new type of loadable object Again the goal has been to make these easy to write so In order to do that the module Describes where in memory things should be going You don't actually as the module author have to do the work of actually putting it into the proper place You just have to tell the rest of the library Where things should be going and then there is this thing called the shuffle library the shuffle library Looks at where things are now Where things need to be and what scrap memory it has to play with and Computes a set of move operations That will put that will end up with everything in the right place at the end An example on this is the Microsoft system deployment image format I had never heard of this until someone asked me. Hey, I want to boot winpe with syslinux It's like, okay. I don't know the first thing about winpe Turns out that they have this this thing called sdi, which is roughly the equivalent of Roughly the equivalent of having a linux kernel and a RAM disk It's kind of basically a windows kernel and a RAM disk Supporting this format took a module that was 199 lines of C code 139 lines of those were were actual lines that actually did something including lines that has single curly braces on them most of that is taken up by error handling meaning The you know, look at this header field, you know, this is not the right type of header field, you know, this If this problem then error this if that problem then error that if this problem error that that's actually You know the end the the resulting code is is is very small diagnostic modules are Modules to tell you about hard about your hardware the One of the first ones was PCI dump or written by R1 Here Now what we have done is we're taking a lot of that dumping the actual code that gets the state out and Moved it into the library. So it's available to other modules This both lets us build better diagnostic tools in fact some I Wish I could find it again because someone showed me this this kind of hardware Inventory thing that's actually been written on top of this linux, and then I lost track of where I found it because I hadn't even Because I hadn't seen it before but It's some it was kind of neat where you could actually get like cascading windows of Describing what your hardware looked like be at pre-boot and Finally we have policy modules where an example of policy is Boot boot this kernel if I'm on a 64-bit machine Boot this other kernel if I'm on a 32-bit machine that has PAE this presumably high mem kernel and Otherwise boot yet another kernel There's an example of policy you can tell the user to pick different menu items, but why should you have to? This is stuff that can be done automatically. It should be doable. This should be done automatically So and a module to implement this particular policy was 129 lies wrong 70 of which were something other than blank lines or comments Most of most of that code was taken out by picking apart the command line and putting it back together again And this is because if if these things had been fixed compile-time constants that wouldn't have been necessary, but this was meant to be a general purpose module and Sure, you can you can combine things into into modules that do all kinds of strange things We have code right now that can that can probe your PCI bus Map those devices to the corresponding modules in the Linux kernel Then build you an init ram FS with only those mod with those modules and no other modules and do all of this At boot time, you know for the system that you're booting on This is already there already working code but Soon as you have that You you also you also You know discover that there's always going to be limitations right now, for example You have you're starting to see a lot more devices on USB Even on fire wire and so on and these sort of extended buses are harder to discover because You need a driver for the bus itself For the HPA of the bus in order to be able to further enumerate it so The this technique has been less useful than I was hoping it was going to be but it's it is there and You know, maybe someone will figure out the right way to use it. So what's happening right now? Where we're working on integrating a Lua interpreter and the purpose of this is primarily to write policies People really don't like having to compile a C module to do a new policy and I and that's very understandable So doing it as a Lua script seems like a much more useful Thing why do we pick Lua? Well, it had a clean relative, you know relatively small sort of Isolated interpreter that was and it's a you know clean small language a Reader support this right now syslinix doesn't support actually examining directories This is partly because over the network you can't do that but On this you would be nice to do that. So that's work underway The really big one is to is to get rid of all this legacy assembly code First of all the first thing that needs to happen is that file system code just needs to move out of assembly the Implementing something like btrfs In assembly would be just a nightmare But even getting rid of the rest of the code from assembly Would it's going to be important to make this ever port beyond the exodus 6 bios platform in particular? for EFI exodus 6 EFI which is becoming a more important platform This needs this needs to happen and at this point a lot some of the groundwork is happening But there's a but the actual work is is still upcoming This is what's the core looks like right? This is the components of the core as of right now the first stage loader disk and network IO and The the BIOS extender, which is what allows us to get into protected mode and The shuffle system that is the part that makes all the memory magically end up where it should be at the end all of these are Kind of core platform support things and they will always need to be in assembly if nothing else for size however, the rest command line interface config file parser the kernel the kernel loader and parser and More most important of all the file system drivers really have no business doing this and it needs to be rewritten See this is about 80 to 85 percent of the size of the current core So the part on the left That is always gonna have to be assembly code is really a very small portion So why hasn't a lot of this happened before well? It really kind of comes down to the fact that it is getting to be too large for it to be a one-person side project Which it really was until a couple of years ago now We really over the last few years I am absolutely delighted how many people have kind of come in and We're starting to form a real development community and we can always use more developers and In addition to actually writing code Things like help helping newbies. There are so many things that can be done in the booting space in so many ways and just and honestly, I Have to say if there's anything in syslinux. I'm ashamed of it's the documentation the state of documentation is just miserable But again doing the core rewrite it's gonna it's gonna take a lot of work and So it really You know in in addition to me that it really you know we need other people now Also all of this really only makes sense if the users want it You know we can build the coolest piece technology in the world and if no one uses it it doesn't matter So we need to make sure that we end up building something that people actually want to use That makes sense for users make sense for administrators make sense for distros so Feedback is always welcome and this is where you can find the more information as well as these slides and at that I I wonder if we have any questions Yes, I'm sorry So the question was whether or not we can check a cryptographic hash of what what we download off the internet Right now there is support in in Gpixie for HTTPS The problem is that at boot time you don't have You don't have any sort of random number generator available to you that this means that The security you get is pretty weak Because most authentication schemes rely on having a random number generator available Could we do something? You're specifically asked if we could check a hash of what what is there? It's certainly something that we could do relatively easily But you also have to ask the question then where do you get the the cash to compare it from to This is a lot. There are a lot of issues when it comes to the when it comes to that in the booting space is You you people really want to make the boot more sick more secure But then you'll kind of have to ask the next question is how do I know my security works? I Think I think we may be able to to make the HTTPS support better It it hasn't been a huge priority for for for the Gpixie people at the moment, but We probably we probably can make that better if not perfect Why why don't we use the harder TPM environment to do that? That's actually probably the answer right now right now TPM's are They're They're availability is somewhat spotty, but that is of course changing so the TPM if nothing else should be a should be a if we can find the TPM and Straighten out the fact that there is at least four different hardware protocols involved At least they should provide us with a reliable random number generator and possibly secure storage for things like keys and stuff like that so It's it's it's actually the right way to do that the question. I believe is do we support IPv6? Is that what I heard? Okay, so the question is do we support IPv6 or or or only IPv4? Right now we only support IPv4 IP IPv6 is not supported by the pixie standard the G pixie people are working on it, but we end up with a funny notion of How do we you know, what does it mean for us to support IPv6? Given the fact that the underlying protocols we use don't for the parts of the pixie spec that are That are a in you know the on the local machine Sort of protocols we can we we can deal with that, but but it also includes some over the over the wire components There is a pixie spec for IP for IPv6 under development Unfortunately, Microsoft is heavily involved in it and therefore I have not You know You know, it's there. They're basically keeping it away from the open source people So I have no idea what what it even is going to look like and To be honest with you if it ends up looking as bad as the original Pixie spec. I don't have you know, it's gonna be another nightmare So the question was if I can talk more about building an initial RAM disk So the question is why do you get modules? Are you allowed to use DCC? No, we don't have DCC ported to the system in DCC, I'm sorry. Oh DJ DCC No, okay Okay, I don't believe I'm familiar with it Okay, yeah, it's a DCC thing The answer Well, the answer on the module front is that the normal thing is that the modules will actually be already compiled on disk I Could you I mean, I'm sure that we probably actually could build an environment where you could compile a module on the fly It's you know, there's no technical reason why it's not possible Yeah I Think it's probably not it's probably not the most effective way of doing that but But we can as I said, we can take files we can generate file We can take files and put them in the net ramfs. We can Also generate file content and put them in the net ramfs So all all the pieces are there certainly the question is if there's support for boot boot once. Yes, there is right now, it's only in ext Linux, but that's That's a fixable problem. Yeah any more questions. All right, then I think we're done