 Good afternoon everyone. I'm introducing Ben O'Chinks. We'll talk about what's new in the Linux kernel since what's in squeeze Yeah, what's in wheezy and what's missing in Debian. I'll start with the obligatory biography, maybe not the obligatory I spent my day working as a professional software engineer doing that for 15 years and I've been a Debian contributor for the past 10 years and For about five years now, I've been working on the Linux kernel in both of those roles In my day job, I'm maintaining a net driver for a hardware company, Soliflare, which is the sponsor of the conference And I've been working on more core kernel code as necessary A member of the Debian kernel team and currently I'm doing most of the work on the packages in unstable aside from Supporting specific architecture specific ports I'm also maintaining the stable update branch for Linux 3.2 as used in wheezy That's maintained on kernel.org and goes through review upstream So the Linux kernel is released early and often It's released about five times a year before you and that's the major Stable releases and there are updates with bug fixes every week or two Some of the features that turn up in these releases aren't quite ready Either because they haven't been fully debugged or there are some bits still to fill in or You need new user land and that's one of the things I've learned to talk about Today Wheezy as you probably know has the Linux 3.2, which is now a pretty old Conversion being Linux 3.10 So the good news is we now have lots of new features in testing and unstable Relative to wheezy and the bad news is some of them aren't really usable yet So I'm going to go through a number of those features and talk about what's missing what People might be able to do to to fix that the team device driver Is an alternative to bonding bonding is a way of combining two network two or more network links To achieve either greater bandwidth you use them all at once or greater reliability use one of them, but if that goes down because the switch breaks or Something else fails Then you can fall over to fail over to the other link The bonding driver has an awful lot of code. There's a lot of awful code in the kernel That manages this pretty much autonomously The team driver is a re-implementation of that that leaves a lot of the high-level control in in use of land Which we don't have at the moment we have the you can set up one of these devices with the IP command from IP route to But to really get it working you need some new tools which part of the lib team project So there is an open bug to get this working someone has in fact started packaging that if you want to make this work, that's the bug to to look at So there's a new major feature called transcendent memory not to be confused with transactional memory It's it's a kind of abstract storage for memory So there are bits of this in what started we initially added in Linux 3.0, but then More and more pieces have been added Some of them are not in wheezy So this is a kind of extra layer of storage between the page cache which is All the all the files and data that's held in memory and Your disk Inspection to be expected that it'll be faster than writing or reading writing to or reading back from disk But not quite as fast as Simply direct access to memory So where does this where is this storage really where these pages really going if they go into transcendent memory? If you're running a machine under there, they could be stored by the hypervisor in some pool That's kind of just shared between all the virtual machines It could actually still be in local memory but compressed Memory tends to compress actually quite well you might remember Back in the 90s. There were these memory doublers for Windows and this is the I believe a similar sort of idea For the next And you can also have a cluster of machines share their spare Spare memory space with each other. It's called Ramster And that's not really done yet. It's still under development None of these things are enabled in Debian kernels Some of them probably could be but it needs someone to really think about the configuration What's what should be enabled by but before what needs to be left to local configuration? Are there any scripts that need I needed to set this up? so I've got a link there to an article in links weekly news about Which goes from what details about this you want to make it work. Have a look at that Send as proposal to the curtain team So we have new graphics drivers across the the Most of you are probably using one of the I 915 Radeon and Nouveau drivers and those Have gained support for new chips from the three vendors Since we easy, but they're also completely new drivers for several new well several old and new and virtual hardware devices and Well kernel drivers for graphics are good For several reasons partly they're more robust than the next and you don't have this fragile handoff between using text mode and using X graphics and If the XR crashes Then the graphics are not dead on the other hand if your kernel if your graphics driver crashes then the kernel has crashed And your computer machine is completely dead, so it's not all good At least at that point you do get a nice trace back on the screen probably because the the The current driver can switch back to text mode and and do that Another I think important motivation for this is that user mode Drivers aren't really compatible with doing secure boot You can't use a larger not be trusted to access hardware devices directly has to go to the kernel since the Replacement of the X drivers for for various graphics hardware with current drivers Unfortunately the X drivers we have in Debian don't work with this at the moment So if you want to make it work Go and join the extract force package the new drivers or it's okay. It's just new versions of the driver module signing is something that has been seen in Red Hat Enterprise Linux for a while now and It's for them. I believe it's just it's just been a way to Check whether people are using unsupported third-party modules and Then tell them they can't have support anymore But now with the the plan to support secure boot that becomes more important as a means of a security feature So the in mainline Linux you can you can get You can generate a key at build time sign all the modules that are built alongside the kernel and then Either at build time or runtime a boot time that is you can tell the kernel not to load unsigned modules There's a major flaw in this which is that This doesn't leave any room for out of tree modules, how do you get the key the how do you tell the kernel? they should be trusted and That's there's a sudden I've gone to verse about how exactly you do that And Yeah, there's if you actually want to make secure boot work We don't just need module sign. We also need the kernel image signed. We need to sign bootloaders and We probably need to disable some features like Access to dev man, which is how the the X X graphics X Graphics drivers access the hardware and which Can't be allowed because that would undermine secure boot So there's a meeting on Tuesday Where hopefully we can discuss how Whether Debian can make Debian can make its release is bootable with secure boot so the discard feature Some well probably many solid-state disks or flash flash devices Support this discard operation the The way that flash is managed Means You can't simply do random writes and you need The you need a lot need somewhat more capacity on the physical flash than the Operating system and the file system C is being there and The device can work more efficiently The more spec capacity it has So if the file system Tills it that free parts of the disk are free By issuing a discard operation Then it becomes more efficient. You can It can be faster and It's it's well, it's lifetime. It's working lifetime will probably be longer That's also possible to have thin provisioned storage servers Where which pretend to have more capacity than they really do And that's this more or less works because when you set up a But a Set up a server with so much disk space on the sound you're probably not going to use it all and on average If you know that on average your servers are going to use 60% of their of their disk You can maybe allocate in the initially 70% of the of the disk space you pretend that they are giving them that only works again if the If the file system tells the storage server that it's about the free space on the disk So in order for this to work, of course, the hardware need to support it the driver needs to support it The file system needs to support it and any layers of of storage in between them like LVM and raid need to need to make discard work And So I'm mentioning this here because Linux 3.7 finally added support for this in the raid MD raid layer It does need to be explicitly enabled as a file system option and as an option in those storage layers and The devian install isn't doing this by default There are reasons why you might not want to do it by default but moment there isn't even an option to do that So there's an open bug for this if you want to get discard working By default or at least make it easier for people to set up Go and look at that bug see what there is to do so Another interesting feature which is being incrementally implemented is containers In fact, we had containers for Kind of a lightweight virtual machine If you use KVM or Zen then the each virtual machine has its own kernel its own dedicated memory Well, but that can be sort of varied using a balloon driver, but it has an awful lot more dedicated resources Whereas container uses the same kernel for both the host and the virtual machine But everything in the virtual machine has it has limited privileges and limited limited use of the of the physical machines resources And this has been done before and and we had this in Debian with the open vz and next v-server patches The trouble is these are quite intrusive to the kernel. They have to change memory management file system networking the scheduler all of which are Changing fairly quickly in mainline Linux as well. So these projects have had to work very hard to keep updating to the two later versions of the mainline Linux And they've not been able to support every every kernel version and So eventually with we see we had to drop these These patch sets. However This this is all being implemented In mainline Linux now mostly by the same people who did open vz And it's being done in a somewhat more flexible way. It's been done possibly in a more robust way because they're they're talking to the upstream maintainers But it's it's it's slow it's it the development has been fairly slow. I Think it's nearly there now In Linux 3.7 We can now have user namespaces which allow It's allowed to have a root user in a in inside a container That is not the same thing as the root user for the physical for the for the the outer host Which is pretty important No good having containers if if The users in the containers can break out of them unfortunately There are still some flaws with it flaws with the implementation of user namespaces. There was some early security problems with these where you fact you could break out of a namespace and Also, it requires all file systems which deal with user IDs to distinguish between the user IDs in the current container or the current processors container and the the user ID in the Outer machine that every user ID that's used inside container must have a number in the outer machine, but it'll be a different number and so XFS still hasn't been changed to understand this and That seems like that's quite a big job so if you If you've got nothing better to do with your time, you have a lot of spare time and you could Work with the upstream XFS developers To make that work and then we might be able to enable user namespaces and have container support in the next the next dev in release again, so Bcash has just recently been added in Linux 3.10. It's In a way, it's a bit. It has some similarities with transcendent memory, but it's also very different It allows you to use a Fast disk like a solid-state disk as a cache in front of a larger disk. That's not quite as fast and It turns out this is such a good idea that it's been done several times over. There's also dm cache which was in in In mainland Linux and in hearts.io, which is not But all three of these are available in Debian now Bcash needs new user learn tools And in fact, so it has been working on a package of those There's there's a bug number I'm not exactly sure what the status of this is but it might be that just needs a sponsor So if you're interested if it sounds like something interesting to you look at the box see if you can sponsor that package so for PC For cons on on PCs. We're fairly used to having a single image that runs on Pretty much all PCs. There are some differences where we don't remember we on the I-386 flavor we have On the I-386 architecture. We have multiple kernel flavors for older and newer processes But aside from the processor Generation you don't need a different kernel for a Dell machine or an HP machine And unfortunately this hasn't been the case with arm for a long time arm The company Makes these designs for just the processor and they don't standardize things like interrupt controllers memory layouts There isn't even standard Firmware like on the PC you have a BIOS or now UEFI Historically hasn't had that at all or that may be changing in the near future so Every arm kernel image has been had had to be tailored to specific chips or even specific even specific boards Which makes it quite difficult to support a wide range of hardware in in Debian We have to we have to have a different flavor for each one and because we build all kernels We build what we'll all our packages natively So we build all our arm kernels on arm, which isn't isn't a particularly fast Chips even today, they're not they're not really as fast as xx6 processors So we end up with a large number of different kernel flavors and a very long build time It's about two days to build all the army all kernels currently and there's no no With that number of flavors, there's no prospect of being able to add interesting things like doing a real-time arm kernel or Or a container supporting arm kernel and so on Anyway, the good news is this is changing. There's been a lot of work to describe to describe how Different arm machines differ using something called a flattened device tree which says That is a standardized description of all the different All those differences between machines Which the kernel will parse a boot time and then it'll start running the right bits of code for Whichever machine whichever machine it's running on so only the device only the device tree Needs to be needs to differ between machines That's the idea in a way. So currently in the armhf port. We have an arm mp Kernel which is supporting calzada free scale and Marvell Chips and they're more Lately becoming long soon the ti omap chips are sort of supported although apparently Not all of the drivers are working correctly in a multi-platform build but that's In one or two releases that should be that should be good You may also remember a certain person going on and on about the importance of the all-winner chips found in arm-based tablets That's also partly supported you can use You can you can use a serial port and the the ethernet port as yet. There's no there's no Storage drivers, which is a bit of a problem, but there's a little There there there is work upstream to get these drivers included and Supported in a multi-platform kernel. So that's the current sort of right, but we Still need an installer that will the wall knows how to install the bits on these all these various machines And unfortunately that does to is going to still going to need a specific support for specific machines And Similarly with boot the bootloader or Generally, they'll have a a bootloader installed. It might be necessary to which is U-boat, but it might need a second A second stage of bootloader or they might need They might need some configuration at the moment. This isn't easy to do it's possible to do But you really need to be a real enthusiast read up on Details So if you want to make this work, or if you want to make Debian work on new arm systems Talk to the importers talk to the Debian installer team on the Debian boot list And there's another problem, which is the Most of our most of the GPUs on PCs are Now supported by free drivers Intel and AMD actually provide documentation NVIDIA doesn't but has their chips have been quite successfully Reverse engineered and supported by the Nouveau driver. Unfortunately, the same isn't true on arm yet That being said there are there are I think three or four Reverse engineering projects to for the different GPUs that are out there Which have have some success. They're not really ready for production yet but if you want to make Debian on arm, you know, a Usable free desktop environment Then you could have a look and join those reverse engineering projects. So That's that's about it. There are an awful lot more features that Have been introduced but most of those are Drivers and file system features, which pretty much just work. You don't do anything particularly clever or they're already handled by the The The user land package which is already in Debian. So Any questions about any of those or other features that we're interested in? We have 15 minutes for questions so If anyone has a question, please Hi as a known Colonel guy sometimes I need Try to build my own Karen Debian package is is there are any plans to make that easier for people who don't do that all day? So you you build a custom kernel package That is pretty easy already isn't it use make their package. I mean if you download the source and then Or are you talking about building a custom package from the Linux source package? No from from the Sorry from the Debian sources sometimes it gives Very mysterious error message and you don't really know what to do You're talking about you do you start with app get source Linux is that we yeah Yeah, okay. Well, it's if you want to build a custom kernel. It's much easier to do app get install Linux source And then and then you use the make deb package command To build your to build your kernel actually editing the the Debian source package of Linux is It's it's pretty complicated. It's I understand that and that's I don't recommend that you do that So the kernel handbook is the Debian kernel handbook package and the kernel handbook Sorry, kernel dash handbook dot alia dot deb in org explain The sort of recommended ways to build custom kernel. Are there any other questions? You mentioned that there are various tools need to enable the new features Can you also say something about the tools needed to enable the username space features and other namespace features? Are they available upstream and what about the Debian status? Well, I would assume I haven't actually looked at this for a while, but there's There's a package called LXC which is This stands for slinnix something containers And I would assume that that is going to make use of username spaces But I don't know whether it's up to date and raise to use them yet Obviously because we haven't enabled them in the kernel then Due to conflict with XFS then that can't work quite yet any other question Well, thanks Ben and thanks everyone