 Um, I'm here to introduce me help proc up. He's gonna talk about state of Debian based Linux live systems in 2010 so welcome Hi, thanks for visiting my talk. I'll give a brief overview from the upper part of the Debian System into the city of of live systems Just a bit about myself. I'm even develop our project lead of grimmer GRML Debian based life system for system administrators. I'm maintaining or The lower parts for booting stuff and I'd like to know which people of you already use Debian life How many of you don't know what's Debian life yet? Just two or three people. Oh great. Um, Um Where are we flying to that? I'd like to start with um, my motivation about this talk at all We have 50% of all the existing life systems are based on Debian, which is a pretty good count and More than 70% of them provide a life system or even Provide just a life system and no hard disk installation overall. I think life systems are pretty important Nowadays because users want to test life systems on their hardware. How well does the hardware support work? Is are there any problems they want to test before installing them and I think we should Think more about Distributing a life system as a base for each release of us. I'll talk about this later So some bits about the history of life systems Back in the early days of life systems Whereas this is something like five to ten years ago the main problems of life systems were hardware recognition Which drivers are present what modules do you have to load what when you want to hot plug a device? Nowadays we have you deaf and the kernel and they do pretty much all the stuff we need and Life system developers can concentrate on all the stuff. That's really important for the life system itself So the life system build stuff the software selection special mechanism for all the inner parts of it But too many options to choose from At each single step when built within the life system. There are many Options you might choose from but just a few of them really work So I'll start with the bootloaders with to the first one is crap I think all of you know this bootloader for your hard disk installations The nice thing about crap is that it supports several fire systems And the easel loop back feature is great. I'll show it later on But it lacks proper documentation That documentation is distributed all around the web There is some more recent documentation there's auto outdated documentation invalid documentation and within the project that documentation is Distributed within a wiki and Sadly no relevant distribution uses it for CD boot yet as far as I know you want to plans to Use crap for their next release as default for the life system And I think that's something you should be aware of if you're building a life system using crap as a bootloader And there might be issues and that aren't Known to developers or you yet Then we have the easel linux and sus linux suite which is rocking solid I've had no real issues Developing the life system with it for the last six years and the upstream is very cooperative So we have if you have any issues You will get debug versions and help resolving this issue But just limited fire system support So you don't get all the features you might get from grab and the graphical boot isn't fun So if you ever saw the very graphical menus from the larger distributions, that's a highly customized Easy linux version, which is absolutely no fun to remaster or to adjust to your needs The easel linux stuff is for booting the CD. So ISO 9,660 fire systems Whereas the sus linux part is responsible for putting USB devices with that fire system Ways to boot the life system The most common ones are pixie boot USB boot and of course CD and DVD I was talking about this easel feature and it's a nice method to integrate Life systems into hard disk installations This is an example for grab 2. So we just create a menu entry Call the loop back command assign it to an ISO which You can later on use in the kernel and the unit RDA options So you don't have to extract anything from the easel, but instead you can select the kernel You can select the unit RDA and Provide according boot options Find easel is just relevant to search all the devices for the according easel File boot is life is the stuff that's being used in deviant life in all the tools and just further command lines You might want to Use I've documented this in my blog if you're interested to try it out on your own But it's a very decent and nice way to get a life system as long as you have a working hard disk installation So as long as your boot manager works You can just change route into a rescue system without having to plug in a USB device or CD or set up pixipoot now about the tool chain that's that's involved in in the life system We want to distribute much more software than would fit on a usual CD So seven hundred megabyte won't that be that much if you want all the browser and Documenting stuff and the server editors. So you might want to compress them. There are several options available The one that came from knopix Very prominent popular life system was C loop which is out of three. So you don't get C loop Integrated without using the stuff from from knopix GramFS is pretty outdated and no relevant life system uses it anymore and the most Recent one and the most successful one is squash FS squash FS went mainline so it's available within the mainline kernel and There's coming a new version of squash FS LZM a which does just a better Compression about eight to ten percent according to my benchmarks Sadly, it's not yet in mainlines So I have to patch your kernel and your user space tools to get such compression But I think it's worth the effort because you can just distribute 100 200 megabyte more Colin I can Hi Colin CJ Watson. Do you know what the state of Elzadeh may? Is this on? Okay, do you know what the state of Elzadeh may regarding Actually getting into the kernel is the last the last time I heard It was quite difficult because the kernel guys didn't want to have multiple compression methods built into the kernel well, I'll say the may is available itself in the kernel and Philip has just to clean up the his batch. So it was Or I hoped it would reach the 262 635 kernel, but I don't think it will reach before 36 or 37 but I'm pretty pretty sure it it will reach the kernel because he's He has just to clean up and stated. Yeah, well, it's just a Workhead that needs to be done and then for the other part we have Union FS which is method to Provide a live system that looks like a hard disk installation The problem of live systems is that on a CD you can't write just stuff you need a method to overlay all the writing stuff and the First great innovation was Union FS, which is kernel based But used to then pretty unstable Union FS F use is the user space implementation. I never really Try to use because it has some performance issues and it's not that common yet Another Union FS is a rewrite of Union FS by guy from Japan Which did a pretty good job, but Sally. It's not yet part of mainline kernel There's a new version available, which is known as another Union FS version 2 Which works as well, but it's sadly not yet in the mainline kernel For booting a live system. You need a support in your unit RD To search for the according life system itself. So the root file system we were creating with the squash FS tools need to be fined by the unit RD and you have In it RD generate us nowadays in it ram FS tools great cut and yet another M car in it RD The last one is pretty uncommon and not that much used and not widely used great cut is coming I think from the redhead guys and In it ram FS tools and great cut share quite a large amount of code and we are Working together to fix the problems in all the in it are these stuff so on Debian you might use or should use in a ram FS tools and three cut might be Option for you in in some months For the live boot stuff itself you need all the scripts that find the life system and you might have heard about live in a ram FS that's the counterpart to in a drum FS tools and Life boot is the success of it. So it's a rewrite the Debian life project is working on and Currently you can just replace life in a drum FS with the live boot stuff and the rewrite will bring up new features and Cool stuff and Casper is the The code which is used in you won't do and that's the one life in a drum FS is based on so As far as I know you won't do I'm calling please please correct me if I'm not right and Wants to share code with with even life and wants to merge code Yeah, okay, okay great and We have that tool chain for booting itself, but we have to create of course the life system and we have several Would build system where I want to present to my own one is known as grimoire life Together life system is just easy as calling this you specify the Debian suite the architecture and grimoire base is a base class which does all the fancy life system stuff for you you choose a specific Flavor you can of course use your own one and the architecture once more to get a according Colonel version, but being on depth conf I want to Promote even life Just a few words for grimoire life It's based on fire and that's something which is very nice Because fire is a full automated installation tool and provides a very nice approach the class based approach We can just provide scripts in a class named foo With script name or just place a file that should be it ETC foo RC and use the class name and you can just use a base class and pull all your stuff for All the different life systems you want to build based on base classes or further classes so the class based based approach is very nice to Either install Debian of course with using fire or to provide Further scripts for the life system The life build of even life is as simple as in criminal life. It's you just call LH Config it's creates a config tree. You can just adjust the config as configuration bootstrap where you might want to specify the Mirror which is faster than the German one and then call it with root permissions to build the easel itself If you want to build the life system using even life with the divin installer included Daniel and several others other guys from from the life system Just worked on the life system part of the divin installer and as far as I know it works Well during that conf So all you have to do to include the installer is call LH config with the according options There's life helper in as a web front end available where you can just Provide a few essential details Adjust the options according to your needs and the life helper web front end will build an ISO for you So you don't have to install it on your own system Chris lamp developed a very nice Alternative to this approach. You might have heard about SUSE studio which is a web front end with fancy features to create your life system and Nowadays we have Studio dot deviant.net Where you can just register yourself and similar to the previous One you can just create your life system And it was released during that conf So I created a rescue image and you get Build information and according command line that was used for building the easel Now a fancy feature of nowadays life systems is the hybrid feature Hybrid boot provides an option to use the same ISO image with hard disk installation like Grub bootloader as well as a live part So you can just DD an easel to your USB device and you can boot it and no Further fancy stuff needed if you just want to boot a live system from usp All you have to do is called a iso linux Hup hybrid binary which is executed by default on even live nowadays and You can just take the file and did it to your OSP Device and it will boot as well as the same iso will boot from your CD Very nice for nowadays life systems for building is the CDN Debian net mirror Which gives you not just the closest geographically location But also it it's known to be alive and it's up to date So a very nice feature which you might consider for using building your life system So you don't have to adjust the mirror depending on where you are building or or where you are traveling right now For splash boot splash Graphical stuff. Yeah, nowadays solution is the plea mouse approach which Works pretty well you have just to install it and use the splash boot option and I'm pretty sure there might be more Use of it as there were several tools available for these boot splash stuff And nowadays this works pretty well including the KMS stuff So everything great, right? No We have a pretty nasty toolchain and if you want to build your stuff with the lzma compression You have to patch your tools the build system needs to support it. There were several Changes in command line options. You might have two bars Several tools might break that are relevant for live boot and you might not be aware which tool it might be so we Have to deal with quite Break the chest during release cycle of life systems and Guess what? Uncooperative maintainers you might consider a bug report There's a problem. I need to fix it, but nobody cares about it or Reacts in time so you can just continue building your stuff, but luckily we can use the NMU Uploads with the delayed queue. So nowadays we have a pretty good chance to fix all the stuff that's broken ourself And quality assurance Sadly, um, life systems tends to move and we have several software packages on it and If you are distributing more than 2000 software packages like grimmer does you can't just test each Backage on your own. So I think we need some kind of quality assurance We also like from hardware for not so common architecture, so we are good at the Staff users usually have at home, but there aren't that many life systems available for the not so common architectures if you consider this worse an effort you might be Interested in joining an existing team and show up with with the hardware you have because the usual guys working on life systems Just don't have the hard work pool that might be useful for building life systems But they are good news Your dvn and we can fix our stuff and we have a pretty good tool chain in for with regards to developers and and One I want to mention is the pure parts approach for checking the policy and I'm building life systems on daily base and noticed Much better packages as soon as pure parts showed up So might you might not be aware of this this but installing packages and Removing them and checking all this stuff Was much improved as soon as pure parts showed up So I think we should put more effort into this testing suite another good tool we have is lindian because it provides the developer a Option to check his own packages without Installing manually or just checking each single line in within the even directory and Nowadays we even have rejects for very serious errors in the FTP master so if you upload a package that's very Probably to break lindian on FTP masters will auto reject it and lindian improves our pool a lot So what's interesting is that we can do daily builds with the life system So you can just take the more most recent kernel You can take the most recent user space tours from for an unstable and provide your own daily builds I'm aware of of we want to doing this In the grimoire team. We are doing the same and the life at even life team does the same So what's interesting in in this approach is that you can use continuous integration of your own source software So you just upload a deviant package and on the next day you can just test how does it work to get up with all the other software packages, which are not under your control maybe and test them without having to update your own hardware installation or build Virtual machine on your own so you can just take the download the the image and and start testing and For when talking about life systems, I don't want to leave you with all my my Complaints, but I propose some changes in within Debian As a first step, I think we need auto testing for the non German speaking guys auto is is a car in in German language and as Developers or companies test their cars. I'd like to see see a same approach for Debian I'd like to crash in a controlled environment and not when the user gets our images and has to debug it on his own Which part is involved? Is it in a live special tool? Is it in the common Debian stuff or is it maybe in a Debian derivative? Special stuff. So I'd like to crash it in a controlled environment We can just test stuff crash it and know what Why to be fixed and I'd like to verify it without any risks because I don't want to install software on my Productive servers or laptop or whatever. I'm just to see whether it works I'd like to test it without any risk. So I Don't get x starting up again and so on and Groomers started to to work on a key VM test environment where we can boot a system Inject a script that's being executed and report back. What's what might be failed or if it's okay? it was a mainly a Prototype to see what's possible and while working on it. I stumbled upon With together with another grimoire developer. We stumbled upon auto test, which is a fully automated testing suite Controlled or mainly developed by Red Hat and they are doing kernel based tests key VM based tests and several user space tools It provides a nice suite for testing these kind of things and during the Depcon I talked to sec and Colin and You won't do or so has a spec on automated testing and I'd like to I'd welcome if any one of you is Motivated to work in this area because that's the part I'd like to to work within the next few months Colin I can just get I can just give a little more background in that the Ian Jackson did this while back and he took it to DevCon actually at the time and Talked to talk to the number of people about it He's not not operating it anymore. I think that the main thing that this needs is for Yeah, I don't think it needs significant software development except for maybe You know just fixing bedrotten things what it most needs is for somebody to simply operate it So, you know run the service somewhere Deal with filing bugs Deal with things when it goes wrong So I don't think it's I don't think it would be a major project for somebody to take on you're not You're not having to develop fast with software in order to operate auto package test And a problem I I feel for life systems is I'd like to see the package the word for config files Several tools already support inclusion Mechanism mechanism for config files, so you can just use the package dot D directory or just Add an additional include into some config files, but there are several tools out there that doesn't support customization with Additional config files. So I just need to replace some in its scripts and that's stuff I don't want to do without letting or informing the user that I have to do it so what I'd like to see some kind of of Update alternatives or the word Option to replace config files for your own needs Support for config files, that's what I was talking about sorry outdated kernel or lack of essential tools and modules and Sometimes the kernel doesn't provide all the features that are necessary for a live boot and we have to work around this and I'd like to See the live projects being part of the core project itself So the kernel provides always all the features that are necessary for life systems that are essential for life systems And that's several distributions starts to to work together because there are several patches again Mainline flowing around and I'd like to see a common place So we don't have the problem that everyone has to revamp the wheel a Very interesting a second approach interesting sec Mentioned the automated testing and exactly the the constantly usable testing approach And that's what I was thinking of as well because This kind of approach gives us the option to test the Debian system at each single day and I'd like to see rolling releases as part of our daily work on and development as Debian developers and not just wait two years for another stable release and See what might break currently in testing for when as soon as we want to release it So if any one of you isn't aware of this approach yet Please visit this this website and and and read about it And I'd like to see live media as release goal for Debian releases Currently we are releasing and we have at maximum as a short notice in the release notes But I think most people actually wait for life system or live media to check out the new Debian release And I think it should show up at the same time as we release squeeze so if anyone is Interested in in in getting these as release goal, please let us know and Tell the even life people on a mailing list and let's coordinate this this efforts towards the next release. Oh Open office broke up something interesting And It should read as life system So I'd like to to to suggest Debian derivatives, which is a new sub project Which is an excellent idea in my opinion and I'd like to propose such a thing for Debian derivatives life So all that the teams that provide life systems more than 100 life systems are out there And can share their knowledge can share their problems without being implementation specific because I Use my own build system the Debian life Guys use their own build system and I'd like to see Other distributions share their knowledge their problems and where we can might or possibly work together so to repeat The state of life systems I want to mention that we are pretty good at life systems nowadays It was much harder when I started to work on on life systems seven years ago. It was pretty much a mess with dealing with hardware support with building a life system because back those days we used to take Existing distribution and remaster it within a change route Usually manually without being able to reproduce anything nowadays. We have a very nice build systems We have a good amount of existing life systems for all the different tastes and and needs And I think we should work together to words official Life system within Debian to provide users a decent way a nice way to to test a new release I'd like to Points to two meetings. The first one is the Debian life team meeting, which will take Place today evening. Please join us if you're interested in the Life system part of it and the second one is the the cut both session of joy has Everyone interested in in attending or or working in this area. Please join tomorrow in the interschool lab so I think there's some time left and an open for questions Yeah so The there's some real significant challenges and technical problems with the idea of using deep package divert on configuration files Which is the big reasons why we don't do it Particularly well at present and particularly they're the guarantees that you have around configuration files with a user with user modifications Have real problems when you divert them So, I mean I from my perspective I would actually much rather Identify the places where you feel like you need to do that and fix them So I think your biggest one is a nit scripts As it so happened the first talk this morning. We actually had a really great discussion about that particular topic So See at the risk of introducing something controversial That is something upstart would be able to solve for us Is that upstart it will have a different mechanism for being able to override in its script that you will not need to divert it or treat it? oddly Like you have to within its grips So if we can identify the other places where you're running into places where you need to divert config files I would rather fix them than try to figure out how to make deep package divert on config files work and I fully acknowledge your approach, but I see two problems one is in the uncooperative maintainers or the lack of Importance for Debian itself for systems that just want to do it different for example I'm Generating SSH keys just when the user starts the the SSH server and the in its scripts within Debian doesn't support any Way to generate the keys on request and it's a pretty tricky stuff because on hard disk installations you might operate differently and That's the one one thing about it and Sorry for the record, I'm happy to work with you and not as a message And the second the second Part of it is we aren't aware of all the users that are building life systems So we might fix our own stuff, but we aren't aware of the problems that other people do and I I'm I know where several life systems and looked on behind the Scenes and they are doing pretty dirty stuff to get their problems resolved and as soon as we provide them a way How to resolve this issue? We might fix their their hacky stuff as well. I know that the modifications of config files are a problem for for the diversion and I'm not sure whether it's the best approach really But I think we should talk about a way how to resolve this issue without Fixing all the issues we might Come up within in its scripts or any other tools. I mean, I guess I would put it this way I think that The places where you need to divert config files are Technical bugs in packages in that I believe that it at the very least It would be an improvement in the quality of the package if there were some sort of override mechanism or config directory That you would not need to edit a single configuration file and I think that maintainers Who are standing in the way of improving the quality of their packages is a political problem that we can solve? Yeah, hopefully yeah Hi, Henry. I just wanted to say that there's a set of scripts contributed by Brendan Slite in the Debian live community and he in that order tests of my images and You basically he running these scripts you get like a video of your your distribution booting up in a final Png so you can see if the Okay, if the thing worked, so I'm aware of this. I even tried it on my own, but it's not enough for me I don't want to know whether it just boots because that's Something I expect from nowadays live systems, but I'd like to write Tests for each single software package in our pool So software maintainers itself can write as their own unit tests that are executed That report back and do I work as expected too? So the screenshot approach has Sourced the problem or at least partial Source the problem that you get a working system that boots just up, but I Dealing with with bucks in several tools users might not be aware Of yet because they don't use unstable for our life systems for example, but that the break something and that's Nearly on a daily base and that like to see this resolved so so we can just work together and not anyone who Notices a buck is hunting it and might not be aware of another one hunting it because it might be tricky to Locate it even and I'd like to report it as soon as some package breaks or some software with Other tools involved breaks, but definitely worth mentioning. Thanks Okay To pick up on the SSH point and the diversion Thing together the problem one of the problems with diverting config files as if the upstream so the upstream Debian developer then change packages a version that has completely different configuration file and puts a load of scripts in that fixes it you've diverted it fixes the The diverted version that you're not using and everything breaks. So that's not terribly helpful On the SSH front I think that points at a more general problem that we have which it would be nice to have some way of Saying to a system when it's booting that it's actually a new copy of the system So that if you've cloned a disk It works out that it needs to do something about a new system name new IP address new keys a whole bunch of things and Having some extra options to I don't know the init script that says, you know, it's clone time rather than it's start time Might be a way of doing that or it may be a deep package reconfigure option Yeah, so so that because it is quite useful sometimes to just take it a bit for bit copy of a system and then on boot you have to either be a bit careful or it would be nice to be able to say boot as a clone and Sort yourself out and that would be very similar to the booting a live system. All right What would be involved in Using a non linux kernel with the live CDs So it seems to be pretty linux specific at the moment the process Um, I I have never tried anything But I'm aware that the free PSD port might be interesting and that that's Daniel over there might have news for you Hi If you want to port it to another kernel, you only need a layer Union fs or whatever Compression is nice, but you don't need it. So that's almost everything and I'd like to make three points about what you said before first. You said that crap has that Fancy loop mode Have a look at Susan looks. It's much nicer. You can Use memdisk module, then you need only two lines label whatever Colonel memdisk append my fancy ISO and then it puts that Can I respond to that pretty sure? The so yes, they're memdisk works. I think it I think that's That's using bios isn't it? Yeah, and it's so that works provided that you're Operating system pays attention to the bios in any way, but Which I don't think Linux always does But you know, we'd be we'd be happy in the spirit of competitiveness between bootloaders would be happier to integrate that intergrub Second thing was you said that let's make life images official for the next release. We do have official images for Lenny What was the third thing? Well, I think it should be just the official release goal and other other guys and for especially the release managers should support it, right? The third thing was that The buff later on is just a team meeting everyone is welcome, but it's kind of boring I guess if you're not really involved So I think no further questions. Hi, my name is Alex Okay, you mentioned with a package like SSH it assumes it's for you assumed a package like SSH is for It's assuming it's installing on the hard drive if you have a live system That's that's not really meant to use a CD. It's going all into RAM after it loads up What assumptions change on how the live system works? There's much change if you just have a key our system it swallows the does a net boot it swallows the Mixer RAM disk and it starts running what changes with the live setup and well for the for the Live system itself the live system already looks like a hard disk installation So there isn't really a change and you if you put it into RAM Just access to to any block blocks and on the fire system are faster But for me it's also an issue because I don't want to create the SSH key on each boot It just costs time and resources. So I don't want that. That's an approach other guys are taking But I don't want this and I don't have an host key by default shipped with my life system So it's not just that it doesn't look like a life system But I have different needs than people doing hard disk installations Okay, but the question wasn't so much specific to SSH It's just that the assumption for a live CD is is that it's going to access the CD if it's If it's let's say looking for an SSH program We're looking for open office But if you if you just using RAM disk by itself does anything change in How the live system would be designed as to as to Hi, I'm DKG. I'm one of the upstreams for deburf, which is a system that of the kind that you're describing We have packages in the in testing and unstable And it's designed to just build underneath remfs that basically looks identical to a Regular Debian system And you can just drop that in as your remfs and then tell your bootloader. Here's the kernel I know use this monster remfs instead of using whatever else And then you're not relying on any block device So I would I would be really interested in ways that the functionality of deburf could be folded into the Debian live process Because I think for for forensics and for system upgrades and for all kinds of other things It's very useful, especially with the amount of RAM that we have on machines right now It would be very useful to have an all-in-it all-in RAM arrangement Just to be able to manipulate the rest of the machine without reliant being reliant on some underlying file system that you can't remove So if there's a way that we can collaborate on that Yeah, I'd be happy to to participate Well, the key what is stackable and I think there are already existing approaches So we should just talk to each other. Okay. Thanks Here I have a comment from IRC A Martin Zowell says about Phil's comment when you clone you have the problem that the file system has the same UUID So if you at any time later need to mount that a clone file system into Another clone machine your system gets very confused by the same UUIDs UUID of what for the life systems itself? I think that's somewhat off topic for this talk, but we should have a way of dealing with that. So Well, the UUIDs you can adjust them. He's talking about the UUID on the cloned system that you have no interest in that I mentioned Then thanks for flying with me and I wish you a good dinner