 So next talk is Debian CD by Steve McIntyre. You all know him, I guess. Debian CD is part of a release process and a source of a lot of discussion over the years. Are we feature complete? What's missing? What should we be doing in the future? Right. Hey, folks. Again, this is a talk where I want you a lot to join in as well. I'm not just talking at you. If that's what you're hoping for, this is going to be very short. So Debian CD is a large, ugly lump of pill and shell and God knows what code. It seems to work, give or take. We've been producing lots and lots of CDs for many, many years. It's the vast majority of what I do in Debian these days. You might have seen there's been some discussion, literally in the last couple of days, about sizing of CDs. They're just not big enough anymore. We've had longstanding issues for a number of years now, just physically trying to make sure that we get a useful no more KDE desktop onto CD1. We're working on it. Weezy is possibly either going to be the release after we manage to make that work, if not, it'll be the very last one. I'm quite confident to claim that. How many people we've got here? Not a huge number. Out of interest, how many people here use WNCDs for installation? Yes. OK, how many people here use CD images that we create, whether that's on a USB stick, whether it's on a CD or a DVD or anything? OK, cool. Right, we have some users. Awesome. Of you lot, does anyone use CD1 or KDE CD1? Again, either on a USB stick or on a CD or DVD or anything. Hands up. One and a half. Two and a half. Oh, is that another one at the back? Oh. Do you use KDE CD1 or GNOME CD1 at all? The first CD. OK, that's cool. Right, so maybe I'm too worried about this, and actually we should just not bother with them anymore. We'll see. The issue that we have with always with the CDs is Debian is forever growing. Physically trying to get a usable desktop onto CD1 used to be easy. Back in the DIMITISM pass, we fit a minimal GNOME and KDE on the same CD. These days, we're looking at somewhere between one and a half and two CDs to have even a minimal desktop. Do you want to give noodles to the microphone? So I have been paying attention to these discussions on Debian DeVal. The main question I haven't seen answered anywhere is how many people only use CD1? How many people only burn that single CD, never connect to the network, never burn another CD, and therefore would actually be disadvantaged by us not having it anymore? And that's the piece of information. I don't know if that's been discussed somewhere. We've had that kind of discussion offline a few times, and it's very difficult to get an answer. It used to be that we would do conferences and things, and we would do a CD1 giveaway, or we'd sell CD1 or whatever, and we'd expect that that would be useful for people. I keep on being told, being assured, that actually still doing CD-sized media is useful because of people with really bad net connectivity, either because they're in the third world or because they're living in backwards Tennessee, like Joey or whatever, and in those situations, actually, you want to have a single CD that's useful. It's approximately impossible to find out exactly what the numbers are. I wish I knew more. Daniel? Well, microphone. Just to clarify, what's the element of risk if we have an ISO image that's just a bit bigger than a CD? That's like a DVD that doesn't fill the whole DVD, but obviously it's optimized for people to download it, like if we go from 700 up to 900 megabits. Exactly, that's one of the options. The Ubuntu folks have just done that. Their first CD isn't a CD anymore. The Debian Live images that we've been chipping for the last couple of releases are too big to fit on a single CDR. Again, this comes to exactly what the focus should be. I could be quite happily convinced that we do net inst, and then don't bother with CD-sized images. The next one should be DVD. I guess we have some support for that. What do other people think? OK, BDEL reckons the net instance installs too much. Yeah, he's a weirder. Let's not listen to him. Yeah, I was going to say something similar to your proposal. I think with regard to net instance, I like having net instance business card because sometimes if there's network issues during the install or I'd like to sometimes be able to install from the net instance and reboot, but that's a separate side discussion. I think it would also be useful to sort of work on making it easier for people without network to obtain USB thumb drives because even optical drives are becoming scarce on netbooks. And if you are in a situation where you don't have network or you're having a network connectivity issue, like only wireless and no firmware or something, you may want to be able to give away USB sticks at conferences or tell them to third world via vendors or so forth. Yeah, absolutely. So of course, again, since squeeze both the Debian CD, the install images and the live images are hybrid. So you can literally just dump them at least for x86 or I386 and AMD64 that works. For other architectures, I have no freaking idea how to boot off USB. To be honest, I don't really consider that a major issue just yet. One thing I should have said, please someone, or ideally many someones, please take notes in Gobi. We have this up. I'm not taking notes because I'm not trying not to look at the screen. No, that doesn't mean I want rude messages either. So yes, it's a possibility that we could just stop doing CD sized images. The other tweaks that happened post squeeze but is happening regularly with the weekly builds for Weezy is in fact that the DVD sized images, well, they're not quite. The first DVD in the set is deliberately limited down to exactly 4 gigabytes. So that sort of jibby bites. So that will fit on a, I was right first time, 4 gigabytes. So it will fit on a 4 gig USB stick. That being actually probably more common for most people than using a DVDR for it. If we did consider we don't care about CD sized images anymore, it's also entirely feasible that we could then say, let's have a 2 gig first image that you can blow to a DVD or you can put on a 2 gig USB stick. Obviously, we don't want to have a 1, 2, 4, 8, 16, 32 set of options. But I think maybe a 1 gig, maybe a 2 gig. 1 gig right now would cover what we need for KDE and GNOME. So we could carry on with the same setup we have. 2 would possibly fit both. I don't know. I haven't run the numbers. These are all options. Again, let me know what you think. At the back, Mike. Well, I'm from Salvador. I live there. We do sometimes Debian events there. When people come to us and they want a Debian installer, we usually burn a CD because that's the most easy way to install it. And from us, I think it's a necessity to have a CD that includes all the desktop environment for new users and to use the desktop, the Debian desktop more easily. That's a comment. OK. And we've got BDL as well. I mean, the other option is if CD1 isn't big enough for GNOME and KDE, then we could just say you need the first three CDs or something. So there's an interesting question. Does it work better for you for that to be a Debian installer style? Is CD1 or is a live boot CD that you can then install from an even better choice? Well, it worked for us, the Debian installer first, because they're already seeing the desktop in the event. That's OK. So sizes is still an open question. Please follow up notes here. Mail on the Debian CD. Let's Debian develop whatever. Yeah, I've had a play with the live CDs as well recently. I found it quite useful that I've actually sent. I made up some CDs in Europe and posted them out to Australia, to South America, to various people. They all got them going instantly. It was an instant success. They dug out old machines and put them in. So there are use cases for these things, but the question is not so much the technology. I mean, we can do all these things if we really want to. But maybe the question is more about how to get a bigger survey than just the people in this room. Is that where we should devote the effort right now? If you have suggestions on how to get more information out of our users, please mail me. I've tried talking to various forums, Linux user groups, everything already. And unfortunately, the feedback you get is approximately zero. Download statistics are possibly the biggest thing we're missing in Debian, is we can get proportions of downloads from some of our mirrors. But we have very little information to go on for all of the many, many mirrors we don't control. And of course, we don't know how many times a given CD might be used. An individual download could then go to an install fest and do 50 machines, or it might get phoned in the bin and never used once. It's a huge amount of guesswork. Regarding the option of having multiple CD for special use case where people want to install offline, what about having a first CD which is net tensed with the standard package, which is about 250 megabytes, and then a second CD with just GNOME, and that would be one gigabyte almost. Again, it's an option. It's something in the past that we've resisted because of the extra complexity involved. It's easier for us to generate a single CD that does everything. It's also much easier for the users to have put in a single disk and it works. I mean, that's why we started doing DVDs and now Blu-rays as well. I mean, thankfully at the moment, today, the Blu-ray you can fit all of AMD64 or i386 on a single dual layer Blu-ray. Next week, I don't know if that's still going to be the case. We have too much software in Debian, but we know that. Or maybe we don't have enough. It's getting very, very difficult to actually to get it to people. The question about download statistics made me think of an idea. It's in the Debian CD context, so I'm mentioning it here, but it has broader applicability. Could Debian actually possibly consider commissioning some market research? I know we don't want to do things like phoning home by default and similar things out of respect for our users, but it's not disrespectful to use statistically valid professionals doing statistically valid techniques to actually gather data for us in a relatively respectful way. Related to that, we've had discussions about Pop-Con. Every release for the last four that I remember. If we could put something into Pop-Con to say how a system was installed, we could possibly track that. It's all useful information. It doesn't talk about user flexibility. You may actually need to ask questions and words to find out things like, would another option work for you? How important is blah, and so forth? Absolutely. So, we've got lots of CDs. I'm going to move on from that unless there's anything else. Daniel, last question. Yeah, this whole thing of collecting user data in Pop-Con, I mean, it's a two-edged sword. I mean, it can have tremendous benefits and it can be a painful thing for people. I mean, one idea I had this morning at breakfast was that we could gather data from people about their old bug reports. So, when they've got a new version of a package and they've filed a bug report against an old version, then they could be prompted to close the bug report or to provide more feedback. There's a lot of intelligent things that we could do to reduce manual effort in things like the release process or whatever else. Okay, sure. They're all... One looks like the off-topic, so if we leave it there. Yeah, it's off-topic, but the whole idea of collecting data, more data than we're collecting now, whether it's downloads, CD usage, other media, bugs, the whole gamut information. Of course, while not going too far into the big brother state where people are scared and start complaining, I wouldn't be too happy if too much of my system usage was shared with the rest of the world. It's a difficult call. I just wanted to qualify a little bit the comment I made earlier about the problem with NetInstance is that it installs too much. It's actually a Debian installer frustration, not a WNCD frustration. Okay. But we did the biz card size images in an era when there was a physical unit of media called a business card CD. Sure. That was briefly in vogue and popular and all of that. But the thing that I've always wanted that always caused me to go to NetInstance instead of business card was I wanted a unit of stuff, which would allow me to take a cold dead machine and as quickly as possible get it to the point where it's booted, it's on the net, it can run an SSHD and I can install exactly what I want on it. And so things like automatically pulling security updates for the versions of the packages that were on that unit of media are absolutely the wrong thing to do because I may not even want to run the version of Debian that that unit of install media was built on. You just want the system up immediately that you can do useful things with. Exactly. Yeah. And where useful things is defined as I know exactly what I want to install on it. I want to control that, whether it's bringing in all of GNOME 3 and all of its dependencies or whether it's I got five packages I want to install and don't touch anything else, it's sort of immaterial. So one of the things I'd love to see come out of this sort of reevaluation of what are the right media unit sizes and so forth to have is that at least one of them sort of meets that criteria of a standalone object that doesn't need networking to complete its processing at all but the result of using that unit of media is something that can boot and be on a net, run an SSHD and be able to install other software. Okay. That actually does come back to another question that I ask every time we ever do this talk, does anyone here use business card images or anyone on IRC if you can talk? So wonderful. Yeah. Again, the reason I always ask is occasionally people say yes, yes, yes, it's really important for me and yet the business card CDs are also far and away the least stable of our least images because they depend on versions matching with exactly what's on the archive. Yes, I was gonna say the problem I've always had, every time I've tried to use one it hasn't actually worked. Yeah, exactly. So I mean, given that I would be tempted to just to remove that frustration altogether and just not bother supporting them. Okay, fine. If we can minute that then I've got, I've now got people backing me up for this for the first time. Cool, okay. Actually going back slightly reorganizing things. The other issue we have is we have, again, far too many CDs. The build that happened this morning, I believe for AMD64 for Weezy, we are looking at a set of 73 CDs. Clearly no one is ever going to download all of those. If anybody ever does, I want to go and find them and first of all slap them and say, what are you doing? And personally hand them a DVD drive because at least then you get a number of discs that you can count on your fingers and toes. I mean, we have too many DVDs. We're looking at, I think 11 maybe at this point for Weezy as well. This is something that's come up a few times. In the past, we decided already that for the primary, for the main architectures, which are i386 and AMD64 and Source, I treat Source as an architecture and WNCD, it's easier that way, that we do provide every ISO for every possibility. So we have all of those 73 ISO images available for download, for HTTP, BitTorrent. We have all of the DVDs available. We don't do full ISO downloads for Blueways because, frankly, the number of people who are downloading them is so small that I don't care if we make their lives a little bit harder. We do not need another 25 gigabyte image just because. What is the remaining value in actually producing the ISOs while we can just say to users if you really want them, check those there for you? Exactly. So what we've done for the other architectures is we've already switched over to just having, say, we will ship the first three CD ISO images, the first one DVD ISO image, and everything else is Jigdo. If people really, really want the other images, again, they can have the slightly more hassle instead of just a direct click to download for an ISO image. If I ever get back to it, I might finish Jigdo first, which was a fuse-based Jigdo to ISO tool. If we could run that on all the mirrors, that would be really, really cute, although they'd probably then complain about CPU usage rather than disk usage. It's, you know, it's not, there isn't an easy win. So what I'm considering for Weezy is switching over to a reduced set of ISO images. I'm assuming no one here is going to complain about that. Oh, Don, maybe. I mean, I would just appreciate if we continued distributing at least source on DVD, but I definitely don't want CD images. So if the decision was just to have, yeah, if the decision was just to have real DVD ISOs so that if for some reason you couldn't get Jigdo to work or something. Yeah, okay, that's cool. We can do that easily. But CD ISOs beyond, I mean, whatever. CDs for source, actually, yes, true, and not all that useful. Yeah. Okay, so. Yeah, good point. Thanks. So fine, I will change the sets available to match this. If we drop the business cards, we'll reduce, we will produce fewer of the ISOs just because if nothing else, it makes releasing faster as well. And we all like that. Yeah, I mean, except we're now at the stage where we are IO bound producing about half a terabyte worth of images on release day. Now that we got a really, really gorgeous, big fast build machine, Thomas Cren sponsored it. I will name drop them because it's really appreciated. We went from spending basically 24 hours on release day just building CDs down to about six or seven. That was a huge difference. Obviously as time goes on, we keep on producing more and more software in Debbie and so it's gonna creep up. We're actually having a nice big machine to do that on helps. Okay, more general question. Yeah. You've mentioned in the past, using XZ compression instead of. Yes. Is this something that could help us there? Absolutely. Good point. I forgot to mention it in here. XZ is substantially better than GZ for almost everything we have. We've been talking about it already. Ansgar has been looking at specific numbers for it. Using XZ instead of GZ saves about 20% of the whole archive. We're not proposing a total archive rebuild to switch to XZ because I think again, the release team will probably murder us. It's a release goal. Cool. Okay, go do it. Yes. We should switch the default, I believe. The issue with doing it. Simon looked at XZ before, the reason we didn't switch the default instantly to BZip2 was that it was so CPU and memory expensive on small machines to do the unpack is, XZ got the same issues or is it better in that regard? It's better than BZip2, I believe. Compressing is horrendously painfully slow. The decompress is much better. So compression is horribly slow. So I recompressed the entire archive for AMD64 on some 48 core machine and some packages like the Linux kernel debug symbols took half an hour. On the other hand, decompressing them was faster than GZip on my laptop. And the memory usage is also reasonable, I think. It uses only a few megabytes with the setting that EPKG uses. The flip side of switching to XZ is that there is still software out there that doesn't yet support XZ. We've had a constant debate for the last few months about it, specifically Debootstrap, which of course we use in DI, we can trivially change it in DI to support XZ as well. I believe that's already happened. The problem is people don't necessarily want to require XZ if when people are using Debootstrap on non-debian systems. I'm not so convinced by that argument. I'm thinking we should just tell the world, look, we've moved on, keep up. If there are any other arguments for it which make that not possible, I'd like to hear them, but I haven't heard them yet. Neil? So one of the arguments about XZ slash LZMA slash Watt version is often on other systems. I mean, there's fairly modern-ish, like I think OpenSUSE A12 and various things recently, which have issues with exactly which version of XZ Utils are being included, and including the last release of Debian has that issue as well. So depending on where you want to do your Debootstrap, that might be an issue then. So we pick a version that we expect people to have sensibly, yeah. I think that makes sense. So anyway, EFI, I'm not going to mention the SB phrase. Please pretend that that never happened. EFI is something that we do have to do in Debian CD, and really, really before we ship Weezy. It's not gonna be too long before it will be difficult to buy machines that do not have BIOS, and they're gonna be EFI only, I expect. Steve? Yeah, so an additional issue there, which actually following the previous SB discussion we had a hallway talk about was the fact that, in fact, Debian today on XZ6 Max boots well because it has no EFI boot support. In Ubuntu, we've run into a large number of issues with supporting everything on one CD. Fedora has done it, Matthew Garrett's magic hybrid stuff. So if you expect to be able to have a single CD that boots on FE machines of all flavors as well as BIOS systems, there's definitely some work to do to track down what MGG has done and integrate that. Yeah, absolutely. We do need to get EFI working. I presume that's not controversial. Help would be appreciated in terms of testing EFI. I currently don't own any machines that use it. So obviously we can play with the EFI in QMU, but of course the way the world is with BIOS is and EFI is not gonna be any different. Every motherboard is going to be different in terms of its quirks and whatever. So I am going to be starting to push out testing images specifically targeting EFI and I will be mailing the lists about it asking for help. The more testing we can get done on the more different brands of machine, the better. I don't want to be waking up the morning after the release to people complaining, oh my God, Debian's crap, it doesn't boot on my machine. Steve again. I'm inclined to think we probably shouldn't rely entirely on crowd sourcing the testing. Should we put in a requisition with the DPL to get you some or somebody who's going to do the testing on this some EFI hardware rather than just relying on it that somebody in the community will do it because they're. Oh, to be honest, I'm gonna end up buying a machine regardless, don't worry about that. Well, that's one machine and everybody else is different. How many do you need? That's a very good question. It's something I haven't done research into. To be honest, one motherboard from each of the common motherboard vendors, two or three different brands of laptop would probably be useful. You know, and obviously once we know that it works, they don't have to stay around that we can go and use them for other things. So, yeah, if I help, we'll talk about it more. Multi-arch is confusing in Debian CD world in that we already have what we've been calling Multi-arch in Debian CD and we had it years ago. Well, way before these people trying to do it in the archive. In Multi-arch terms in Debian CD, that just means that we will boot on several architectures and we will start the installer for each of those architectures natively. So, the obvious ones are easy, MB64 and I386. We used to have those Multi-arch with PPC until we just ran out of space on the disks and nobody cared about PPC anymore. We used to have the special that Vorlon came up with which was the HP one, so we had a single CD that would boot on Alpha and HPPA and IA64. That required quite a lot of just hackery in the boot sector but hey, it worked. It was a lovely technology demo and I'd love to think that maybe half a dozen people used it. That's what we've been by Multi-arch in Debian CD at the moment. So, one of the more common, one of the most useful images that we've made, people have told me several times and I have to believe them is we have a Multi-arch, AMD64 and I386 and source DVD, which is the perfect DVD if you want to do giveaways at conferences. All of your GPL requirements are met on a single disk and it's a good message to give to people. Here's all the binaries it'll install on your machine and you have all of the information you need to go with it if you ever want to rebuild anything. That's cool, we like that. As Multi-arch, however, we got the beginnings of it in Weezy. It's confusing to use the same name, fine, whatever, we had it first, you lot go and change. Beyond Weezy, one of the things that we've been talking about is having partial architectures or having a CD that doesn't necessarily contain all of I386 and all of AMD64, but for example, you would have your DVD that contains a basic AMD64 system and then would have the I386 equivalents that used to be in IA32 libs specifically. That lot's going to get hard. It's going to get really, really difficult to test. We are going to need support and DI for it. If anyone wants to volunteer to work on that, please do because as we found out in the last talk, and we all know, surely, the DI team is struggling for manpower. There are lots and lots of cute things like this where people can make a big difference for DI. Please dive in. Who else did we get to? And yes, Neil. Just jumping back slightly, one of the comments from IRC about UEFI is that there's some projects potentially around UEFI testbeds, which you can run and emulate stuff. That should do everything you want to find. It'll help. We need to check on real hardware too. The last thing I wanted to mention, finally, finally, finally, it's been a long time coming. I've got some work going right now to start doing regular Debian Live builds on the DS8 hardware, which is Pettison, which is the normal CD build machine. Daniel has done an awesome job in the last few years of developing the Debian Live project. It doesn't necessarily have the time when we do releases to be able to get regular releases of Debian Live out at the same time as we do the install CDs. So it's been far too long, and I apologize. It's just lack of time on my part. We are going to get regular Debian Live builds alongside the install CDs, starting real soon now with the weeklies, and then when we get the Weezy main release, we'll start doing it for the Weezy point releases too. So, DSA have helped some with that. Jimmy is volunteering help as well. Again, testing for those. I'll be pushing out more information soon. Do we have anything else people want to talk about? Or have I scheduled off? I had a question. Are we still somehow producing a CD that has all Debian on it? Or the Blu-ray, two layer? We have the Blu-ray, the dual layer Blu-ray, still for now, you can get a single disk that holds all of AMD64 or all of i386, yes. I don't know, I said, we are really down to like the last 100 meg or so left of that disk. If the release team let any more stuff into Weezy, it won't fit anymore. And do we produce the same for source? We also do the same for source as well, yes. So actually, yes, that is a nice one to be able to give out. Approximately no one is ever gonna burn it because the blank media costs a fortune. But hey, yeah, it's an option. Yeah, but just being able to hand Debian source to anyone. Yeah, I said, if we're not careful though, we'll need a dual layer Blu-ray and a CD. So with the Multi-Arch thing and talking about doing the IA32 Libs equivalent on there, do we know if the existing Multi-Arch CD that's AMD64 plus i386 already includes the right set of packages? I have no idea. Okay, because I think maybe it would be better to try to make sure that that meets our needs rather than doing a whole another sort of image. Possibly, yeah. I mean, what we do at the moment for the Multi-Arch DVD is we treat it like a normal DVD. We try to get all of the desktops on there. We try to get all of the desktops for both architectures. It actually works quite well because there's a lot of overlap in the binary all packages. I don't know exactly how far down it goes. I must admit, I haven't checked the Multi-Arch DVD to make sure that we have all of no one all the KDE on there yet. And we're back to where we started. Anything else? I guess not. Well, thank you everybody for turning up. Hopefully that was useful for you as it was for me. Please expect, yeah, I'm gonna be boring you lots more with emails to Debbie and Develle and cost-posted to Debbie and CD and Debbie and Boot in the next few months as we get closer to Weezy. There will be a lot more for you lot to do as well as me, especially when we start trying to test the FI. Thank you very much.