 How many people know a little bit about M-Debun? or at least know what we've been doing? Yes, I know Nick does. So I've got the main picture up there trying to give some idea of the flavors of the Russian Albanian Mule. The names are based on squeeze but we've kept the names. a we're going to keep these names then for future releases alongside Debian. So GRIP is the lighter touch. We keep the binary compatibility, but we take out about 40% of the actual files within packages. We don't recompile the packages if we don't change them at all. It just means that you get a smaller basic system. The, okay, that's better. So that diagram and the information you need from that is all on the Mdebyn website. The main thing about this talk is going to be based on a GOBI document. So if you've got a chance to load up that at gobi.debian.org, we're going to be going through some of the expectations of what people have. I'm not going to spend most of the time actually talking from here. I want to find out what people expect from Mdebyn, what we can do for them, what we can actually provide already, where you want us to go from here on. So if we go through just the first part of the brief summary of GRIP, it's a question of also what we mean by embedded. We don't just mean another smartphone, another tablet. A big use case for particularly Mdebyn GRIP and one of the main reasons for trying to push it to go into the main Mdebyn archive is these custom media devices that have a single purpose. And the actual system itself is hidden away from the user. You don't offer the main Mdebyn UI to the user at all. You have a completely all-inclusive system that does all the power management, it does all the suspend and the main operations the user wants. But it doesn't actually offer any of the underlying or parts of the system, which means that you can get rid of all the interactivity of the base system and just let it do its main job. You don't need the man pages for deep package and that sort of thing. You'll never see the command line as a user. So you get this whole idea of a locked-in system. It can be, in our case, a communication aid. It could be a unit controlling a production line in a factory. It could be all these kinds of static systems. Question? If you've got a question at any time, just jump in. So if someone's got the Gobi document open, if you can actually start making notes particularly of questions that come up and the answers, yeah. I just was wondering if you could talk a little bit about where embeddion might be being used now. OK, particularly with embeddion grip, we use it in devices like communication aids. So we have a system that just makes sure that we can provide enough space on the device. The actual available space is very limited. If we have to actually put another chip on the board, we have to take something off. So we need to make the embeddion system smaller, but still suitable for what we need to do. Are there any commercial uses of it? That is a commercial product. Three commercial products already based on this in the market. There's also working on, there's a pod point using embeddion. So there's the electrical car recharging systems again. That's a static unit, minimal interactivity with the user, but it's just there to make sure it runs. It keeps on going. It logs things that are going on. It keeps in contact with its base station and it just manages that one little pod in combination with another one. So you have got these fixed installations, minimal interactivity, full functionality. But the key point is that the software on that individual unit is binary compatible with what the developers are using back in house to actually write the software and debug it and get all the problems out. You've got this match all the way down the stack. So I was trying to cover that in the first part of the gobby document there. And again, in our use case, we do have a limited amount of space on the board. We want to try and pack in as much usable data that the user can actually access rather than padding it out with the Debian system we want for our own reasons. But we want to have as many of these big voice synthesizer files that we have to use. We want to get as many of them onto the devices we can without sacrificing actual physical space on the circuit board. So Grip has been going now since Lenny. We made a parallel release with Lenny. We've made a release alongside Squeeze. We are in line to make another release exactly parallel with Weezy when Weezy comes out. So we've got this track record going back through that. And we've managed to now persuade teams like WannaBuild, FTP Master, DSA release team that we have a sufficient backing within the community, within Debian, within the users that we can actually push to have our packages in parallel across the main archive. It'll make it easier for commercial companies to see the packages and get used to and use them. And it'll get us more feedback about whether we're missing a few packages, whether we need to take some packages out of our selection on all those kinds of processes. So I've been working with, particularly with FTP Master recently, trying to work out exactly what we need in DAC to allow us to have a parallel suite. So we'll have something like SidGrip, WeezyGrip. They are binary only upload suites. And the DAC software will actually allow the unchanged sources to be copied across or displayed within the sources GZ as they were in the main archive. So we're running that through a DSA managed build box. It gets notification directly from DAC. It builds the updated version, puts it straight back into the archive. So within a very short space of time, we can process the daily churn within Unstable for seven architectures. The actual processing for Grip is very fast. All we're doing is de-package unpack, remove some files, de-package repack. So there's a lot that we can actually do with that. It's architecture neutral. We can run the server on any architectures. We have a nice fast AMD64 box that's doing the work. We have a local mirror mounted on there. So the whole thing just churns through very quickly. And thanks to Steve, we are expecting CD image support as well. We had to first go with it with squeeze. There were a few problems we picked up. And so we've tried to build those into the processing from then on. And we're looking to make another go-through with CD images with Weezy. Now, if there's any questions on what we're trying to do with the integration, what it's going to mean for you as maintainers, because you're going to have your package listed now in two suites. You'll upload the SID, and hopefully within the day, you'll have your package once it shows up on the buildees. It'll show up on our buildee, and it'll be uploaded to SID Grip. We'll give it a version suffix so you can tell it's actually been changed as a binary. It'll end up with whatever your version is suffix with EM1, but the source is completely unchanged. Your signature on the DSC will be valid on the signature in the sources that you download from SID Grip, and the release team will be working on allowing the package to migrate from SID Grip into Weezy Grip at the same time as it moves from SID into Weezy. It'll be Weezy plus one by that point. We were hoping to actually have a lot of this integration in place by then. So, what are the questions on what it means as a maintainer to have this parallel versioning in the archive? Hector? Yeah, if I have a Mdebian system and then I find some bugs, should I file the bugs against the devian packages or currently we are using buildee.mdebian.org as a pseudo package? Or how far this integration is going to the full devian? In the initial stages, particularly when we're putting into the main archive for the first, probably for the first stable release, we are working with report bug maintainers to allow that when report bug picks up through your reporting against the version that is EM1, the first place it'll go is buildee.mdebian.org because although policy says that no packages should misbehave if their user shared doc files are removed and their copyright file is compressed and various other changes that we make to reduce the size of the doc dev, then we're just being cautious making sure that there's no knock-on effects from those. We haven't seen any knock-on effects from the packages we've been looking at and we do have a limited selection of packages. We only take 10% of the archive currently. Will there be tools available to do these procedures with derivatives? If derivatives want... EMdebian is a derivative of a kind already but if derivatives want to take our packages on to another level that is fine, that's just going to work through... Paul, do you think there's going to be any... particular call on that? Have you heard anybody? I haven't heard any derivatives interested in the sort of thing, no? Okay. So when you say that you're going to have bugs filed against EMdebian, is there anything planned for changing that in the future? Because I think if EMdebian is indeed going to be part of EMdebian, much as sports are part of EMdebian, maintenance are responsible for making sure that their packages work in every situation in EMdebian. If EMdebian is part of EMdebian, then that is also part of their responsibility. So maybe in the short term that might be a good thing, but you have plans in the long term to change that. Long term, yes. Long term would be the goal to actually move. For example, one of those, we have a few little bugs in Dash for one which seems to insist that Usage M1 exists before it runs its pre-compiled binary maintenance script. So there are lots of little bugs like that that we have to work around currently. So we're just making sure that those kinds of things can get ironed out between a normal release cycle. There's a minor annoyance with XOROG. I think it's the XOROG core package. It expects a file to exist in Valib X11. And little things like that. It's just making sure that we've got time to push those through the system. But yes, the aim would be then you can just have a quick check through the MD5 sums because the check sums have not changed on anything you've compiled. So you can just say, well, that is the same check sum. Therefore, it's likely to be a bug in the binary itself. And then get that reassigned. Nick, did you have a? If I want, I'll just clarify as well how your packages get into MD5 and GRIP. Core packages, yes, we have to have them in there. So anything that is priority required, important like that, the majority of those will end up being ported into MD5 and GRIP. But the optional and extra packages, they'll be brought in if they are asked for, if they're not or they're ready. We're looking for packages that have some kind of embedded purpose. So we tend to go for the smaller implementations of a particular protocol we tend to go for. If we go for a graphical desktop, it'll be LXDE, XFC. We'll pull in little bits from here and there as people ask for them. But it'll be the libraries that the maintainers look after. They'll be brought in as automatic dependency checks via the building as we do it. So if you're maintaining a library, you may suddenly notice that it's been brought in because an application we've had in for some time has gained that dependency through some other means. So I know that the primary focus has been on trying to reduce the on-disk footprint of the installed packages. One of the things I've run into a couple of times again recently playing around with very small systems with constrained RAM in particular is how badly some of Debian's core tools behave in small memory, small RAM systems these days. Has there been any attention paid to that if you thought about it at all? I mean, obviously, if you can deliver a subset of all packages in Debian represented by different packages file, you immediately relieve a lot of the pressure that DPKG and apt cause when you try to use them on a really small system. But I was just wondering if you or anyone else here has given much thought recently to the executing memory footprint of core Debian tools important to maintain a .deb based system. The only real way you can deal with memory footprint is to change the binary functionality of that particular package. So that isn't going to be a grip. But we are trying once we get multi-arch cross-building back in place and we can actually have another building processing these kinds of uploads on our regular basis cross-building a number of packages. Then one of the things we're looking at there is dropping big dependency chains turning off a lot of functions in core libraries. So the default in Debian is that if it works you enable that switch in the configure options. The default with mDebian crush will be do we need it? Do we want it? Can we get rid of it? And will the system still work if we turn it off? So we turn that on its head and that should deliver at least in the packages that have that kind of support already upstream. The difficulty with that and the bugs that come out of that is that that particular combination of options is probably going to be only activated in our distribution. Most testing of these kinds of upstreams will be in the Debian model to turn everything on until it breaks the actual limited role of, oh that's not there anymore and getting strange breakage at the later point. I was going to say in addition to that it also could cause weird dependency problems if you've got something that assumes functionality. So what part of the crush process will have to be right? Now we've changed that little library. We've got to rebuild everything that reverse depends on it and get them all to look for the new symbols. One of the advantages of having a reduced package set available in GRIP is that the package list become much smaller. The package list become much smaller on device and so when you run out for example you don't find that you run out of memory just when processing them. Also if anyone's worried that you haven't got the full access to the repository you can of course just mix it with the full Debian suites as well so you can have that one package that hasn't actually been involved in GRIP. When you mix Mdebian and Debian there are a couple of things you want to think about. Mdebian versions suffixes mean that the Mdebian package is always a newer version. So but there is a delay particularly if you are using one of the main architectures AMD64 or i386 which maintainers upload as binaries. If you have source only uploads this will change but we have to wait for the package to arrive off the build ease and onto the local mirror before we get a chance to process that package and put it back into the archive and then there's a wait for the next de-install before it appears on the mirror. So you will get these particularly in an unstable you will get these jumps where if you update regularly you'll have the Debian package suddenly appearing and then at the next update being replaced by the Mdebian package and that can it can cause a bit of flux particularly if it's a developer machine because then you've got a you'll have the library and the dev package constantly switching over. So that kind of churn is unavoidable and if it starts affecting you in that kind of situation you can start looking at whether you use a testing and testing grip as your base because then the packages will migrate together and it's all done. So just a comment on the memory thing you're presumably basically talking about package management stuff and yeah by far the biggest difference is just having a shorter list and the problem is that then if you do want something from Debian so you put the big list back in now you lost all the advantage again such as life and I'm not sure there's things we can do in-app to reduce the amount of storage space it takes up by gzipping lists and things which are options you can turn I don't know if there's any options it's got for use less memory because there was a thing about doing it does run out of memory and complain very loudly if it does and that is highly predicated on the size of the bin package cache or packagecache.bin and SRC package and you can turn those off so you just don't have those files but I don't think that's such a memory but then they generated each time you ask if you ask out for this cache data it goes back over all the list so the sure answer is no we haven't done much and I'm not sure of how much we can do but I've made the list sure unless anyone's got any bright ideas yeah since you were mentioning about the fact that it would take a while between the upload then the build and then you're doing it because it has to go to mirrors it occurs to me that that's the problem which has been solved really because buildies already build before it goes to the mirrors so why can't you be just another kind of build demon that builds out of incoming it's because we're not actually getting a source package to build from want to build we're not completely a buildie a normal buildie will be given the DSC and say build it and upload it that's not right that's not true what happens for build demon is that they there's a script quindiff which gets the source packages which actually gets the packages stores it into a database and buildie checks which package is ready to build and it can download it from incoming from a specific archive which is only available to build demons but there's no reason why your server wouldn't be allowed to download from that currently F2V master aren't quite willing to give us download access to incoming at the moment so this would be something we could look at after integration if users complain a lot and say this is really causing a lot of churn then we could look at that but currently it isn't something that Gannif wants to give us access to directly and the other thing is that you've still got the delay because we have to wait for the buildie to upload RML before we can download RML and put up RML em1 but if that was the wrong side of a de-install run well that's going to be us as bad luck but the actual processing of each package even a kernel package takes us very more than a minute to actually unpack, repack, upload and a lot of the time of that is actually the upload again so it's a very decent box and it can do this kind of processing very easily Paul Mr question about the Gobi document here what's this thing about enhancers what's that about? Ansgar, do you want to cover that? That basically requires that for every binary package in a suit the source is also in the suit and enhancers basically allows to upload binaries when the source is not yet there and it will copy it from somewhere else So when the upload comes up from Blabbit then the that will actually check to see whether the source package for that upload already exists in CitGrip and if it doesn't it will list it so because we're only making binary uploads and we're actually uploading one other thing I need to mention to you your source packages might build 7, 15, however many binary packages not all of the binary packages that come from your source package will show up in MW and Grip we are very selective we don't take every binary package you build because we need to do static dependency check-in on all of the binaries we have and one of the main reasons we one of the main methods we have of keeping the MW and archive small is by flopping off particular binary packages that would otherwise bring in long dependency chains that we don't actually have a use case for within MW so we do cherry pick the binary packages that come from a source package that's one of the reasons why the DAC changes have to be a little bit carefully done to make sure that the source package always matches up Who decides about the packages list that is part of the deal? Package selection has been done fairly arbitrarily up until now we'll be pushing a more orthodox method of bug reports against building MW and then as long as there's some kind of reasonable cause for having those in MW those packages in but it's just having a chance to make sure that they would not just randomly add in packages so again, maintainers won't be expected to make any kind of uploads to MW and Grip if you want to you just make the bug report yourself so in terms of as you just said you're not necessarily going to take all of the binary packages from source upload so how do people find out which ones I mean where's it stored obviously you've got an override database of some sort tell us about that The list will be kept actually within DAC it'll be part of project B so any developer can look it up on rice but there's also already a search engine on mdb.org which you can look at if you want to look at particular packages and see what's there once it's in the main archive it'll show up in packagesdb.org we're hoping that the web team will pick up on this in due course and actually list it in the pts so that again there'll be a little box on one side that says these are mdb versions of your binaries all those kinds of assistance methods DDPO, things like that they can all have these little listings once the packages start arriving in the main archive we can talk to these teams and say look there's your packages file that's what's in it how can you show that kind of information one of the things I'd like to have particularly in packagesdb.org is when users view when you've got squeeze wheezy grip in the top corner there there'll be a squeeze wheezy and said you'll have wheezy grip and seed grip next to it the package description on the page will be similar to experimental it does have a little banner that says this is the mdewian package and it doesn't contain files that you might find in the main devian package so something along those lines could be added to the the pdl page so that's that's covered grip fairly well crush is something that we want to put back into discussion once we have multiarch in place crush is changing binary compatibility is changing the functional behaviour of packages it's not going into the devian archive as a first stop it might be once a couple of releases in the future we have some kind of secondary archive arrangements depending on agreement within the within the community as a whole but certainly we're not expecting crush to follow grip just automatically that's not going to happen it's too complex to put functional changes into the archive along those ways but we are looking to restart development on that and it'll be multiple architectures the problem with the first release of crush was it was only and no v4 arm at that time so we really want to broaden that out and try and make crush almost as broad as grip can be if we can do it that way so one of the things that's changed since last time we did all this a few years ago is the bootstrapping stuff in d-package where it's not in d-package yet we're still arguing about it but the d-package maintainers preferred method is in fact profiles so that might be that provides us with a nice integratable mechanism in Debian to be able to say build this stuff with reduced dependencies so we can have an mdebian profile as well as a bootstrap stage 1 profile and that's looking to me like quite a sensible scheme it's been borrowed by lintian as well lintian have this kind of profile matching as well so we've got that kind of behaviour because even with grip if you're building packages on grip you want to have a different lintian profile because there are things that lintian is checking for that are not going to be there so there is support for that already the similar profile support for d-package vendor has been in for quite some time now it's been available there, it needs updating, it needs widening but yes the mechanism is there how these kinds of systems go through and to roll it through as you so what do people want from grip, from crash from mdebian as a whole from our tool chains what are we not doing or what are we not doing enough of Paul multi-arch cross tool chains bring them on on their way there is a google sample of code there are some already available multi-arch cross tool chains in mdebian this funny mark t-bol slash repo you can look at t-bos what you have but maybe somebody in the audience not going to have a look just search for t-bos blog he has full explanation how can you test the tool chains someone is putting in I don't remember the past you spoke about a reduced base system trying to get a system that doesn't depend on pearl or python that kind of thing that's something we did with the first crash it depends I'm assuming that people will want something again it got us down to a very small base install system but it is a lot of work to take core utils out and put busybox in because there are a lot of mdebian maintenance scripts that expect particular tools to have particular options I think it was grep-x was one of the strange ones grep supports and busybox grep just can't do so that's the those kinds of things then need to spread out through a whole range of other packages so when you start looking at that problem the number of packages you're having to cross build and maintain cross builds of suddenly goes up exponentially equally with taken out pearl it's a very good idea it does shrink the crush installation by a lot so it takes out a huge chunk but again there are maintenance scripts that expect some kind of pearl behaviour you have to try and re-implement or make certain scripts no ups all that kind of work was done for mdebian crush against lenny a lot of package changes have gone through by then so we need to really ask the community do you want to have that level of work applied do you need to get below say 32 megabytes for a debon install and ask the community at large if we take out that level of support is it still debon are you looking at actually changing the nature of the distributions in that way can you give us some examples of the sizes that you were able to get grip and crush down to with arm and maybe something like a radical replacement using busybox might get down to oh you've got numbers there great there we go minimal installation of mdebian crush as we had it for lenny got to about 24 megabytes without x and we pushed it to about 38 I think I got it down to on the balloon 3 board with x actually running I think it was about 38 or 40 odd meg by the sum we finished there was someone very disappointed we couldn't fit on this 32 megabyte comeback flashcard but we couldn't actually do that we couldn't get x that low even taken out coattails and pole and put in a busybox it is true that an awful lot of other things have happened in the time since then so I think that Intel and others have done on ycto and tools like that and the amount of attention they're getting does mean that when you get to systems that are that small and you're having to perform that amount of surgery and are changing that many of the sort of basic expectations people have about what is there if you call it a a Debian or a Debian derivative system I would really encourage if my opinion is someone who doesn't currently use any of Grip or Cross or any of the other pieces is relevant at all I would really encourage you to stay focused on Grip because I think that's the place where the potential for having something that feels almost as good as Debian but is just missing some of the fat around the edges is potentially really valuable and on some of these other things if it's really a lot of work like that there are a lot of other ways to build that's useful that's useful to feedback because we need to think about these kinds of things how much work we set ourselves up to do the one thing that occurs to me since Lenny is a lot of this geo object introspection stuff has come through that's really complicated the dependency chains for all of those packages all the clutter stuff all the GIR packages trying to work out a new path to those dependencies to get down to the kind of sizes we had to crush we need to work out is it worth doing to be honest I'd have to echo Redale I mean Grip sounds very very awesome for not zero effort but much reduced effort I can see a really really good use case for it already Debian live with Grip reduce the memory footprint get more on the CD people just testing Debian frankly they're not going to care about copyrights and things I don't know how much Debian live relies on the content being put up being translated that might be an issue because we do remove all the translations as well as all the man pages so being live yeah we might have to look at exactly what's happening with that but yeah there are lots of other situations where you don't need the extra stuff that's in these Debian packages I just want to say that the copyrights are kept otherwise it's violating they are compressed with Gzip9 but they are retained so yes of course can we have a half grip with the translations kept hahaha that's something you've asked me for a long time ago when I was still thinking of doing t-debs there are problems with that mainly down to how the translation workflow would need to work with the rest of Debian so that's a different issue but yes there was always this idea that you could again cherry pick which packages and which languages you brought in the translations for because in Debian at the moment one of my packages drivel it's got 34 different translations and they're all 30 kilobytes and you can't think of a single system out there that would have all 34 locales in use by a user so why are we bundling the entire wrap of an extra 2 or 3 megabytes to the download of translation files and then translation pages as well as we all observed there's a lot more storage available these days and we don't really care about tiny images anymore just sort of not ridiculous images but on the other hand we do also make nitred images out of stuff we make stored images and we make live CD images and there's all these somewhat reduced and we have machines that still don't have enough memory so we do have various aspects of reducedness that are usefully targetable and I think the most useful work we've done by a long way is stuff that ends up actually in Debian properly and it's working on often mechanism rather than necessarily distro to make stuff possible I'm not quite sure how our magic works and maybe we should look at that seems to be the only case for a busybox based something that might be called Debian that's the only one I can think of and that's quite a small targeted set of packages so there's a fairly small amount of stuff that you would have to make work and say these things are small and ought to work on busybox as well because we already do it in fact we should do it in the packages maybe we should do something a bit more structured I don't know what I'd like to do with crush when we get to that point is think about not preparing a suite that is one entire distribution but it's actually a collection of modified packages critical little points within the dependency chain and you can superimpose that on your other package selections from grip and you can just trim out big areas so that you can make it easy to drop LDAP from a huge range of things but also to be able to do exactly that to have different versions of busybox crush in an archive and just pick that one according to its functionality according to what you want to do with it you don't have to take we don't try and build an expectation we're building an entire but actually building components that you can rearrange yourselves times nearly up are there any other questions Wookie the question was when the integration integration is going to be ready I was hoping to get a lot of this work into easy it became too late there's a lot of stuff to do in that there's a lot of stuff to do there was a big change had to go in how we actually process grip in the first place to get the new mechanisms working on a DSA host and using a local mirror and all these kinds of things we had to do so the ideas came up at DEBCONF last year it was January, February before the package is really starting to move through that new mechanism so yeah we've missed Weezy but certainly I can't see any reason why we still got the support of the FTP team we still got the support of the release team and want to build the other core teams in Debian so there's no good reason why this won't make Weezy plus one I can't see anything blocking it at that point so effective we'll carry on with our sort of semi-unofficial Weezy grip and that will look back at this because it's basically turning through I will be making a Weezy grip release alongside Weezy at the end of this freeze cycle it's more difficult than it would have been with either of the two mechanisms either the fully integrate one or the one we had before we started the integration but it is possible to do and it will happen Thank you very much Neil