 Welcome to the tower. This is the first talk. This is Embedded Debian by the man who needs only one name, Wookie. If you're watching this at home, the ILC channel to submit questions to is hash dc6-talks-tower on free-node. If you're watching the wrong stream, you might want hash dc6-talks-hacklab to talk to Aber and his Python boff. Without much further ado, Wookie. Hello. So, yes. This is supposed to be a boff rather than a talk, so I'm not quite sure how to play this. But as there's lots of people that don't recognise, I think I should cover some of the ground of where we're at with Embedded Debian at the moment and then open it up to talk about whatever people want to discuss. So, I'll do that by basically talking about the Extra Magira meeting because that covers everything we're currently doing. We went at the behest of the Spanish government to spend five days having a big session in Extra Magira, which was a fine event. We got quite a lot of people together, all the ones who are actually doing stuff and made some real progress on all the things we've been meaning to do for years. So, the first of those is Scratchbox 2. I don't know how many of you are familiar with Scratchbox. It's a cross-compiling environment which, by means of devious techniques, lets applications think they're natively building but actually cross-build them. So, you can do things faster and not have to worry about all the things you haven't built yet, which particularly for Debian with its wider range of dependencies makes it much easier to get started building from scratch. The problem with it is that you actually need a great big 400 meg pile of stuff because of all the documentation tools and things that stuff build depends on. Keeping that up to date is a bit of a pain. So, Scratchbox 2 is a mechanism to actually use the standard Debian packages but still perform the magic sandboxing so that stuff works. They prove that a scheme using, well, there are two possible schemes to basically intercept calls to open all the file calls via Geolibsy and send them to other binaries. So, if you actually wanted to run an ARM version of something, it would go off and go, ah, you should want this version over here, really. So, it would run different files from the actual paths presented as a mechanism for configuring of that. So, that's possible and hopefully will be available to use maybe this year, depending on how fast people work on it. Enchip over there can tell you all the sorted details. We also worked on cross-tools. At the moment, there aren't a standard set of cross-compilers in Debian, which is a bit of a pain. So, anybody who wants to cross-build for ARM needs to build themselves a toolchain. So, we've been maintaining some at mdebian.org, but it's hard work keeping them up to date because people keep uploading new versions of the compiler every three weeks. So, we have to rebuild all the cross versions so that they still install on testing and unstable. We've been doing that, but it looks like we've now persuaded the powers that be that we can have a set of common cross-compilers in Debian. Obviously, you don't want every conceivable version of SH3 to MIPSL cross-compiler because they're all nearly all useless, but all the ones that run on fast machines that people actually have like x86, AMD64, PowerPC, two common targets will be in main, which will make life a lot easier because you just apt-get whatever cross-compiler you want. So, we have patches that work, but we just have to actually put those in. And if anyone's volunteering to do that work this week, that would be marvellous because then it might actually happen. Bulls have been working on SH3 cross-tools. A guy called Jonas, something or other, Maya, has got all enthused about making the SH3 cross-build stuff work and has been doing a fine job. So, since last year, I don't know if any of you came to see me witter on about embedded Debian at the previous Debconf. We had a lot of plans and schemes, but not much in the way of actual working stuff, and that's changed enormously this year. So, we now have two working distributions. There's Slind and Stage. So, Slind's been done by a couple of guys at Siemens in St Petersburg, and that uses the d-package cross mechanism in Debian to cross-build tools properly in the auto-conf way. It uses a BSD installer, and the only bit that isn't beautiful is there's no real way, with the current way that Debian's install scripts work, to tell, if you've just built a whole load of stuff on your build machine, and you now have the target charoute, it's difficult to run the scripts on that charoute without them going and affecting your build system because there's no way of telling a post-ins that its roots moved. So, what they did was actually copy the post-ins over to the target, and, well, NFS mount it on the target, and then run the scripts there, which is neat. But the problem is that if your target doesn't have Ethernet, this mechanism is no use to you for getting things actually configured. Scratchbox is actually another way of solving this problem. It can pretend that the root has moved for your purposes, and thereby deal with the problem of running install scripts, which was something I hadn't realised until recently, so that's a really cool feature. So, Slind is good. It's about 40-odd packages done. You can build things with it. It supports ARM MIPS. PowerPC, I think. A set of common targets, basically. We also have Stage, which uses an extension of the STAG scheme, which I detailed last year, using an MWN directory instead of a Debian directory, with changed rules and postscripts and dependencies. But it uses Scratchbox to do the actual cross-compiling, so the whole cross-compiling thing is just solved, and you make your Scratchboxes problem. And again, there's 40-odd packages done. One useful thing that happened at the Extra Majora conference was that we realised that one of the advantages of the Stage mechanism with a separate directory is that you can have an arbitrary number of these directories, so it's easy to do. So, you normally have a Debian directory with the rules and build information in it. Then you have an Mdebian directory with the embedded version, but you can also have my favourite widget directory, and all you do is change this one variable, Debian dir, and that will override the normal rules with your rules. So, it makes it very easy to personalise the distribution for your particular purpose, which people usually find they want to do for any given embedded project. So, all that means, in fact, is that you can just put the slinned rules in a directory called slinned, and the slinned becomes a subset of merely one particular flavour of this mechanism. So, they realise that, in fact, they are the same project, really. And we're in the process of merging the two sets of patches at Mdebian.org, and then, basically, you'll have a choice of whether to build it deep package cross style or scratchbox style. In principle, either can work. So, that's all jolly good stuff. You can download those and try them now. I'm intending to try and do a build for a board I've brought with me this week. And if anyone else wants to have a play, please come along. Is Ed here? No. So, the man who's done all the stage work is actually at this conference, but he hasn't got up this morning. We also talked about the ARM EABI transition, which isn't, strictly speaking, part of embedded Debian, but many of the people involved have an interest in that as well. We've got a separate buff on that, so I shouldn't cover it at any length now, but, basically, ARM is changing its ABI generally, not particularly because of Debian, but just, well, because of ARM Limited, really, and the rest of the world. So, we're going to have to follow suit at some point, and we've been working out how to do that, which, basically, is going to be a new ARM architecture, which is different from the old ARM architecture, because trying to move from one to the other is difficult. Other people worked on Debian installer for various widgets and QEMU, which is the emulator that is one way of running non-native binaries on your build machine. So, I explained about Slendon Stage, the basic differences and commonalities. So, that's where we're at. Quite a lot of progress since last year. There's a lot of stuff happening. There still isn't, like, an Mdibian distribution you can just download for a whole range of targets, but we're getting quite close to that. More people coming along and doing work will obviously speed things up, so anyone who's really enthused is most welcome. So, the question really that I was going to talk about today was, now what? The things that became clear from the Extra Majora meeting were that the next thing, simplest of the next thing to do, is get cross tools in Debian main. I'd like to try and do that this week with a bit of luck. Anyone who's enthused about that sort of thing, please come along and kick my arse, or otherwise help. The stage and slind both will benefit from more packages. Obviously, 40 packages is a minimal-based system. You get X and some networking and a shell and a few utilities, but nothing terribly exciting. So, packaging more packages. For stage, it's generally pretty much a case of replacing every instance of slash Debian with dollar Debian dir. That gets you a functioning package and then you can worry about the details and just remove a few bits for installing documentation. We'd like to do some slightly smarter ways of dealing with the metadata. So, one of the fundamental things that people want to do with embedded systems is throw away all the docs. It's not complicated. We just don't want them. They're too big and they take ages to build and bring in a whole load of dependencies you didn't want. We are working on policy for package maintainers so that there's a whole load of stuff you can do as a package maintainer which makes it much easier for the embedded people to build parts of your package or whatever. Of course, people don't know what they should be doing. We've started some documentation on a few more Debian build options like no docs and no test which will basically not bother building any docs and not bother running any tests. That would be a huge step forward just if we had that in most packages. There's also a whole load of stuff about build dependencies. At the moment, because Debian's always been designed for native builds, there's no concept of dependencies which are actually required on the target as opposed to the build machine. Quite a lot of things build. Most of the dependencies are actually on the build machine but sometimes there's a few things which get run natively during the build. If we were to improve the to segment the dependency lists a bit it actually said whether these were things which were supposed to be native packages or supposed to be build architecture packages that would make our lives a lot easier. But we need to get changes made in the policy to get people to start doing that. The same principle applies to dependencies which are either run time or install time. So again, there's a whole load of stuff which is actually only used by the install scripts for example update menus. So you don't need that on your target. You only need it at install time and then you can throw it away again. But obviously to have a generalized system to do that we need to know whether this is a dependency that's needed at run time or only at install time. So again, a bit more segmentation there would improve our lives a great deal. Multiarch is also related to what we're doing a number of things in terms of cross building are made easier by, well, and installing are made easier by the multiarch proposal. So we're very keen to see that happen. I get that there's a small team trying to push it forward and we'll do our best to help out. Is there any multiarch people here? No, I know Matt Taggart is at this conference but he's not here this morning. So those are the things that need doing now. There are a few issues still under debate. The biggest one probably is where to put our metadata. So the thing you need to do to build embedded packages is have extra metadata that says things like these are docs so you don't need them or we don't want to do this and we don't want to do that. And there's two fundamentally different ways of doing this. So there's the way it's done for Debian installer using the Udeb mechanisms and extra information and there's the stage mechanism of having another directory which changes the rules normally in the Debian dir or you could change them in the Debian dir. And the philosophical difference is do you want to produce a set of packages with the same names as normal Debian packages but different contents, smaller, maybe a few obscure things missing because you really don't need them? Or do you want to produce a set of different packages with different names and these let you do different things? So if what you wanted was something that looks like Debian but is in fact a bit shrunk and we don't necessarily do the name thing, the same things then the stage mechanism is the way to achieve that. The problem with that is that it's not compatible with standard Debian packages. So if we miss out a load of bits and bobs from various things because most of the time you don't need them, the problem is there'll be some package somewhere which you might then install from normal Debian which did actually need one of those things we throw away. And because we've got the same name so we're claiming to be that package, the dependencies will work but it'll just buff. Now that doesn't matter unless you're trying to mix packages from Debian main and from the Debian repository. So that's the problem with that scheme but it's conceptually simple. Everybody can understand how it works, maintenance can understand how it works, it's great. So the UDEB system is rather more rigorous really if a package is different it's got a different name and so you have to have different dependencies if you're going to depend on those. So you get a whole separate section of stuff and you can have the standard Debian packages and the UDEB versions in the same namespace. So if that's what you want to do which of course is what Debian installer wants to do that's the way to do that. So at the moment nobody is really working on building embedded Debian in the UDEB way apart from the DI team with their sort of specialised version of embedded Debian effectively. So there was quite a lot of argument at Extra Majora about which of these things we should be doing. Some people like one, some people like the other. So that's still an open issue. We haven't decided particularly but right at the moment people are working on the Debian scheme so that's what will get done. If anyone believes this is a terrible idea and the problem with this is that once you start doing something it tends to collect momentum and that will be what we do forevermore. So that's an issue and I'd be very happy to talk to anyone about it. We can discuss it some more today if anyone is interested. The other thing is getting what we're doing back into Debian main because the main problem with both Slindan stage are currently just forks of a snapshot of Debian which is fine but you can't maintain that for long. Once you've got a few hundred packages you're now maintaining the hold of Debian again yourself which at the moment with about three people doing it part-time is hopeless. So we have to get that stuff back into main so that package maintainers look after it by and large. There isn't a great deal to do but if they change their rules which people do occasionally it would be nice if they changed the Debian rules to correspond if there's any changes needed. So we've got some mind share I think still to get which is one of the reasons why I'm here bullshitting to persuade people that embedded stuff matters and Debian ought to work on small devices the same as it works on laptops and servers and a little bit of effort at all round is needed to make that work. We need lots more docs we're starting to improve that and now that there's actually stuff to describe which normal maintainers can understand we've started writing down things people need to do and things we'd like to see. Right, that's all I've basically got to say I hope that made sense obviously if you have taken no interest whatsoever in the embedded sphere before most of that was probably complete gobbledygook and I apologise. As I say I was planning for a boff rather than a talk to loads of people who knew nothing about the subject I suggest people ask questions feel free to ask very stupid questions because I probably missed out most of the basics at the beginning Yes that makes sense so he was asking whether as a normal maintainer he needs to he's going to have a mechanism for testing his rules within Mdebian and I think the answer is it's not something we've thought about at all yes what you would need to do is install the so the way stage works in order to this with this change Debian DIR variable basically a remarkable small number of tools actually care you need to teach d-package about it debhelper d-package cross d-package c-dbs so basically you would just install the Mdebian tools which in fact should be in main all the changes we make are totally invisible unless you change the Debian DIR variable so everything will proceed exactly as it did before if it's just if it isn't set it defaults to slash Debian everything works as before so you'd install the new version of the tools tools, which, in fact, hopefully you would just have, because they'll become the standard versions of the tools, and you just set $debion dir equals mdebion, and build your package. And if it comes out okay, then yes, it works. So, Enchewt, you wanted to say something? Yeah, I would just like to advertise that we actually wrote a document that tells what are the best practices for developers to make their packages cross-compilable, and is it easily usable in embedded, basically including stuff like don't use GIMP to make your logos in the Debian rules? Yeah, so there's a number of issues beyond just testing to do with, yes, doing strange things at compile time, like auto-generating your logo by running up a version of GIMP, and if you just put the file in, then that enormously reduces the complexity of cross-compiling. And so people produce all sorts of barock systems for doing strange things at compile time, which need to run on the target. So, you know, because they never think about the fact that the target might be different from the build machine, you know, it's not something most people are thinking about. So people need to think about that, and obviously it's up to us to explain what the problems are to people who haven't really looked to this issue before. So we've started doing that, and we're going to fill that out with a lot of examples so that people actually understand the point, and basically if you have a look at that and go, am I doing any of this bad stuff, that should be all you need to do. And it's a one-off thing, you just have a look at your build system and take out anything that's unhelpful, and you never need to change it again. Any more for any more? Hi, thank you. I'm Mark Allen. I'm a very much newbie in the whole area of embedded processing, and this is primarily for a personal project in mind, and I see an awful lot of hardware being offered, like small one board computers, stackable module computers and stuff like that. But I am such a newbie, I don't know what I'd want, and I don't like, I don't know the very basics. Where can I go just to get the very basics, like embedded processing for dummies, and like basic information, how do you load, once you cross-compile, how do you load it into the machine? Do you use something like JTAG or just what? How do I find this very basic information? Thank you. Yes, it's a good question. The problem with the answer to that question is that it depends enormously on the hardware you actually choose. For example, how you load in the image you've created depends almost entirely on the bootloader on the hardware you've got. It might be supplied with a bootloader you've heard of, like U-boot or Red-boot, in which case you use the Red-boot and U-boot commands. If it's got Ethernet. Usually the very initial stage is indeed JTAG. When you've got a bare thing straight out of the factory, it doesn't do anything. You've usually got to use JTAG to fill up any CPLD or FPGA logic on the board and get an initial bootloader in. Usually the manufacturer will have done that, so you as a user hardly ever have to do that unless you're messing about with the bootloader and royally screw things up and turn it into a brick. There are a lot of open JTAG tools which are increasingly functionful. JTAG has in fact expanded beyond its original purpose, so you can also use it for a lot of testing. There's lots of stuff you can do to see whether your hardware is actually wired up the way it's supposed to be wired up and you can use JTAG for that. But normally you're only ever dealing with a bootloader to get images loaded in and each bootloader has its own commands and functionality and obviously it depends on the board whether it's got, if it's just serial or USB or Ethernet, they're the usual ways of uploading things. So that part is always terribly specific. It's almost impossible to generalise, but whoever provides your board will need to tell you how to get stuff on it. Whoever provides the board, whoever you buy the hardware from, will generally tell you how to get stuff onto it, unless of course it's hard whether they weren't intending you to get Linux on, in which case you have to work it out. Yes. That's right. Your life will be much easier if they do. And for an awful lot of portable devices like PDAs and things that come with pocket PC on them, the vendors won't tell you anything useful about how it works. So in fact there's very unlikely to be a kernel that works on it because if they don't, it's extremely difficult to write working kernels for hardware that people won't tell you how it works. Hardware is generally too complicated to reverse engineer in a sensible period of time these days. So that's right. And if you particularly want to put embedded Debian on, you can ask them whether they have an embedded Debian port for their hardware. Now at the moment the answer will be no from absolutely everybody with a possible exception of me, but that is coming. I have a comment in Japanese embedded industry. Semiconductor doesn't officially support Linux or Debian, but there are many engineers at semiconductor company who has enthusiasm and want to support Debian or Linux for their chips. So if you go to officially to semiconductor company, you perhaps will get an answer, we don't support Linux. Say in Hitachi or Alunesus technology in Japan, the famous semiconductor company, but the fact is there are many engineers who support Linux. Right. So you're saying if you ask the company officially, they'll tell you no, we don't. But in fact there'll be a mailing list somewhere where you can find an engineer who will help. Yeah. So that is often the case. It's a bottom up thing. You know engineers discover that Linux is better usually before managing it. So it's a bottom up thing. You know engineers discover that Linux is better usually before management do. So it is often the case that you can find a helpful person, but that's not the answer you'll get from marketing or sales. Yeah, I mean there are companies, at the moment there's a lot to be said. There's a small number of companies who are very focused on for example Debian and provide hardware and you should support them and buy their stuff basically because they are keen to help people like you. So for example there's a board called the loft from Giant Shoulder Inc. Giant Shoulder Inc, which is a bloke in the States. He makes some nice ARM based IXP hardware, which makes quite a nice dev board and you know you can already put Debian on that. And balloon board is something I've been involved with and I'm in the process of putting embedded Debian on that. And there's other things. Nokia 770s are well supported by the tools because most of the people doing the work have been using those as dev boards because they work for Nokia. And so on. But basically I mean if you come to the mailing list of the channel and say recommend me some hardware, I need it to do this, that and the other, that's a good way. You know people like us have a reasonable overview of what's available. Well it's increasingly difficult actually because there's thousands of things available, but we can certainly tell you the people who are involved. Yes, hi. I have a couple of support questions. Like what is the stage of support for Usylipsi based Debian system? Right, yes. One of the things I forgot to say is that one of the very cool things that SLIND does is support Usylipsi and Usylipsi builds. So they have cross tools for Usylipsi as well as Usylipsi via new architectures. And all the packaging assumes you're doing either. But how do you manage like several packages need Usylipsi patches, specific patches to be compiled against it. That's right. So that's like another delegation to maintainers. Yes, well, yes. So those packages are in SLIND at the moment and the idea is that we'll push those up into normal packages so that they'll support either. Now I don't know how contentious that is. I'm not quite sure if Debian's ready yet to support an alternative C library. At the moment that's not something we do. But it's very much something that embedded people would like to see happen. And those patches are fundamentally utterly unintrusive. If you don't use it, you won't notice it, it doesn't do any harm. So I don't think it's going to be too difficult to persuade people to say, please add this, it makes some people's lives easier and it's not going to do any harm. So yes, that is... Okay, that's for Lipsi library. How about the standard C++ library you see Lipsi++? The same principles apply, but I don't think anyone's done the work yet. Okay. And how about Busybox-based system installation like the base system based on Busybox instead of core hotels and whatever. So that's done to Slindy's Busybox? Yeah. Again, Slind can be set up so the base system is basically just Busybox. And you don't bother installing a whole load of other packages which it replaces. So yes, we have that choice. But that will make some problems with dependence of the rest of the packages. That's right. So what we're trying to do is define a kind of new essential because the existing essential is basically okay and is provided by Busybox, almost all of it, apart from Perl, which is currently deemed to be essential and that's really annoying because it's huge and it shouldn't be. So a slightly longer term goal is to get the essential definition moved down a bit to what really is essential. Okay. Which basically means kicking Perl out. So if you don't need it, you don't install it. Then basically we can say essential is provided by Busybox or the stuff that currently provides it. How will your life manage, they need their scripts and stuff like that that tend to be delicate on Busybox? Sorry. How do you plan to manage any of their scripts to initialize stuff and stuff like that on Busybox-based? There's a few things. A couple of years ago, someone went through all the init scripts and made them ash-compatible rather than using any special bash-isms. So it doesn't really do anyone any harm. There's very few things that you actually need to use which aren't in the Busybox versions of things but are in the full versions of things. So really it needs somebody to just go through everything, taking out anything that isn't going to work if you replace bash with Busybox, well, sorry, if you replace all the mount and so on with the basic versions. Mount's probably a difficult example, so that's one case where you sometimes do need the fancy features. But yeah, it's basically a matter of going through and finding anything that doesn't work and fixing it. It's usually utterly painless, but it does need doing. Okay, perfect. Also with respect to your life, you were talking about having different packages providing not the complete set of the full package for embedded systems. Have you analyzed the possibility to have like an install time mask of which part of the hierarchical file system will you populate? Do not install everything in your search slash share slash doc. So you're saying just at install time, so you take a standard package, but you just don't put some of the things in. Exactly. Yeah, so the d-package maintainer has been looking at a mechanism for having manifests so that you can basically have some rules which it will follow at install time. As you say, just don't put anything in user doc. Just miss all those out, which as you say gets you quite a lot of what you wanted. So the disadvantage of that system is that you still have to build all that stuff, but you never install it, which wastes a really impressive amount of time. It's quite scary how much of the time you spend building docs with strange doc tools. But clearly it's a useful thing to feature to have. It's kind of, it's orthogonal I think to most of the other stuff. You can do it now today and it will be really useful. Yeah, but you actually already have the full scale packages built. Exactly. So you don't have to rebuild everything. Precisely. So yes, it's been looked at. I think Guy M would welcome some help in terms of actually implementing that. It's a little bit tricky because you, you know, a deep package is something you don't want to break. So you have got to be careful making changes to that kind of thing. But I believe he has a plan. I'm not sure it's written down in much detail anywhere. All right. Thank you. Just wanted to say something. If you're quick. Just a quick. Just one thing. I'm on multi-arch. It seems to me, I read somewhere in the case that I didn't, that there's no idea, no plan to actually have a multi-arched user bin or bin directory, which seems a bit strange to me because I think multi-arch would be useful if you use it in conjunction with QIM to actually allow people who do not have the certain personal architectures to be able to run or test and run their own PC. Yes, I agree. We should talk to the multi-arch people. I think we need to find some if there's nobody here today. Okay. I think, do we have to stop? There's more questions. I'm sorry we have to stop. So you can come and ask me afterwards. Thank you kindly.