 Welcome to this talk on bootstrapable Debian. I'm Wookie, just in case you didn't know that. This talk may be slightly random because it's a little bit complicated and I'd like to think that I can explain to you how this all works without being terminally confusing. So, this is about various things. Bootstrapping is when you need to make a new architecture, fundamentally. That's the main use case in Debian. If you wanted to actually build everything from scratch, how would you go about it? It's actually very hard because of the way our system is always built on what we already have, so there's no requirement to build it all from nothing and, in fact, you can't. So, this is a real problem every time we start a new architecture. So, I'll talk about that, why that's a problem and what we might be able to do about it. Fundamentally, the problem is cyclic build dependencies. You can't build this because it needs itself, or at least there's a loop of things such that it needs itself. I've done some analysis on exactly how big a problem that is in Debian and Ubuntu. And, strictly speaking, bootstrapping the architecture is nothing to do with cross-compiling. It's fundamentally independent of that, except that, in practice, that's the only way to do it. So, the two subjects are highly intertwined. So, I'll talk a little bit about that at the end, just what's going on. I've been doing this work, essentially, because of Leonardo. It's actually quite a long way down my list of things I'm supposed to be doing. So, I'm not really supposed to be worrying about how hard it is to bootstrap Debian. So, anyone who wants to help will make things move along much quicker because people who give me lists of things to do put this one down at about number six. Nomenclature's a problem. For those of you who aren't familiar with cross-building, the GNU names are terminally confusing to anyone who doesn't know all about it already. Build is the machine you're actually building on, and host is the thing you're targeting. That's not the same as target. Confusing, isn't it? I'll try and use the correct terminology throughout just so I'm not confusing. Also, deciding how to do this so that it makes sense to package maintainers is tricky. It's important that the terminology of whatever gubbins mechanism we come up with makes sense to a random package maintainer who's got to read it in their packages. So, I've used in the documentation about this both staged builds and bootstrap builds for the same concept. So, those mean exactly the same thing, and which it should be kind of up to you guys, which seems most useful. Staged is already used in some packages, most noticeably GCC and the bootstrap builds. For a while, I thought bootstrap made more sense, but the problem with that is that it means six other things as well. So, actually, I've kind of changed my mind and gone back to staged builds. When talking about dependencies, there's actually two sorts of dependencies. There's build dependencies and what we call dependencies. But I shouldn't use the term dependencies at all here. I'm going to talk about install dependencies, because otherwise everything I say will appear to be nonsense. It's important to distinguish between the dependencies you need because you need them to build and the dependencies you need because you need them to install. So, as I said, strictly speaking, you can bootstrap without cross building, except that in practice you can't, because you've got to build at least the tool chain. And then once you've done that, you might as well build a few more things cross and so on. And there's an interesting question about how much stuff you should build cross before you call that a native system and build the way we normally do. You know, you could build the whole damn thing cross in theory. People take different approaches in different ports and there's different trade-offs. But there's always some. So, that's what makes these two subjects closely related. There's a concept of the set of things which, you know, you have to produce as if by magic from somewhere. That's pretty much the tool chain and some base packages and any language which depends on itself to bootstrap. Language authors like writing their language in their own language, which is all very nice except when you have to build the damn things. So, you can't build Mono because you haven't got Mono yet and so on, and Haskell and all of that is all terrible. So, there is a set of nodes, effectively, from which everything else is built. In theory, building all the packages in Debian is a tree. The problem is that it's all wrapped around in loops. We don't have good tools to identify what the set of things the canonical nodes are yet, so I'm not sure. I know what some of them are and there's probably some more. So, what do we care about this? I guess I've already mentioned that the main reason is anybody who needs to make a new architecture. We kind of think, oh, that doesn't happen very often, but actually, it happens fairly often. I think they've been about 17 so far, so basically, we do this every year. The most recent, obviously, has been the RMHF bootstrap, which has effectively been done twice, once by Konstantinos, who pretty much did it all native and again recently by Sledge, who crossed most of the base system. There are other uses for this, though. If you had this technology working so that it was actually possible to disentangle all the build dependencies, it makes it much easier to do an optimized build. If you wanted to build everything for Neon in ARM world, in theory, you could use your existing packages to do that, but you're never quite sure whether that's actually going to work. So, in some ways, there's something to be said for starting from scratch and building it all like that, then you're sure you haven't got random bits, which are old-style. If you were pulling packages out of Army L to build your new thing, you might end up with a bit of a mix. It was pointed out to me by whoever it was that recently bought the M68K, or has been working on bringing M68K back up to date, basically that's now a cross-built architecture. They don't build it native anymore or not most of it. For them, they have the same problem of cyclic dependencies. There's also architectures we might want to have in the future one day which pretty much can't build themselves because they're too weedy. AVR32 is a good example. And it just seems to me fundamentally that this is the universal operating system and it's slightly embarrassing that you can't actually make a new port without Gintu or open embedded or something in order to get started. That's how Konstantinos got RMHF started by using a Gintu-based tarball and then building enough Debian stuff on it that you could boot into that and pretend you'd started with Debian. It works, there's nothing wrong with it, but I just feel that we ought to be able to do this more correctly. So, as I say, people use, so the Army-L build was done with Angstrom. Yes, please, interrupt me. Microphone. Technology. Anyway, my point was that I've seen recovery of broken upgrades completely fail because of cyclic dependencies. So if you have an upgrade that's triggering a full disk, cyclic dependencies make it impossible to automatically recover. You have to do manual hard forced installations of some packages to get going again and that's because we have cyclic dependencies. The package system will just break the upgrade in a random position and the resulting systems will be inconsistent and very, very hard to recover. So it's not just a nice thing for some setups. It's a day-to-day problem when you run into full disks. Although you're talking there about cyclic install dependencies, which of course is slightly different from cyclic build dependencies, although they are, as I've explained in a minute, surprisingly tightly related. Just a comment. I've boosted up a few architecture in the past and now still again and the situation is getting worse and worse. Today, for example, for installing developer, which is the minimum, it depends on main page and for main page, you need basically Xorg, you need Qt, you need GTK, PHP, and just because there are a lot of Python bindings, per bindings and all of documentation building. Yeah, it's true. So, yes, it's hard work for a lot of reasons. At the moment you take your package, you just keep removing bits until you can build it without all the stuff you didn't need until eventually you've got some packages you can run, enough to call a system. But still, you need to build hundreds and hundreds of packages until you've actually got all the doc packages needed for many perfectly boring builds, because you need the whole of tech, or the whole of Qt, or the whole of something. So, the most we can minimize that, the better. Just mention xDeb here, which is a handy tool for cross-building things. It's relatively painless. Steve found that using that for the RMHF bootstrap was actually relatively painless for the base 112 packages you need for the bootstrap or something. There's actually only a few cyclic dependencies in that bit, I think. So, the reason why this is a more tractable problem when you're cross-building than when you're native building is that all the stuff which is just doc tools, tech, and anything which is just a tool, you've already got a version of that on your build system. So, you haven't got to build tech before you can build anything with a man page, or graph, or whatever. If you're doing this natively, you have got to do all that stuff, or stop all the docs being built. So, the cross-case is basically where I've been concentrating my analysis. You could do this again properly for the whole archive natively, and that would also be interesting, but that's not what I've done when I look at a slightly more tractable problem. So, in fact, if I get the right pictures... So, here's a good example of a fairly typical build dependency tree. You go, let's find out what the dependency loops are in this package. So, you run some software and feed it to dot, and you go, oh, dear. Well, that hasn't really told me very much, has it? Well, it's told me. They're true. You all think it's a tree, don't you? You always thought it was a tree. It isn't. It looks like that. It's horrible. And actually, that's been post-processed to get rid of some of it. That's ghost script in a maverick, which is particularly bad. Now, Jonas, I don't know if he's here, actually fixed it, so it doesn't depend on a million things, and in fact, it's dramatically improved already in squeeze and in natty. But that's the sort of problem you have in even trying to analyze the problem. What should we look at next? So, what I did, so here we are. This is a more tractable tree. So, this is one which actually you'll find comes up hundreds and hundreds of times. This occurs about 600 times in the base of maverick, for example. Now, this is more or less what's Morkin was doing earlier, but this is the free software version, even that sort of thing over here. So, within that loop, there are in fact several loops, as you can see, that's not just a loop, it's several, but a common problem in practice is poplar. Poplar depends on some bit of QT in all the display things, which means you need to build QT, and that requires LibOdbC2, which needs GTK, which needs CUPS, which needs poplar. So, that is absolutely typical of the problem you get. And because, in fact, within this set of about 20 things is an awful lot of popular stuff, like GTK, and QT, and Arts, and CUPS, and Pulse Audio, and Blues, and GComp and Glade, it's very easy to get into this loop, which is why hundreds and hundreds of packages use something in here, and as soon as you use it, this is a problem for you. So, if I just show you, so, yes, how do we get that tree, tree loop? So, I use DexDeb because that understands about build dependencies and install dependencies in a way that Deb Tree, the other obvious tool for doing this, doesn't really. So, you go with DebTree, that's the right thing, isn't it? And that produces some interesting graphs, but it doesn't have this thing about something depends on a library, and that library requires a particular source package, and as soon as you need to be able to install that source package, you've got to install all the things which, so you need to build that source package, and then you need to install whichever library was you wanted, and therefore you also need all the things that that library depends on. So, although about build dependencies, in fact there's one level of build dependencies, and as soon as you install anything you suddenly care about all the install dependencies of that particular binary package as well. So, a lot of the time you care about both these things. And DexDeb actually does a reasonable analysis and understands which architecture is which, and therefore when some of them don't apply. The problem with that is that it produces graphs that look like this. And you go, oh dear, this is a bit of a mess again, isn't it? If I get my buttons right. And you go, oh dear, yes. Hmm, okay. Right, yeah, this still isn't really helping me understand the problem as much as I'd like. So, that's GnuPg. And you go, right, well I didn't help, did it? So, what you need to do is find out the set of stuff that you actually care about in there. And it turns out that the dot tools are marvellous for this. There's a thing called SCC map which is the strongly connected set. And that's the set of stuff which is actually connected back to itself again as opposed to all the bits which really are trees and nodes and sticky out parts, because you don't care about those, they're easy. So, you fit it through SCC map. So, we added to DexDeb the ability to spit out effectively the dependency set, dependency graph that is found in dot format. You feed that through SCC to throw away all the bits that stick out sides. And then we feed it through a couple more handy scripts as this GVPR which is magic set foo which basically means you can colour in the packages which are source packages and you can make them round or square and you can say, oh, there's 15 different lines to that package. If we go back here a bit and find one. So, you can see in the middle there there's a lot of these that have got about 12 lines, this has got like 407 lines between two packages because that dependency happens a lot and you don't really care about that, you just want to condense that down to one and then in fact we added a number to it to say how many times it occurred. So, after you've run through all that and processed it, you end up with something like this which is so this is a post script file which shows each of the separate loops within required by that package so a green blob is a source package and a square blob is a binary so that source depends on this binary and to get that binary you have to build that source and so on and we do actually have different shades of green according to how many things depend on it which gives you a clue as to how many packages you have to fiddle with so it turns out if you do this so I ran it on all 1500 packages in Ubuntu and about half of the $20,000 in Debian before I got bored that you end up with the same ones over and over and over again which is good, our problem is slightly reduced in complexity there so these are the typical cases, so there's this QT Cups thing Kerberos, an open LDAP and something go around in a little loop of five Lapac and Atlas depend on each other that Glib one is thoroughly unimportant I don't know why I wrote it down, WebKit happens quite a lot and in recent stuff, object introspection has read it's even ugly ahead and suddenly appears in hundreds and hundreds of packages whereas it didn't I think it doesn't in Natty but it does sorry it didn't in Maverick and it does in Natty it didn't in Squeeze but it does in Weezy I believe that's correct so here's some actual numbers just because we like gratuitous statistics so I ran through 9200 odd packages actually I'll just say how I ran through 9300 so if you look in the XDED package, if you download it there's an examples blah directory which has got a little script called find cycles which essentially runs this stuff and has all those scripts in it so you don't have to work that out again because it was horrible. I should point out at this point the guy who did the heavy lifting on all that graph work was Jonathan Austin I've just stolen his very fine work and used it so yeah so out of 9000 packages 2200 of them have some sort of build depth loop in them 24% nearly all of them have got this glib2 and gammin thing that's about to disappear because I think gammin is almost extinct and I hope that's just will suddenly evaporate but in the meantime this object introspection problem has turned up instead which again happens nearly everywhere the poplar Kerberos thing so that's actually two loops, poplar and qt and cups and there's Kerberos and open held up and something but in fact they've merged together in more recent packaging because of changing dependencies webkit and libsoup happens a lot this lapak thing rpm, I forget what it is it's rpm something something maverick and natia fairly similar I just analysed the main set which is rather smaller and a bit more tractable it doesn't take very long to run the xdev generate thing on each package but you know it's maybe 30 seconds or something each so to do 10,000 takes a while so what are we going to do about this the way to make to build all this stuff most of the build dependencies in these loops basically don't matter it's because we build our packages maximally to include all the possible functionality if there's a handy little script that does something qt-ish or there's a little ui piece we build that but the things that depend on a package don't care about all that fluff they only care about the core functionality so actually when Kerberos depends on an open held app actually Kerberos can be built without open held app thereby breaking the loop so what you have to do is modify the packages to build reduced versions staged builds or bootstrap builds and so the question then comes down to what is the best mechanism for doing that terminology as I say is a bit of a problem there aren't most of good words we haven't already used for something but I've settled on this after a certain amount of thinking for the work I've done so far so what you do is add a new dependency list to a new build depends list just in the package next to the existing one so instead of the normal build there's build depends stage one almost all packages only need a stage one I think the only thing I know of that needs a stage one and a stage two is the tool chain build where you build the really and that may be true for some of the language bootstraps as well where you've got to build a mini language and then that builds some libraries and then you can build the real thing so basically you set an environment variable to say or build options interesting question of exactly again what the best mechanism is I reckon sticking it in build options is probably most sensible because this is fundamentally a different build much like not running the tests or whatever and make sure all the tools recognize that so that they'll use the modified build depends list and then the rules file will not build the bits you didn't want that weren't necessary so you get a modified package now it's quite important that we don't go uploading those modified packages to real repositories so you need a bit of marking on them to say a wuga this is a horrible modification for getting bootstraps done the microphone wanted down here but hello this one works this is working thanks for that so I'm just thinking about whether they shouldn't be uploaded but why I mean the package as it should work it just doesn't provide full functionality so it might be a time in boot it might be useful to share these of course they need to be replaced before going to testing, stable, whatever but what is the issue for example to upload those to experimental and I can see someone wanting it to answer I'm a bit short in precise examples but I believe that there are cases where the staged build is sufficient for working your way around the loop until you can get to a proper build it's not necessarily sufficient for building everything else and I think there are a few cases where if you build things outside the loop with one of your staged builds they'll accidentally leave out bits and you'll never know I was thinking that we probably just wanted you do want a repository of these things there's a complicated sequencing bit which we'll get to in a minute about how many things you build based on the modified package rather than waiting for the real one I'm interested to hear are release chappies consider the possibility that uploaded them might not be total craziness so so if we're going to modify all those packages in order to be able to do bootstrap builds or staged builds you know how many are there having looked at the loop set it's not that many it's quite difficult to work out when staring at this how many actual changes you have to make to make that into a tree structure that's not an entirely trivial question you know how many loops is that some is the answer but I don't think so as I say although there's thousands and thousands of instances of depending on a loop there aren't very many loops actually and most of them are actually coagulated into that big lump of 20 or 25 depending exactly which distro you're looking at so once you've chopped all that up with maybe 10 cuts so I don't think it's a very large number of packages affected which is good and the patches we need to make on the packages I've done so far not too ugly so I think this is a maintainable thing to do it for the native case where you've got to get rid of docs everywhere at least is more intrusive on the other hand if we just have a Deb build-ups, no docs which is actually honored everywhere that's quite generic it's also useful for other things like embedded work and probably does most of the job here is it working? for the documentation part it's also probably a lot of packages can move the documentation building to building depth so we don't have the dependency there to complete the documentation from the libraries to a common package and still also there is a lot of package to fix that are building the documentation in any case even if you build only the arch package even if it's not in the final deb so probably there is a work to do there without going to the no doc option yes it's a good point if we actually make the build in depth thing work then that separates out the architecture independent stuff at build time so it's easy to do part of it and that will reduce our trees not trees I'm going to stop doing that graphs enormously so here's an example of the sort of patch you end up with this is Kerberos 5 stopping it using LDAP so it's quite easy in debhelper there's this marvelous thing I didn't know about till I tried this of dash-capit-len package name she basically says don't do that package throughout the whole set of debhelper commands which is great there's actually this little wrinkle which wouldn't fit on my slide which means actually you have to modify a few more lines with plus i and plus a because dh options are used for more than one thing which is a little bit ugly but basically that's all you need to do change the configure options ignore a package so it doesn't get built and a little bit of jiggery-pokery for not moving around some files that don't exist and similarly for popular is a CDBS package where we can still use dh options so there's more dash-capit-lening and more configuring and you're done so that doesn't look too ugly and evil to me I think that's the sort of thing we could probably sustain in a set of packages 40 or 50 in order to have this actually working in useful as well as modifying the packages to have staged builds reduced builds you want some tools to do this for you because it's hard work staring at all those graphs working out which thing to build next that's the thing we're missing so you need to analyse the tree or the graph and then when you find it's got a loop in it you go have any of these packages got a staged build I can use to chop the line and turn it back into a tree if so do that build so just deep package build package deb build options equals blah when you've done that you can then build the other things in the loop that depend on it and then you can rebuild a normal build of the offending package so I have a GSOC student Gustavo Elkman who's been working on this he's discovered that it's quite complicated it's a hard GSOC project so I'm not quite sure how much actual tool we're going to get because I think we're finished in about two or three weeks time but we have started on this project of generating a handy tool we might end up sticking it in xdeb for now as something that has most of the right infrastructure already the problem with this is that we don't use it very often I mean okay we do a new port every year or so but package maintainers are not habitually going to be building their packages with staged builds so it's the sort of thing that will get terrible bit rot because one of the things we've done is reproduced list of build dependencies you've got build depends blah and then you've got build depends stage one which is the same list exactly with one package missing or something I did think about syntax that just said accept foo but the problem is that sometimes it's an alternative and not an exception you could invent some new syntax but then you've got to change all the tools to understand the new syntax whereas if you just reproduce it slightly different you can do it to the tools exactly as is and it's very simple and I can write the patches so you said you could write a said script which would generate the change set of build dependencies but that's because he's an evil man so I think it's quite important actually that we have some machinery trying the staged builds packages that have them to see whether they build and filing failed to staged build from source errors somewhere or at least a website so you can check current status so if anyone wants to help with that that will be marvellous because like I say I'm not supposed to be doing this I would expect that most of the problems there will be some problems with in the rules file or whatever but I would expect that most of them will be graph re-untreeing problems the tree will change, the graph will change shape so suddenly a new package becomes important and has to be modified too so you can once you've got to the point of doing the graph analysis entirely automatically then you should be able to spot a lot of the problems entirely automatically without having to go through a full reboot strap order build to discover it so we can start off with a fairly simple mindless thing that just builds the ones we know are an issue but at some point we actually want to analyse the whole archive graph decide what to do and then check that those packages that it's possible to turn the thing into a tree actually I really think what one should do after this working is for example just a bootstrap AMD64 again every few times because it's quite fast to be built we can see it and I really think that once it's really working we should consider to make bootstraping harder should be considered as a very bad behaviour and so if you do it routinely it will eventually let it drop so I think we really should go in that direction and now I give the microphone back to Alexander to move again there was someone at the back there I wanted to say something oh sorry that's exactly the kind of thing we're looking at maybe set this up automatically to do a bootstrap of a new pretend architecture every weekend or something and then submit the bugs clearly that's where we want to get to a certain amount of work to be done before we're there but yes a small comment regarding the breaking cycle algorithm it makes me think a lot about the APT algorithm that we used to install the packages that are in a cycle and I mean all the code should be fairly should be kind of the same it is except that it only looks at install dependencies it doesn't look at build dependencies we have to do that we're mapping back to the corresponding source packages in practice in practice is the same problem I would love it if somebody who understands graph theory and that sort of thing and recursion and stuff were to do that because it makes my head hurt um yes so what else there's some documentation about this and if you want to read it is as ever slightly out of date with respect to this presentation but it's pretty good if you get something like this change you need to write down exactly what it is that should be done so I started writing down exactly what it is that should be done and that's what's on the debut and bootstrap wiki page so anyone who feels this is interesting please read that it doesn't have the patches that I've done there yet but it will have any minute now there's also I said there were a set of very common build loops there's actually about 20 odd I've found so far and the circular build dependency page lists the boring text file of those um the other aspect of this is cross building because in practice the analysis I've done is only for the cross building case sorry before we get to cross building I noticed that almost all of those loops involve at least one multi arch for a package you don't care with what architecture for example tech is when you build the documentation so almost all of this problem with disappeared moment multi arch no because the analysis I've done is assuming that that problem has already disappeared so the much larger problem in worrying about all that stuff will indeed get much better but I've done the analysis for assuming we're already there not every single loop will disappear but I guess like that so the graphs I've showed you are with all that stuff stripped out apart from that first one 10 minutes yeah so I created a new cross tag just to keep track of cross build issues because there's quite a few of those so there is now a tag and user name thing to file cross bugs on and if you go to that URL you can find out about them and there's also a fair amount of documentation appearing as a result of Leonardo caring about this and me wanting to write things down before I forget this is useful if you haven't done much cross building before and would like to get started especially in new multi arch world where everything will be lovely currently it's not actually very lovely it's slightly lovely if you're an Eric than if you're in Weezy but you know give it a few weeks and things really will improve dramatically multi cross builder so this is the same problem as for the bootstrapping issue there's also do packages cross build to which the answer is in many cases no but you can only find out by trying it at the moment it will be handy if there was a build D web page with thousands of packages saying did it cross build or not using the standard infrastructure which has been on my list to do for months and I haven't quite got around to it yet so again anyone who's trivially enthused either just do that or come and see me and I'll tell you how I think it should be done so to open this up a little bit in the eight minutes that are left what do we think about this do we think this is something which should be in Debian generally we should perhaps make it policy that it should be possible to bootstrap the whole thing there are no irremovable circular build dependencies I'm interested in whether people think the staging name is sensible or the bootstrapping name is better and what the whether we should use build options or some other environment variable or even just options there's all sorts of ways you could implement this I've picked something package maintainers so looking at that big graph there's all sorts of places you could break the loops and some of them will be easy and some of them will be hard so we've done a few in fact the ones I've looked at it's usually fairly obvious but anyway if any package maintainers who see that their packages are in the graphs and go oh well it's trivial to just not build it with foo because usually upstream does not have this problem upstream will use some configure foo and if it's not present and it doesn't need it it'll just not bother and carry on doing the build it's normally the Debian packaging which is less flexible than the auto foo underneath so generally this is our problem to make our packaging a bit more flexible in the way the auto conf is as I just spoke about auto foo stuff I've also already consigned I mean we know which build depends we are and we check them we could let the auto foo stuff a bit more open and say well if these ones are missing we just do the right thing just not so much so we don't need to filter everything out so perp sets a direction which might be even helpful in other case where we say you could leave these parts out of the package it will still be correct it will do something but perhaps not the full backend functionality which might be useful for other things are you suggesting build depends recommends it's not depends but basically something build recommends just as a thought I'm not sure if it's good so I think we did think about build recommends and decided that was a little bit scary and we weren't quite sure what to do with it but it is an interesting idea and would be a little bit more like what you're trying to represent so you briefly skipped across this but there are some really awkward packages particularly languages really very hard to cross build or indeed bootstrap or anything are you basically just decided that's too hard to think about now and we'll do it by banging our head against brick wall as usual it is possible to get your language everything in the archive bootstrapped we do it regularly now some of the cases that are very difficult to cross build are simply because the upstream's build infrastructure is shoddy and we should probably just get them to fix it Colin when we've worked in these problems so far we have fixed I think Perlin, Python so far Haskell is resisting sanity but I think most of the major languages in the archive should cross build now to some degree oh it as well yeah but a number of while it's orthogonal to this to some extent because it is within one package and can be dealt with by existing mechanisms we don't have to invent new mechanisms for it but you do have to solve that problem as well yes I mean ideally every package in Debian would also cross build and practice for the moment probably 20% of them do I don't know no I guess there's a lot of fairly simple stuff further up the tree actually it gets a bit easier in many ways because you know it's a standard thing that depends on some libraries and that kind of stuff usually works but yes that is like I said these problems are closely related but they're not the same thing and we can separate them yes so that's it I've finished is there anything else people wish to ask did I make sense I came here late but I just want to add that you showed a big huge graph in the beginning with the ghost script in it and I believe that one is solved yes and thanks to you showing me the horrible graph I'm sorry it was the biggest evil looking graph I had to handle on my computer so I thought illustrated the point even though in fact it's now moot yeah oh well so there's already as Wookie said we have a load of cross build patches already been filed in the BTS as I was going through cross building it was going to be a natty base system and Wookie's been looking at other stuff too for every bug that we found just about I think we have patches ready please if you have one of those patches against your package if it doesn't if you're not convinced it works if you want to talk about it please talk to us don't just ignore it we would really like to get this stuff going yes actually one thing I think would be useful to have an argument was whether failure to cross build should be counted as something other than a wish list bug which is traditionally the status they've achieved in debut at the moment because we go we don't care we build everything natively now I don't know whether if I were to suggest that at the very least they should be normal bugs there'd be any dissent is there any dissent in this room oh that'll do then oh is that some dissent I saw over there do you want my phone well I'd say if it's a package that should be able to cross build easily then it's okay having it normal but packages like languages that maybe need themselves as a compiler to compile for the other architecture if it hasn't been supported previously it might be a bit too much to say it's a real bug it's just something that doesn't work and it's upstream doesn't provide it so that's a little bit of dissent just to make sure we're done a piece of metadata saying whether you expect this to work or not might be useful if someone could think of a sensible way of defining that that yeah actually I think we should handle it like lots of other cases I mean if it's rather trivial to do it then I really think it's a bug that should be fixed and we should really be ashamed of not fixing it which translates to it's an important bug I mean if it's quite an obscure thing where you need to put a few weeks of porting in then it's a bit different I mean if you provide a patch I would assume that the maintainer should apply them anyways which means yes I would rather consider them as important in the normal case but in other cases might be wishlist like please make GAC to do ghost building I'm not sure if that's a really good idea but that's something else I mean sometimes you get a bit of an argument with Upstream who has an opinion about how it should be built which isn't the same way Debian is doing things and that can be Apache stuff for example is an abomination and they've tried to make cross building, cross platform stuff work but they've done it in such a way as to completely break cross building which seems a bit perverse to me but that's quite a long argument because it's complicated I guess I should stop so thank you all for coming to listen I hope that was informative