 So, arm ports. This is supposed to be a boff, so I've diligently not given you any fancy slides. Port status, a little bit of background on what other people are doing, and then we get to argue about Raspberry Pis, and the ARM64 port are the basic subjects I think need to be discussed. So, as you know, we have an ARMEL port and an ARMHF port, and I hope we'll have an ARM64 port soon. ARMEL, for those of you who don't keep up with these exciting details, is ARM V4T instruction set, and EABI, no floating point. ARMHF is EABI compiled for ARM version 7, and VFP3 floating point instructions, and thumb 2, importantly, which is actually 16-bit instructions. Right. So, both those ports still exist. We have the annual question of should we change the ARMEL port to a V5 base? Are there any V4T machines left in the world anyone cares about? Does anyone have any V4T machines they still care about in this room? V4T, which is... Right, of course. I saw someone using an open mock-up yesterday, so there's at least one. As far as I can tell, there's very little to be gained from that change, except that the GCC people have pretty much given up maintaining the 4T part. At some point, something will probably break. But until that does, I can't see much point-changing things. But if anyone's desperate for that extra 3% to whatever you get, say so, and come here. They're probably... So, increasing nobody else in the world builds anything with a V4T, so we will find things that break. So, yes, we can save ourselves some aggravation, I think, and that will increase over time. So, it's a trade-off between, like, yeah, how hard is it to build chromium versus how many people still use their open mock-ups? Have we got... Anyone have a poor soul who has to actually make chromium and build an arm here? No? Good work, sir. Yeah, so I don't have any strong opinion about that. I'm inclined to leave it as it is until somebody bitches really hard. But, you know, increasing the... I don't use my open mock-up anymore. It is getting to be, as I say, mostly because the toolchain people aren't taking much notice. It will become painful probably over the next couple of years. So, I just mentioned the other distros. Just so you've got some idea. So, Ubuntu, Fedora, and Sousa all have RMHF ports exactly the same compiler options as ours. And everybody's building ARM64 stuff too. Fedora are furthest, I think, with a mostly built archive. They got past QT recently, which only takes two weeks to build a model. So, it's very annoying when you get most of the way and it goes wrong. They can only build natively, so they've got thousands of models building things astonishingly slowly. Ubuntu port. I did a base image. We'll get onto that in a minute, actually. And Sousa have also got quite a lot of packages built. Posing secret source, which I can't remember is still secret or not. I don't think so. I think they're using QEMU rather than ARM as a model, which is noticeably faster. But in the way of corporateness, that's the one little advantage they've got. So, they're carefully not distributed to anybody just yet. They'll have to eventually. And then there's the Razabeean people, which is ARMv6, vfp2 floating point destructions. But exactly the same ABI as everybody else. So, leaving that argument aside for a moment, ARM64, a port bootstrap. ARM64 is going to be a big deal. Right now, today, hardware exists, but is much rarer than rocking horse shit. I don't know how many boards there are, but I know everybody wants one, and very few people have got them. But that will change rapidly over the next few months. So, next year, there'll be plenty of hardware. I hope, assuming some of it works. I think there have been some issues with the very first hardware, mostly working. But I did see a demo of it running, like decomposing two movies at once inside Zen containers. It was all very fancy and really quite fast. Does work. So, the Debian port of ARM64 is not at all pretty much at this stage. So, I did load of work, but basically because of the freeze and the state of multi-arch and the cross-building tools, I had to do it in Ubuntu, which is annoying, but such is life. So, there is an Ubuntu bootable image, which you can download and run in a model, or on some hardware if you were so lucky. In Debian, we haven't even got a cross-toolchain yet. I keep trying to build one, and it keeps not working. This is very annoying. And if anyone would like to help, why I can't get a compiler that actually looks in user-include, that will be great. So, nothing's happened since February on that front, basically because I stopped doing it and work at Lanaro said, no, no, stop bothering with that. We need you to do this other stuff instead. The distros will do it for us, they said. I said, I have it, Debian won't, you know. I think we are the distro. So, oddly enough, none of you stood up and spent your lives cross-building ARM64 things very slowly on a model. So, nothing has happened since February. Apart from Docko, to be fair, actually, who has done some more stuff, but that's Ubuntu. So, there isn't any hardware yet. We've always go well until there's some hardware. Why should we start? So, there'll be some any minute now. We probably need to start. So, cross-toolchains is the first thing. That's the main reason nobody's done anything. They go, well, there's no cross-toolchain. I'm not building one. When you give me one, I might do something. So, I realise that badly needs fixing and I failed to do it on the train on the way down all this week, but real soon now. What else happened I mentioned? There's a port page which contains all this information again. There's a bug list. There's about 15 ARM64 bug fixes that haven't been applied. So, apply your pit patches. And a big pile of multi-arch patches which haven't been applied. Some of them are well over a year old. Please apply those. I suspect that the people in this room probably have. It's all the people who aren't here. Here we go. I don't know what that's about. I don't know if I've ignored it. Why should I apply that? Nearly all of those patches are updateyourconfig.sub. Turns out that's 60% of a new port work which is a ridiculous state of affairs but that's how the world works. Perhaps it's tried to fix it upstream but they've been quite resistant. So, yes, we need to set up a Debian port box. This has been helped by Huawei. Huawei are our best mates, it turns out. They recently joined Lennaro and went, where's my Debian? We want Debian. None of this is a bunto nonsense. Our customers want Debian. So, they're big and important and have said so, so suddenly. To be fair, everyone's gone, well, you've got a bunto. You really need to do Debian work as well with a lot of things to do. You need to write. And also, the reasonable point that Lennaro is not a distribution so, they try not to do the distributions work, they just try and do the upstream work. How is the name of the company spelt? Which company? Lennaro. I know how to spell that. Huawei. HUA HUAWEI HUAWEI Write it down. Oh, right, yeah. No, I closed that window. I think about it up here, isn't it? I could save it online as well, couldn't I? Yep. How do we know whether it's working? I don't know. I click connect to a server. Oh, right, yeah, okay. Sure. You've got a copy. Not the copy scares me or anything. Oh, come on. I'm clicking. Can I disconnect and reconnect or something? Does that help? Subscribing says. Someone else could bring their laptop here. That actually works. Is anyone watching IOC, by the way? I've just noticed people. Right, where was I? Yes, so Huawei have said that they wish Debian stuff to get done. Which, with a bit of luck, means I can get back to doing this in my day job. Which means it will go a hell of a lot faster than it does otherwise. Is anyone here who is specifically interested in helping make the ARM64 port happen? Yeah, yeah, right, yeah, okay. It's fun, honest. It's quite cool. Yeah, okay. So, everybody can download the free beer thing and you all have hardware. It's just slow. It's not infinitely slow. It's only like crappy ARM boards. Oh, wow. It's not infinitely slow. It's only like crappy ARM boards. Oh, wow. So, what else do we get to? And the other thing that's slightly holding matters up is the build profile spec. So, this was mooted last year by me and Johannes and people went, we don't want to use up a meta character. That would be a disaster. Please think of a different scheme. So, I'm not going to go into that too much detail. There's a wiki page. But if anybody objects violently to square brackets instead of angle brackets, please say so. I want, by the end of this week, anyone who's going to object to have objected because we need to fix that. I sent you a mail last night about how I thought we should proceed. So, we're slightly stuck on getting the deep package people to agree, as usual. Because the kind of best way to implement it means changing the deep package API a bit. So, we're going to do that. The deep package API a bit. And they don't like that, which is fair enough. So, we'll probably have to implement it in a slightly less flexible way, which would get things done. So, the build profile stuff allows us to do group strapping automatically, which there is a GSOC project running again this year. Johannes is doing an excellent job of actually mentoring of my assistant mentor. And Alkmim in Brazil is doing the work because I did an awful lot of hackery to get this done the first time. And we'd like to have an actual mechanism you run that says please bootstrap this port from nothing. Check it still works today. Automatically unroll our circular dependencies as I've talked about many times. So, in the way of Debbie, and it takes three years to work out from working out how to do something to actually getting it in, but we're getting quite close, I think. Unfortunately, we missed getting that into Weezy, so there'll be another cycle before we can actually use it in the archive. So, yeah, the R64 port is there as I say, if I'm allowed to carry on working on it I will do that, otherwise stuff will happen very slowly and basically we get to rely on Ubuntu. It's probably easiest at this stage to actually use their bootstrap and rebuild everything. The only problem is that they're on to really see 2.18 now. I'm not quite sure what happens if you use a 2.18 image and then try and rebuild all of Debbie and it's probably easiest to just move us along as well. So, the Raspberry Pi argument. So, there's those bloody Pi people with their stupid ARM V6 proprietary platform that irritates us all and the best marketing machine in the world by several orders of magnitude. So, they've sold millions and we did our best to ignore them and be rude about them. Sadly, they didn't really work and quite a lot of them are using Debian effectively because they're all running Raspbian which is a fine piece of work I should congratulate Mrs Thompson and Green I think, you've done all the work there. Sadly, they're not here but hello out there on the internet. I don't know if they're watching. Here's a plug wash. So, you know, we can just carry on like this going well it's not right and we don't support it very well. They should run Army L nothing wrong with that or we could try and do more. There are an awful lot of potential users out there who are getting into the joy of ARM Devboard fettling and might be nice to bring them a bit closer. There's various ways we could do this. So, the most obvious one and what we always do and have done for years is build for the lowest ICER within an ABI that is popular. Hence, we still build for V4T for those OpenMoco weirdos, right? Only about 12 of them. So, going, oh, we're all going to build for Raspbian seems a bit perverse. We should just build ARM HF for V6 VFP2 without thumb. It would go slower. I am not sure how much slower. It will be extremely useful if somebody did some research and said, is it 10%, is it 15%, is it 1% The thumb 2 thing probably makes most difference. Actually, the ARM V6 versus ARM V7 is not a very big deal. There's a few extra instructions. The optimisations are a bit different on the V6, V7 core and I guess, again, VFP3 versus VFP2 has a few extra instructions. But thumb 2, the code is half the size for effectively the same code base. So, that does save you a lot of cash pressure is why all the phone people demanded it in the first place and why we've now made it to the default. But I still don't know how much difference it really makes. Here's Colin who might actually know. Colin, excellent. We have a question for you already. Do we know how much faster VFP, ARM HF has done everywhere is than without thumb 2 and VFP and ARM V6? No. So, it will be interesting if someone actually did some tests. Effectively, you just run the Rasbian binaries and ours on the same box and tell us how much difference it makes, not very hard. All the work is done for you. Now, we would then be slightly incompatible and I'm not quite sure what sort of breakage that gets us. Anyway, so there's that option. Just build for the lowest common denominator. We could invent a new arch. I personally think our architecture is really ought to be about ABI is not about instruction sets, but we could build both and call it ARM HF-V6 or something. That would work, it's not very difficult. It's a matter of adding one line to the package, rebuilding everything. And then dealing with all the things which make V7 assumptions because we've told the world that's the right thing to do for the last couple of years. Or we could actually do slightly more involved things to basically make it possible to use ARM EL with a VFP2, so optimised binaries for things. So ARM EL allows there to be no floating point. It doesn't require there to be no floating point. The whole point about the difference in the calling convention is that you can use floating point instructions if you've got a floating point unit. So you can make ARM EL binaries that use VFP2 and they will run nicely on your box and use the floating point unit. What we don't have in Debian is a mechanism to actually you could probably just build a repository of stuff. It's getting de-packaged to go oh I've got these extra hardware because I should pick this package instead of that package. That's the piece that's always been missing to do this. We thought about doing it a few years ago mostly in the Naro context and went that's a lot of work and we didn't really care enough so we never did it. I was really hoping that Raspberry Pi people would implement it that way. They had a big incentive to do it properly. But it was much easier for them to go off and rebuild it all in a different bucket and call it the same thing. So that's what they did. So yes, there's that. The thing there is does anybody care enough to do that work because that actually involves some work and it will take at least a year before you get it into real Debian and you've got to go through another cycle because it's a de-packaged change, blah blah blah so by the time they're finished maybe we really won't care about Raspberry Pi anymore. And yes, you could argue that anything we do actually maybe the world will start taking notice and start using QB boards but I suspect there's so much momentum behind this juggernaut that will go on for some time. Or we could just let the Raspberry Pi people do the fine job they're already doing. Does anyone have any comments or opinions on with this problem? Hello? So I think we should try to ask why was it easier for the Raspberry Pi to do it on the side instead of using it the Debian infrastructure so that it doesn't happen in the future? Mostly because we told them to get lost. Okay. So that's easy to fix in the future. We said we're not changing our base ABI a base instruction set because all the distros have agreed this one and you're weird so they took the answer reasonably and went okay we won't try and do it in Debian so we could have been more accommodating in the beginning and said let's just turn it down but I think the right answer probably was to say we're not doing that because we've gone to a lot of trouble to get everyone to agree on a baseline so yes, that's why it wasn't at all technically difficult to just rebuild ArmHF for two guys did it But would it be possible to get these two guys to move it to Debian as a new architecture? Well so it's a new architecture that's slightly harder because now the package has to have a new architecture added so a new architecture is just a new ABI a new instruction set sorry and yeah it's not technically that hard to do that but it won't be real Debian again for another release so yeah and that's probably the easiest thing to do if we wanted to bring it inside we just tell the package so they'd get all their packages changed from being called ArmHF V6 or something which they may not be that enthused about at this stage because now it's a transition for them yeah so I have had people emailing me saying we really need to fix the Raspberry Pi problem and I tried using Raspbian but it's not actually that good because of things that aren't in there and once we maybe sort of a partial architecture thing out or we have a system similar to the V6 I think maybe V6 or something for optimising some stuff on the use of the new instruction set we can do this all within the same ABI so I think that's the thing to remember if we compile for a slide we'll be able to review and then compile to the ABI that we have 2,000 ArmHF and that's the message for the time that's there I think everybody will read it and the possibilities for other interesting things to come out of working together so for that to happen someone needs to do some speed tests just to see so we have some idea how much a hit we take there we've always traditionally picked the lowest thing and not worried too much about the 10% speed it cost you it's annoying going backwards but such is life and we really ought to fix the we don't have a good way of picking packages according to my hardware capabilities so that we could rebuild the stuff that matters for faster optimisation so interesting that the x86 people have had this problem for years and we've just tacked round it by inventing mPlayer i686 in packaging and LibC i686 which is kind of hacky really, we should do it properly but again it's all very nice saying we should do it properly, who is volunteering to put hardware capability flags into d package, it's not very difficult we wrote a spec at UDS about three years ago we already have hardware capabilities in the kernel that's how the MMX or SSE or whatever it is works on x86 do I have these instructions so we just have the same flags in arm now at the moment the flags the kernel uses aren't quite what you want but we've already asked the kernel people and they said send us a list of flags we don't care it's easy so that's effectively the set of flags you want to store in d package to say this was built for these options, does it match the options running on the box in front of me if so use that optimised package and then we just have to work out how we layer our archive just that would mean that the architecture would be kind of a two-stage thing so you'd have packages for explicitly built packages for all the different versions or would they all go in the package you'd rebuild some packages where it actually makes a detectable difference for a different instruction set or floating point option and then you'd have a mechanism so it's basically partial archives so it's having an overlay of part of the archive so that you put both in your app sources list and it picks them all we have magic in the archive so they kind of appear as an overlay without you having to say so explicitly because they have proper hardware which is all v7rmhf they can just run rmhf works perfectly it's a special and have v6 hardware that's not true there's shivaplugs of v6 is that right or are they v5 well whatever came after a shivaplug a dreamplug is that of v6 well there's a few other v6s out there there aren't very many of them which is why it got more or less skipped in the world of watts instruction sets should we build for that sorry what you missed the beginning open Marko people but you're right as I pointed out there are many fewer of those than there are Raspbian and we cater for that so by the same token we should cater for this so well I don't care except in principle I'm doing arm64 things so if anyone wants to see this move forward we can do some work plugwash doesn't think much of a new architecture name ie rmhf-v6 so that leaves downgrading and having a partial archive I think so yes what we really want of course is to get some of those people to come and actually do this work because there's millions of pi users some of them must be competent I know they'll make them competent they'll become competent over time that's the whole point that's how this works most of us didn't know how to build things for different arm APIs a few years ago other things just before we run out of time arm kernels now there really is an arm MP kernel which builds for multiple platforms so they're trying to make arm world a lot more like x86 world you will soon be getting UEFI on your boards instead of uboot and crazy other weird bootloaders actually we've pretty much got rid of the crazy other bootloaders microphone already you'll soon be getting grub as well because especially arm64 is driving this but there's already arm32 servers you can get a 32 blade arm server thing at great expense stick in your rack and they want those computers to look exactly like their old computers US UEFI and grub for booting which is actually great for us distro people because it means every board isn't different and we haven't got to have a whole lot of mechanism in DI for whatever crazy shits needed on this particular board much as UEFI smells is at least consistent and you can do sensible things with it sorry just a question I don't know if you know the answer but in the secure boot session on Tuesday whenever it was you know people observe the well-known thing that Microsoft's logo guidelines for arm forbids the user from installing your own keys now the assertion has been of course floating around that most people who are dealing with Linux based arm devices don't care about the Microsoft logo guidelines because they're on say a Windows surface or something and we don't care but do you have any feeling for what people who are actually making arm based UEFI devices that we do care about are doing in this regard not really so all the ones doing kind of little dev boards you know the cubie board people and eoma PC 68 whatever it is all that sort of stuff they're not very enthused about UEFI really they were quite happy with their U-boot and they didn't really want to have to change and I suspect that a lot of those just aren't doing secure boot at all exactly and not them to care less about Windows although there are certainly people who care about the secure boot when the user is in control of the keys so the dev board people I guess they don't really care at all but the people making anything kind of commercial server boxes and Chromebooks and that kind of stuff I guess they do care so I suppose Chromebooks Chromebooks are fairly locked on aren't they yes and in some ways it's worse there's a nice developer switch but once you flick the developer switch I think it bleeds all your data and you have to start again okay that's sort of understandable I can see why they were forced to do that but yeah so yeah I'm not sure was there something behind your question like what how much do we have to care about that problem just curiosity about where there are assumptions that it will probably be okay that we won't get screwed on secure boot and are it's just a bit of a question that's a bit of a question I think the problem is checking whether that assumption is actually valid the thing that's going to happen is in a couple of years time you'll actually be able to buy half decent arm laptops I mean a Chromebook's already quite a good laptop apart from having a peculiar keyboard there will be more of that sort of thing and you know those people will probably follow the windows rules so that you could run windows on it maybe that's how it'll ship and in that case that would be very unfortunate for us we're just going to be knackered for those devices just like all those thousands of windows phones and things which basically nobody's ever been able to get booting Linux because it's too hard so yeah there'll probably be quite a lot of hardware which effectively ends up in the Tevo class sorry 10 minutes right what I was wondering is I'm not sure how much work we have to do in Debian to make multi multi-device kernel, multi-platform kernels work the idea is that your device supplies a device tree with its bootloader and all you have to do is load your multi-platform kernel but right now that doesn't work because you generally don't get a device tree in the bootloader and even if you did it will be wrong because they keep changing it I know that people have got so I mean we've got Calceda nodes working on Ubuntu as our new buildies I don't know what their bootloader setup was and how much effort was involved there I believe that they did work with essentially the distro multi-platform kernel so there's some hope there I think that work is ongoing in Lunaro we're all trying to make this work so that there's actually a very small number of kernels and from Debian's point of view this is marvellous we don't have to build 15 ARM kernels that makes our kernel build much faster of course we'll soon have super fast ARM hardware so we won't care so much anymore but it's all good but there's probably some work involved in making it also at least for OMAR it's important anyway because they shut off their old support so they need device tree even without the multi-platform kernel right so if you buy Beaglebone Black does it come with a device tree that actually works with an arbitrary kernel anyone tried it ok we don't know I probably know someone in Lunaro who knows I guess I should ask them before I run out of time I should mention there's going to be a mini debconf in November at ARM's offices in Cambridge Mr McIntire has made this happen and ARM are coughing up some facilities and money so there's a wiki page there so far me and Steve are going I'd like to come please sign up we probably need some talks and stuff yes exactly so it's a mini debconf please come along and give interesting talks it'll be four days two at the weekend two during the week basically we get kind of a room during the two days it's work basically we can fill the atrium place at ARM this is very nice and yeah it should be cool so I guess there will be a focus on the army things and this kind of subject please microphone over there I just wanted to say that if you need funding to be able to attend just talk to me cool we're hoping to make ARM pay for it all but I think they said they wouldn't pay for the food or something so I'm not sure there's some bits they've got pots of money they really ought to be able to afford I've seen the figures they don't charge very much for each chip but they have sold a lot is there anything I've forgotten we have five minutes is anybody actually going to do anything about the rasbian problem volunteers looks at IRC okay well it might change it might not I guess come back next year for yeah I've read that unless it's changed well let's say it excludes one of the options for how we might change things which would be inventing a new architecture called HFV6 even though it's quite an easy option they're saying they don't fancy that much fair enough I wouldn't either at this point I guess the other thing we didn't really cover was if we could do more for rasbian just leave things as they are with this sort of external port because they'll be able to change any infrastructure anyway that just pretend it's more like real deviant you know it's a derivative it's based on the practices in our with the instructions how would you go about that to prepare everything in a port to move it over you'd just rebuild each package to go along Ubuntu who have gone back from v7 to v5 so they can probably tell us just what breaks so that was for our ARM EL port which probably shouldn't have been bumped up to v7 in the first place but it was before ARM HF existed and some companies you might be familiar with were pesturing us about it the so we decommissioned the our ARM EL port in in rearing that's 1304 so 1304 and onwards do not contain ARM EL but we thought that before we did so it might be quite nice to back rev it to to ARM v5 ARM v5 ET I believe so quantal men so 1210 men we rebuilt everything in in men components so just sort of supported packages against for ARM v5 ET with the intention that people who wanted to start from there and do something useful might be able to do so of course they're stuck in an unsupported release so soon to be going out by and large things not too many things broke but by and large it seemed to work it's quite hard to tell because nobody was using it at any point so I guess the problem actually from our point of view is we use our standard mechanisms we don't get a rebuild until we get a new upload you can bin enemy right anybody who's want to build access can do that so yeah so the point is you could just rebuild as you go along because it's forward compatible all these binders will still work just tiny bit slower so yeah you just rebuild the whole damn thing until you've finished some of those builds will in fact if done on a v7 machine which they will be will come out wrong haven't actually gone back to v6 because they ignored the compiler defaults and set their own and you will have to catch all those but it's not hard to I think some tools have already been written to just run through and look for v7 instructions so you just run it through another lintian shape that says does this have instructions it shouldn't have technically it's not at all difficult to do that part the technically difficult part is the being able to partially do rebuild some of the archive as it is now for optimization purposes and have both available to someone we don't have that capability at the moment because the package couldn't care less about instruction sets, optimizations it will work for libc and mplayer because they've done it in the packaging I don't know what else, how many other packages are bothered could do that I suppose easy yes true yes so it's true there is a functionality called ifunc in libc which allows you to build multiple versions of a function and then it will choose at load time I think once which one it should use for this hardware so effectively it's kind of fat binary thing so you haven't got two copies of the binary, you've got two copies of some functions the problem mostly is that you have to rewrite your software to use that and do multiple builds with different options when you build it anyone can do that anywhere that already works so you don't need any Debian infrastructure for it so that's using multi-lib doco can tell us how hard that is you just have to change the packaging for every package I don't make this so you're right, that could be done too and maybe it's not even craziness we've run out of time sorry thank you very much the best pill everyone goes how many people think we should downgrade armhf to v6 how many people think we definitely shouldn't more how? twice as many we're not entirely persuaded but there is some thought that maybe who thinks we should do something to be nice to the Pi people we would like to be nice to the Pi people but we're not sure how thank you very much