 The recording is now going. Hi everyone, I'm Peter Robinson. If you don't already know that, I think most of you probably do because I have a big mouth. I'm here to talk about the state of ARM, both ART 64, the 64-bit variant and ARM V7, the 32-bit variant. My slide deck is fairly straightforward in that it's just got subject headers and I will basically go through it relatively quickly. Feel free to button at any time you like, except about device support. We can do that at the end. So core tool chain and features, actually not much to say about this anymore. It used to be, you know, we had GCC and not much else and then it was web added functionality. Now we've got everything from Golang and I think even Rust all the way through. The only big set of tool chain that we're currently missing on ART 64 is Mono. Since the Microsoft purchase, that has all landed upstream and we're just waiting for basically the release with all those patches actually in it. And I think be around in the F25 cycle if not, you know, it's not far away. So the first time I did this talk with ART 64, it was this massive big length of exceptions and now I'm happy to say it's not. And on top of this we have, on top of say the Golang stack, we now have Docker and I think Rocket, although I'm not sure about that, Kubernetes, OpenShift and various other bits of the Golang stack. So and in Fedora 24 we now ship ART 64 and ARM V7 Docker images out of the box like as in the base images. We're looking to work with Adam Miller, who I don't see, to get OSBS and Docker layered image support in there as well. So that probably won't be quite the same time as X8664 but it shouldn't be too far behind it. Server edition, again very, very boring. I think probably the biggest news here is that as part of the Fedora 26 cycle, which is now running, we're going to propose promoting the server edition to a primary architecture. So there'll be discussion about that sort of thing probably going out on the mailing list in the next couple of weeks as part of a slightly different proposal which I will mention later on. Cloud edition, alongside the Docker images, we're now also producing QEMU-based cloud images and as part of the new compose process that was introduced in Fedora 24 for primary, which we are running on across the secondary architectures as well. We're doing nightly cloud images, nightly Docker images and so from that perspective again, nicely boring, looks a lot like X8664. Workstation edition, a little bit more interesting here. All the stuff's there, on R&D 7 it runs. We even now have accelerated graphic stats primarily on the NVIDIA stuff. So at home on some of the Jetson devices, I have a fully accelerated Wayland GNOME 3 stuff. AR64 is a little bit more interesting in that there's not really that many devices. The GNOME team have lent me a Dragonboard which we're basically working to get and that should run GNOME out of the box. I have NVIDIA sent me a Jetson TX1 which is a 64-bit processor with a Maxwell GPU in there and we should be supporting them in Fedora 25. So from an AR64 point of view, they will be our first sort of fully accelerated desktop devices. Similarly with R&D 7, there's a lot of 2D stuff that works really, really well, 3D on a handful of devices here and there. So yeah, relatively boring but no when. So what do you mean by electrical bolts? GPIO plus any graphic support device. Is any device in Fedora best? On AR64, probably at the moment full desktop and GPIO, nothing at the moment. In the Fedora 25 cycle, we should have support for the Jetson TX1 which is the NVIDIA one and possibly the Dragonboard which is a Qualcomm one which has a 96-board form factor which will do fully accelerated 3G but it has, the major issue there is it has a terrible boot loader. So it will actually run and boot and there are people running Fedora 24 on it with fully accelerated desktops but there is vast amounts of gaffer tape and bubble gum and various other bits and pieces applied between the boot loader and the kernel to make that work. And someone actually posted a remix on it and I'm not sure the kernel they ship but it's a weird one. So there are stuff there that will do that and do it relatively well but not like beautiful. In Fedora 25 we'll also probably be able to support the Ojoid C1 and potentially some of the Rockchip stuff but that won't be fully accelerated 3D stuff. It'll probably be more very basic or marginal accelerated XFCE style desktop stuff. And you can always grab one of those server boards, put Radeon or an NVIDIA part in it and it will work. I have one. Yeah but so the Mustang board which you're referring to there doesn't have generally usable GPIO that you can use for sensors or whatever is that you wish to use it for. Did you say you wanted to defer a device to support discussions? Yes. Thank you. There is another device that I'll mention later on as well. So general user space, yeah pretty much complete. 90, so I think of packages that we don't currently build on AR64 that we build on primary architectures there is something like 120 out of the 18,000 in Fedora. So it is less than a percent of all the packages and there's sort of things like prologue and Pascal and a few other bits and pieces like that which potentially are possible to support but nobody's really shown the interest to do the bootstrapping requirements to do so. But it's tiny. So I think about 0.5% of packages that don't actually build on AR64. Arm V7 doesn't have that problem. So Pascal totally supported it? Yes. We've had support for Pascal in there for probably about three or four releases. It wasn't quite as accelerated but Jens who is here somewhere worked on it and it is now I believe fully accelerated. Kernel, again pretty good. A few releases ago we used to carry a patch of around 100,000 lines which was we currently have one patch set for AR64 which provides PCIe ACPI support which is needed for some of the enterprise hardware like the Moonshot chassis. The vast majority of that patch set has landed in or is landing in the 4.8 kernel which is in the current dev cycle and so we'll end up with I think around less than a thousand lines of patch. In terms of device support it's improving a lot but again so is the availability of actual hardware. So it's relatively straightforward, relatively reasonable. It tends to work in most cases. Some like the Odroid C2 and other devices are still sort of dealing with upstreaming stuff but are getting there and improving a lot all the time every sort of new kernel cycle. In 12 weeks or so with the new kernel, 10 or 12 weeks we tend to add quite a bit more but it's all relatively straightforward and the main Fedora kernel team doesn't generally hate me so much anymore for having to deal with 100,000 lines of patches. Yes! I'm not the most hated anymore! Bootloaders. Speaking of things we hate. So there's a couple of major bootloaders that we support. Obviously on ARX64 it is the lovely UEFI support. One of the things that I kid you not I was working on or with a lot of other people working on for about four years was there is an open source UEFI implementation called Tiano Core but it had a implementation of VFAT that was licensed and patented and numbered and hence we couldn't ship it. And we were kind of bothering a whole bunch of different people to try and get either that licensing problem solved or a different implementation in that could be shipped and interestingly I've mentioned them once before already Microsoft as part of I believe the whole agreement that they've done with the open source community and in conjunction with Intel actually fixed that problem so that they could basically ship UEFI or that version of UEFI stuff on OpenStack and various other bits and pieces. One of the nice side effects of that is we now have a firmware that works out of the box with VMs and other such things that we can ship in Fedora. So that fixed one of the problems. So Uboot is obviously the other bootloader that we use a lot on the SBCs such as Odroid and Chip and various other bits and pieces and there was an implementation of I suppose you would call it Immulation of UEFI that will or UEFI boot services that runs on from Uboot to enable us to boot a standard UEFI based grub and kernel from that it kind of works but not quite that well I believe the problems are now fixed I just need to get the time to actually sit down and hack on it and confirm it and finish up working. Once that happens it makes it much easier for us to support things like the Pine64 the Odroid C2 and various other devices with the standard boot installer Anaconda stack that we already have supported for a number of releases on the enterprise level 64 devices which obviously then makes the user experience much much simpler and much much easier for a lot of different reasons. So one way or another how high water I intend on having that working to some degree for Fedora 25 again I'll come to those more details about those particular devices and as part of getting that working we can then start shipping the standard disk images that you can DD onto the single board computers similar to what we do for R&D 7 So we're expecting I'm expecting to have that one way or another in one form or another as part of Fedora 25 I wanted it to be there for Fedora 24 but from a release engineering perspective Fedora 24 was immense and they just really I just literally ran out of time where it's like I actually have to be able to sleep at some point so that was one of the very very few things that we had to sleep in Fedora 24 sadly because I just didn't have enough time and cycles to get it done. Do you envision people like a workflow where people would actually be able to just put a little stub on the SD card and you could run out of condo on these devices? So the question is for the video do I envisage a situation where you can put a little stub on an SD card and run out of condo yes you can do it now you've been able to do it pretty much from the outset so on a number of my test devices at home that have SATA ports on it I have Uboot on the SD card and it just sits there and it pixie boots off the network and does a kickstart install directly onto the hard disk using anaconda so that's been a standard there are a few caveats and if you've got a device that has onboard NAND where the Uboot is running on the NAND you can do that to like SD cards there's a few corner cases where it's a little bit iffy like if you want to if you want to run the Uboot on the SD card and install to the SD card it tends to blow away the Uboot and the way people have worked around that is then as part of the post-install process they rewrite the Uboot out in the kickstart as part of the post-install process but it works now and it has done on RV7 it's worked like that since about Fedora 18 I don't think a lot of people know that I would like to learn about from you then I'll write a blog that would be awesome thank you and you're on video yes agreeing to it so AR64 is a primary architecture this is a question that I get asked a lot like three or four times a week a lot I am in the process of doing a proposal which has a Fesco ticket and we'll be going out to the mailing list probably later on today because there is now a mostly organised Wiki page with an FAQ and various other bits and pieces around it of redefining asking Fesco to redefine what constitutes a secondary architecture or as I'm now more referring to it an alternate architecture so at the moment the designator for primary and versus secondary is the instance of Koji that it runs in and that doesn't really hold true anymore with a bunch of I686 being sort of pushed to secondary architectures in various different forms and in the case of AR64 for server and even Docker and certain components it makes perfect sense to promote it to primary because there's a lot of people that want it as such and are using it as such and have an expectation of it as such but then the promotion of say workstation on AR64 to primary makes no sense at all because there's not a lot of out of the box devices that even work on it so then watch out for develop mailing lists basically the proposal is that we run one instance of Koji that builds all the architectures and then the artifacts that are output whether it's the workstation live images or the server installers or the Docker images or layered images or any other thing that we may end up producing is the definition of what is primary and not I mean a lot of the components of the proposal are already there and are already so we have like in the case of release artifacts we have blocking and non-blocking release artifacts and so that then basically becomes what is primary and what is secondary and you don't then have a primary architecture and a secondary architecture you have x8664 as a default architecture and then you have alternate architectures which are IBM Power AR64 even on v7 to some degree and then certain components of the release so that is a proposal that's going through and once that is accepted and those changes have begun to enact I will then propose promoting AR64 server to a primary release blocking artifact as part of that process so that's because just so that it's recorded and on video and so I can now refer people to a 5 minute presentation every time they ask me as to what is the state of AR64 becoming primary that is currently the state so they'll be I'm basically preparing to put on my asbestos suit and go into the firefight that may well be the developed discussion over that and from there we'll take it from there so I mentioned I would leave questions about individual devices until the end one of the reasons for that is I'm sure someone's going to ask about Raspberry Pi because that is the other question that I get asked hundreds of times a month will it be free in 4.8 gets not 64 bits of I know it will be fun so we actually support Raspberry Pi in Fedora 24 on the Raspberry Pi 2 and the Raspberry Pi 3 and it all actually works the only thing I never got around to doing alongside the AR64 disc images was actually producing an image that you could just dd onto a card and have work because you've got to have this horrible little VFAP petition at the beginning and you have to set it up in a special way and I can do that on my machine at home no issues like generate the image the tool chain that currently builds the nightly images in Koji is an antiquated pile of crap which we are working to replace with live media which is part of Lorax and the Anaconda stack the problem is it's and we made that change for the live CDs in Fedora 24 the intention was to make that change for the arm disc images as well in Fedora 24 but there was some functionality that didn't work and so my intention was to do unofficial Raspberry Pi images that people could consume on arm V7 for both the Pi 2 and the Pi 3 and I just not got around to them yet but we do actually support the Raspberry Pi 2 and 3 in Fedora 24 in Fedora 25 again one way or the other we will have disc images to enable that support and that should have fully accelerated GNOME stack with GPIO and that could be the solution that you are after I want to use that on the arm device so I can actually do some stuff please hand for you one of the major blockers I had for sending it out and blog posting it and like media releasing it and everything else is that I wanted people to be able to to be able to DD it onto like bootstool graphical login and so HCI works and I don't want them to have to have a serial port to work stuff out to poke stuff to make stuff work because we are going to end up in a situation that as soon as we announce this there is going to be like 10,000 downloads and I don't want 10,000 people coming along and drowning the few people that are actually there to help for support so I basically almost want it to be use of proof I was going to use another word but yeah well now you are equating those two things I didn't say I didn't know if one of your concerns about that is that you can't retroactively add the image to the mirrors if you don't want another image in the mirrors just to plant an idea write a docker container that pulls down the existing F24 image which is okay now we are going to cross call and windows and map people which there are a lot of so the image that I have so the image that image I have at home is a single image so we would produce one for each to say XFCE KDE GNOME etc etc that for the Raspberry Pi use case you can literally on a Mac or Windows or anything that has DD DD it out and it will boot by default out of the box on the Raspberry Pi then if you wish to use it on a BeagleBone or any of the other devices any of the other 180 odd devices that we support in Fedora 24 you use the Fedora image installer which is the standard way any DD is out the U-boot and other bits and pieces and it will then boot on those devices in the standard way that everyone is used to and then the only difference is that those devices will have like a 30 meg VFAT petition at the beginning of the disk that they never need to deal with or look at or care about and if they wish to delete it manually themselves they can but otherwise it will just sit there and basically be ignored that's awesome no no I've literally thought this through for a couple of years to work out the simplest solution so that any Raspberry Pi user can try and just do it on a Windows or Mac or whatever and it's easy and it just works but we don't kill the other users and we don't end up in a situation where we have two images of everything because that's just confusing which comes to the my intention is only to support the Raspberry Pi 3 as an RV7 device so that there is one image that if people want to run it on the Raspberry Pi 2 or the Raspberry Pi 3 it will run like that which is exactly the same way as the Pi Foundation is doing it and the fact of the matter is that a device with one gig or one gig of RAM with a broken USB controller and like a single broken USB controller that does all I.O. is never going to be performant enough that a 64 bit operating system is going to make a shit of a difference so yes the 4.8 kernel supports the 64 bit version of the Raspberry Pi 3 we will probably enable it just so people don't bother me but it won't be it will be there and people can consume it but if it breaks they get to keep both bits and the officially supported one will be RV7 that's awesome the same user question so what are the devices you would suggest our users to buy now if they were to try out new things to do do you have any suggestions for the users so the question is what are the best devices to buy now for people to use and the question I always have when people ask me this is what do you want to use it for desktop users what's that one so desktop users are Jetson TK1 because that will give you fully accelerated Wayland desktop with all sorts of nice things it is not a cheap device though it will cost you $200 or $200 what about poor users so any of the all winner A20 devices so the QB truck is quite an expensive version of that the Banana Pi, the Banana Pro are very good devices and they are very good for server related stuff because they have an SOC attached Gigabit Ethernet port they have a real SATA port so if you want to do Gluster and stuff like that you will actually get real performance as opposed to on the Pi where you get no performance the Beaglebone if you want to do IOT related stuff is awesome and we support a whole bunch of stuff on that and it's very well supported the Wandboard for a bunch of different use cases and there is now an accelerated driver which should do full workstation acceleration none of the user space stuff has been released yet and the people that are riding that stuff are too busy feuding within their own little community to actually get anything done so the Wandboard and a bunch of devices like that are pretty good for XFCE style desktop and audio related stuff because they have digital audio and digital audio through HDMI that works pretty well so it's a hard question to answer because we literally now support about 180 devices and depending on your use case and what you want to use it for depends on the device I would recommend I would recommend that the Snapdragon has potential as well because it has been for desktop use cases so that was the Dragonboard that I mentioned and I have completed another horrible, horrible, horrible bootloader experience I did try the the one where I gave you I did try the Damian image I think it's a Damian image that they have and I'm sure it runs more smoothly and it runs more smoothly and some recent stuff which I think is landing into the 4.8 kernel the developer of that has done some really interesting hacks around the tiling thing which adds about in certain circumstances 30% of the performance on that I'm hoping to somehow support that in Fedora 25 I'm not sure how we're going to glue the bootloader bits for that together there is Uboot upstream support for that now so we may be able if we can write that Uboot out in a particular way that is half sane it may, we may get to the point where we can just support it but it's a little bit interesting so the question is what sort of objections am I expecting it is usual community bickering from half a dozen or a handful of people that will often just object to something for the sake of objecting I mean I first floated this proposal as part of Flock last year I've done a lot of discussions and talks and Q&As and things with people that I care about their opinions and answered a lot of questions and the people that I expected originally to object to it because of things I have said have been mostly positive about it which to me was awesome there is a couple of individuals that have already started even though I haven't posted it to the list of raising objections but they tend to be just objectionable for the sake of being objectionable Some objecting people mix Oh Adam you are there I didn't see you up the back with arm 64 bit when you say about arm B8 quite fast builders etc they think, oh the slow crappy armbobs so Marston mentioned builders who have in Phoenix racked up a moonshot chassis with 30 blades in it which I am waiting on a firmware fix from the manufacturer for a bug I managed to discover in their firmware not the first one and we should have that in place shortly there's a few other bits and pieces around orchestration of that two to three weeks we will have the first batch of new arm V7 virtual builders which so on the test that I did building the kernel and I think the Java stack on a Mustang which is a similar hardware spec but with a terribly slow hard disk we sort of cut the builds time by a half at least a half and the moonshot blades have SSDs in it so it may even be a bit better than that I was actually hoping to have it in production by now I wasn't quite expecting to have to go quite so deep into kernel and firmware issues as that but in the next few weeks and certainly by the end of next end of this month that will be in production so you'll start to see that happen and so the arm V7 builders should improve an order of magnitude in terms of speed and build times so I think that will alleviate a lot of the complaints 16 gigabytes around builders 32 yeah so 8 cores 2.4 gigahertz 64 gig of RAM half terabyte SSD versus the current ones which are 1.4 gigahertz quad core 4 gig of RAM and terribly, terribly slow laptop hard disks so it will be oh and much faster network as well yes my question is if you're not an Apple you mentioned the Apple phone do you have any ah so I actually have so the question is about support for the crypto cape on the Beagle bone that will plug into the Beagle bone I believe we can mostly support it I actually have one at home I have on my laptop a list of all the bits of hardware in there I need to basically go through and work out which bits have kernel modules and whether we've enabled them or not the biggest issue with it is support for device tree overlay and so basically device tree overlay is the ability to take a base device tree such as the one to support the base Beagle bone and then layer a subset of extra device tree depending on the stuff that's plugged into it such as the crypto cape the support for that upstream is almost there and that would basically mean that you can say I have a crypto cape here and it will auto configure all the hardware and it will appear and you can just consume it without it you can still use the hardware but you've manually got to tell it that you know this crypto thing is hanging off this ITC port or this GPIO port and things like that but it would still work come and speak to me afterwards and I can it's a pretty compatible device so I've showed it to a couple of people and like a couple of people just instantly went and bought Beagle bones and crypto cape so yeah Adam did you have a question up the back no and earlier you were talking about the discussion you had about the I just wanted to mention for anybody who wanted to read your proposal a lot of the discussions about the content of the proposal I think that by 1980s there was a question in the answers about the content I just wanted to comment on that it's a very good write up and probably one of the better writes I've ever seen for that in Fesco yeah so I did a big long write up to Fesco and there was a number of queries backwards and forwards I have tried to condense the content of that and the queries that have been answered into a wiki page I'm intending on sending that link to the wiki page and the link to the Fesco proposal out to the mailing list I've been dealing with F-25 branched and various other bits and pieces over the last couple of weeks and so as much as I've wanted to get it out there there have been other things that have been screaming at me to be done and more important and I had some family stuff and some time in Australia and I occasionally do need to sleep for a few hours a day anyone got any more questions yeah has there been some testing or demos or whatever on mobile phones or other kind of so the question is has there been testing or demos on mobile phones and other sort of things no but yes so it's not a device form factor that we're actively aiming for the lead developer of the Fridrino driver which is the Qualcomm GPU that ships on like the vast majority of Android phones for example he runs Fedora on a number of mobile devices to do the development and test to that driver there's a number of really really cheap all winner based tablets that you can get that are like 30, 40 dollars and we actually nearly gave away one to every flock presenter last year but there was a whole bunch of stuff that failed in the logistics which meant we couldn't actually get them but most of those are actually unlocked and so you can literally shove a Fedora like XFC image into the micro SD slot and it will just boot but there's not a lot of graphics acceleration so and there's I think in a lot of cases issues with touch screens and a few other bits and pieces which make them a little bit interesting to use there's a lot of emphasis on getting the Nexus 7 tablet working on an upstream kernel at which point it would be relatively straightforward to use a Fedora user space with that the biggest issue is the boot loaders so A boot, H boot various other letters with boot attached to the end are a complete and utter train wreck when it comes to a generic distro because they're designed to see the 60 billion petitions that Android phones have and they will boot from a single kernel so if you screw up a kernel update you end up with a brick device where we try to have a standard Fedora experience where if the kernel doesn't boot you re-boot and you go to the previous one from a menu and so it does work I know of a number of people that have used it or use it for various different purposes with Fedora it's not a particularly pretty experience but if you, like all the user space stuff is there and work it's not a device form factor that we really have an interest or time because the amount of people working on the hardware side of things is a handful and so we're focusing on things like ESPCs and IOT devices and stuff like that now the Raspberry Pi that people are interested in that are cheap that they can and we can support easily in a very and yeah, so yes, you can run it use the space there quite happily the bootloaded kernel is a bit of a mess the Nexus 7 is probably the best supported moving 3D sorry platform but I mean that yeah, I mean the Nexus 7 is a device where there's a number of kernel groups that are concentrating on getting that support upstream so that it can be used as a completely open source Android AOSP development environment but it's focused around the Android and completely open source upstream kernel so that they can test for things like kernel regressions and stuff like that I think at the moment they need around 30 or 40 patches on top of a kernel but I think if you wanted to use Fedora on that you'd probably need very few of those patches because the vast majority of those patches are aimed at the Android side of stuff so but it's not something that we really have time to focus on because it's not really a primary user experience that we support anywhere else in the project I think we've exhausted the questions excellent