 Hi everyone, I'm Peter Robinson. I think everyone knows Tom, Spot. And thank you, Spot, for proposing this talk. Spot came to me and said, do you want to co-present with me? And I sat there for a full five minutes before I decided whether I would sit in the audience and troll him or stand up on stage and contribute. Yeah, we felt it would be for the best if, you know, because if I tried to give this talk, I do know most of what I'm talking about here, but it would be the sort of thing where Peter would be like, actually I changed that last night. And so I figured it would be better if he was up on stage to give some of the color. So I wrote some slides, I gave them to Peter, and then he sends me this email on the first night of flock and he says, I rewrote all of the slides. I hope that's okay. And we should sit down and talk about it. And then that didn't happen. And so it's going to be fun for everybody. I didn't rewrite all of them. I just added some sort of more truths. So I really don't know what I'm clicking through here. It's going to be interesting. It's going to be like PowerPoint karaoke. All right. So a little bit of history here. The first rise of rabbi comes out. It's intended to be a $35 microcomputer to try to teach children how to code. Obviously it does weigh more than that now, but that was the original purpose for the design. The pros for this thing, it is super cheap. It has standard connectors. A lot of the other microcomputers in the space have fun, you know, power or weird serial connectors that you've got to use. You end up having like a rat's nest of, do I need this with this board? And what do I need for that board? I have a collection of several microcomputers that all need nonstandard cables. Everything on the pie is relatively standard as far as connections go. And relatively cheap, most importantly. Yes, exactly. A nice thing about it was it was designed to be connected to a television. And thus the outputs are HDMI and on the original pie you had the composite video out as well. You actually still have it on this pie. It's just through the headphones connector. Yep. So it's sneaky. It's hidden there. Again, reducing cost, basically. Yes. The cons, it is an R&B 6 chip and the first pie because they had that chip lying around at Broadcom. And it wasn't really being used very broadly, but they had it and they said, hey, you can use this if you want. So interesting side point that Evan tells about the original Broadcom chip. He had visions for this chip. It's primarily used in VoIP phones and in set top boxes. And when they were tapping out the silicon, he had a one millimeter by five millimeter spare spot on the chip. And he basically begged his bosses at Broadcom, can I put an arm core in there? Because it was never going to have an arm core. So basically one of the main reasons for the V6 was that he could fit it in this spare spot on the silicon and it basically added a very small amount to the cost of producing this chip. Unfortunately, because it is an R&B 6 chip, it is slow and it was slow when it was at the time. But as Peter points out, the point was not to make it fast. It was what was available, what could fit in the space, what was cheap. And ultimately, ARM themselves had already announced anything that wasn't R&B 7 was end of life. And so we'll get to that a few slides down. Also, the only networking that was on the original board was Ethernet. There was no wireless built into it at all. And there's no real-time clock. There continues to be no real-time clock, but that's a problem you can work around. Let's do it. Don't click too many times or you'll end up five slides in. I might. Do you do the slides? Nothing. Did you put a video in there? No! I troll other people's talks, not my own. Just in case any of you who came in late aren't realizing this, we are attempting to give this talk on a Raspberry Pi running for the R28. It's working. Nobody has any plans for dinner, do they? Bad news, Libra office is not here to be. Which to the laptop at this point? We can, if you like. I think that may be a better experience for everyone involved. As much fun as watching us try and make the Pi work, we're going to swap it. I don't have that many jokes. Yeah, so Fedora18 was the last official R&B 5 release. We made a decision about six months prior to the Pi being announced that we were going to move to R&B 7 hard float because ARM themselves had said basically anything that's not R&B 7, i.e. the Cortex series of chips, was dead. And at the time we had a handful of Kirkwood boards that were R&B 5 and absolutely no V6 chips devices whatsoever. The other issue was that because the V6 had only ever been focused on phones and primarily even low-end phones, ARM themselves had never really invested in the GCC tool chain and things like that. And it basically, while it worked, it had huge amounts of problems and nobody had any interest in fixing them. And so it was primarily actually the state of things like the tool chains and stuff like that for ARM V6 and the fact that there was no chips out there, that we made the decision in Fedora that it was going to be ARM V7 hard float and who cares. Yeah, and so that was basically it. And then of course you get, you know, almost overnight, the Raspberry Pi sells out of the store immediately. Everyone is rushing to buy one of these things and the instant demand for environments for this little $35 microcomputer. So you get some students over at Seneca College in Canada who decide that they would like to run Fedora on the Raspberry Pi. So they went in and did a full rebuild of everything in Fedora. I think it was 18 or 19, one of the two. Yeah, something like that. And then they rent and rebuild it, optimized, optimized isn't the right word, but compiled so that it will actually run on the Raspberry Pi one. And then the task was completed and the students got a grade on that and then they went off into different things and it sat there and bit-rodded. Yeah, so they actually did a couple of different Fedora releases. I think the last was about 22, but obviously like each year a new influx of students would come in, they would be given certain bits of it as a project and the teachers would sort of vaguely keep stuff glued together in between classes and stuff like that. But you know, it was primarily students. We did get a few very good Fedora contributors from them. There was some controversies around the PiDora name and generally it was an interesting time. So if you go on the Raspberry Pi Foundation's website you will see them recommending Raspbian as the default operating system. They delivered this through a number of channels. And basically Raspbian is a fork of Debian straight up that the Raspberry Pi Foundation put together. And it started as Debian at one point, but it really doesn't resemble Debian in a lot of ways. One of the big changes is that the Raspberry Pi Foundation has their own fork of the Linux kernel and they use that fork for driving Raspbian entirely. They also have a whole bunch of firmware inside of it to not only boot the device itself but also to power up most of the peripherals on the board. Yeah, so they have a number of utilities for controlling the video chip. So the Raspberry or the Broadcom chip is quite different than most chips in that you have firmware that's on a fat partition and it's actually the GPU that boots the machine and then the GPU starts up the ARM core and hands over control. But the GPU still keeps control of a lot of the hardware peripherals. And so you have this mailbox interface where the Linux kernel actually speaks to the mailbox and says, hey, I need to scale up or down the speed of the CPU. Can you adjust these voltages in the GPU so that I can do that? So if it's running, like the Pi 3 is running at 600 megahertz and it needs to go to 1.2 because there's a bunch of stuff happening. It actually has to go and ask the GPU for permission for that. And in fact, one of the early libraries that they did was a library in user space that allowed for programs to bypass the operating system almost entirely and talk directly to the GPU to do output. And so a lot of the early Pi applications were using that library to get high performance 1080p video out of the Raspberry Pi because it just sort of said to the operating system, you don't need to know anything about what I'm pushing through over there. Just all this is going over there to get handled. Yeah, I mean, and ultimately that comes back to its history as a chip designed primarily for set-top boxes. So Raspberry Pi 2 comes out. The big change here is that we get a V7 CPU in the mix. Unfortunately, because of the unique design of the architecture and the hardware, you still can't use the upstream kernel for this just because it got a newer CPU. And by this time, like there were people, you know, sending support for the Raspberry Pi chip upstream. You could actually boot Fedora on it if you wanted serial console and no USB. So it was certainly getting there. There was work being done to get things upstream. The Pi Foundation had done their first major sort of kernel rebase because they needed a newer kernel for the newer hardware and realized that, oh my God, all the work that we have to do because none of it's upstream. Funny that. Yeah, one of the big choices that they made was to do things in a, what they thought was a reasonable, sensible way, but was completely different from how everyone else was doing ARM embedded work in the Linux kernel upstream. And so every time they rebased for a while, they had different models for things. Yeah, and I mean, to be honest, it's still the model for a lot of the ARM SAC vendors that they don't do a lot of stuff upstream. And if you look at most of the ARM vendor kernels, you'll see that they very, very closely align with the versions needed by Android. And Google, with its push to get newer kernels and newer functionality and say, like if you're running Android P, it has to have at least this kernel is helping drive the vendors towards the upstream so that they don't have to do all this extra work for newer versions of Android. So I have a question. So Fedora as a policy has a one kernel policy because we have like three kernel developers and basically to support hundreds of thousands of hardware combinations. I do most of the ARM kernel stuff and it's done in the evenings and it's like not my day job basically. And when I first started doing ARM eight years ago, I started poking at kernels because we had problems. And now I know more about kernel code than I ever thought I wanted to know. But it's still a hackathon in the evening and on the weekends sort of thing. There are people that have put the Raspberry Pi kernel source into an RPM and used it in Fedora remixes and the current Raspberry kernel is either 4.9 or 14. And so it does work fine. But basically Fedora as a project doesn't have enough kernel developers to be able to deal with a thousand different forks of the kernel. So and also by Fedora doing that it's amazing how many of the board vendors and that have actually switched to upstream so that they will get Fedora support. And so it's taken us a while but that sort of we're not going to have display on the Raspberry Pi. And it was funny because like I first tried to get so on the non free binary bits the things that we needed in Fedora was the boot firmware. And at the time like when it took us about five years and spot is notionally like the Fedora legal person. He cat her to anything to do with legal through the lawyers for the project so that tech nerds like me don't need to deal with legal nerds. And it took us about five years to work with the Pi Foundation and Broadcom to get them to change the wording because there were distros that were shipping the Pi firmware. But the license explicitly said it couldn't be redistributed. And so it took us about five years to get just be able to redistribute the firmware to boot the device. And then around that time we got like the Pi 2 came out which was on V7 and I packaged up the firmware and it was actually in Fedora for probably two years before we could actually support it. And like people that looked at the kernel stuff would see commit messages from me that were like at support for Broadcom device. And like I wasn't screaming it but we officially started to support it in Fedora 25. You could run it in Fedora from about Fedora 22 but it wasn't what I considered minimum viable product and we'll get to that shortly. So the Raspberry Pi has like four major binary blobs. One was the boot firmware. We eventually got that fixed by probably a few lines of change in a text file. The video core of VC4 as it's generally known has a binary driver and a binary user space that's not MESA and it's all very proprietary and we could never use that either. And the fact of the matter is most Raspberry Pi users want the graphical interface. Broadcom via the Raspberry Pi Foundation employed a developer to actually develop an open source driver and that's all upstream in MESA and in the kernel and things like that. And there was a bunch of other things. There's a bunch of user space libraries for the driver as Spot mentioned. There's a bunch of user space libraries to be able to speak directly to the video core to do things like 1080p video rendering to speak to the ISP which is the camera sensor processor to get stuff from the camera. You can actually speak to the camera sensor and tell it to take this stream of videos and write it straight to the HDMI output and so it doesn't even hit the Linux kernel and things like that. And so that's why you get a very CPU low powered crappy SOC that can actually do like full 1080 video streaming and things like that. And so there was a bunch of that stuff that was binary only and then a bunch of random utilities to poke at various registers on it to adjust things like memory clock speeds and various other bits and pieces like that. Speaking of someone that worked with them for a number of years to try to get a lot of this opened up, initially there was this intent that they didn't think that anyone was really going to care about any of this code and that it would be sufficient for them to just be the people who are the experts in that space. But when it was obvious after a couple of years that there were a lot of people that wanted to help improve the situation of the environment and make it maintainable, then they started to open up as much as they possibly could in that space. I'm really eager to figure out how they could get more eyes into the mix. I think they really didn't necessarily understood the box they had opened when they started working in this technology area. So pretty much everything except for some of the boot firmware has been put under an open source license and the boot firmware has been re-licensed like Peter said to allow people to distribute it. Ultimately if you look at what the Raspberry Pi Foundation wanted to achieve, they're primarily focused on education and so they wanted to basically make it easier for kids. The fact that they produced this device that was $35 and absolutely changed the single board computer industry with a single announcement was extraordinary. But if you speak to most of the people at the organization it was about kids getting hold of devices to code and you look at kids faces when they first get to light up an LED and it's like you can fully understand why. But as far as they were concerned the actual device itself was more a tool or an enabling tool to enable that. So they're not so much focused on things like open source. If they have a kernel that works and that they can support and like one of the biggest problems we have with the open VC4 driver versus the closed driver is that the amount of monitors that it just goes black on because it can't negotiate the EDID and work out what's there has caused and it's improved a huge amount. I'll get to that shortly. But yeah, it's an interesting problem. So they continue to iterate the hardware. We have the Raspberry Pi 3 and the 3 Plus. The hardware now has an AR64 core inside of it. It's ARM V7 compatible. We do have AR64 builds for the Pi. But what? Should people use this? Well, I mean, ARM64 on the Raspberry Pi is interesting. The image that I promote primarily is the ARM V7 one because you can have one SD card and you can put it in the two, the three and the three plus and it will automatically detect everything and just boot. It's only got a gig of memory and the problem with the 64-bit address space is you actually end up on low resource machines, end up wasting a lot of that memory. So depending on what you want to do with it, you're much better with the 32-bit version. A lot of people on the Pi want to run things like Kubernetes and other such things and basically stuff that really needs 64-bit. So a lot of people use it in the service space with 64-bit. But if people just want a desktop to browse around and blink leads and things like that, I always recommend the 32-bit one because it's just a little bit faster. Yeah, exactly. The performance difference is noticeable between the AR64 build and the ARM V7 build. And when Spot did the slides, he said more USB, also more reliable USB. All lies. I may have put that in there to troll Peter, I don't know. Yeah, I mean like when they went from the V6 to the quad-core V7, from the quad-core V7 to the quad-core V8, all the peripherals and everything changed. It stayed the same and they were just swapping out bits of silicon. And the USB control is terrible. And if you look at most ARM SoCs, you'll have five or six MMC, half a dozen serial, a couple of CAN, a real-time clock or two, half a dozen USB root ports and various other bits and pieces. The Raspberry Pi has one buggy USB bit of silicon, which basically with the upstream kernel runs half-duplex. And so your 480 megabits is much closer to 240 megabits. In reality, I think we can push a little bit more than that. The way the Pi Foundation deals with that, and basically it doesn't have a bit of hardware silicon in there, so it has to be that bit of the USB stack, which is normally done in hardware, is emulated in software in the kernel. The way the Pi Foundation works around that to get a bit more speed is they use the FIQ. When that patch was proposed upstream, basically the USB stack maintainer and a whole bunch of other people fell off their chairs laughing and went, no, not over my dead body. So the Pi Foundation's kernel has this terrible hack to make it go faster because, you know, cheap. And it doesn't have a lot of other things that a standard ARM board has. It doesn't have a JIC, which is a global interrupt controller, and various other bits and pieces, which makes the performance compared to, you know, a similar ARM SoC substandard. Yeah. There's an entire chapter in my O'Reilly Raspberry Pi book on why your USB devices aren't working properly in the Raspberry Pi. It has gotten slightly better in that regard, because we've been able to do some clever workarounds to see devices when they're plugged in, as before where you would have a device plugged in and the Pi might just turn off or refuse to see the device at all. But it's still unreliable, and so one of the things I tell people when they're trying to build KIT based on their Raspberry Pi is that if they can figure out some way to wire it directly into the GPIO, they're going to have a much better experience than if they try to go through the USB port. Well, and USB devices draw power, and probably the number one biggest support problem we see in Fedora is that the power supply is not rated enough. Which was actually the problem we were having from 8.15 to 8.45 this morning. Yeah, so I mentioned, we've mentioned the boot firmware. There's the open source VC4 driver, so we have fully accelerated MESA, so GNOME will run with Waylon fully accelerated. It's still not exactly fast. There's still a whole bunch of kernel changes not upstream. There's no camera driver, there's no video acceleration offload drivers, and there's no support for the official Raspberry Pi touchscreen yet. And like, there's a bunch of that stuff in motion, but we're not quite there yet. The Pi Foundation is now actually looking, and it's at moving from their close source driver to the upstream open driver in Raspbian as well. I believe the latest release that just happened to support the 3 Plus actually has it there and people can enable it, but it's not there by default. Amusingly, so about a year ago, like, the Fedora project hadn't had a lot of communication with the Pi Foundation because of something happened in the early years of the existence of that device. And finally, last year, I got connected with Eben and I went up there and we actually had a very fun face-to-face meeting. I demoed, I think it was Fedora26 running ARM64 GNOME shell fully open stack on the Raspberry Pi. That was with grub menus and everything, and Eben was blown away. It was the first time he'd ever seen his device running a fully open source stack. And that was kind of cool. And so, we've had, you know, and the Pi Foundation hosted back in May a GNOME performance hackfest, so in Fedora29 there's going to be a bunch of changes that are landing into GNOME which should help speed things up and give more memory and make it generally better for Raspberry Pi and other low resource devices. So, that's kind of cool. And so, there's still some close binaries and bits and pieces, but we're certainly getting there. Most notably, we can't ship the firmware for the wireless or the Bluetooth controllers because of Broadcom. Yeah, so the wireless is interesting. We actually do ship the firmware. We ship some of it. What we don't ship is the NVRAM. And the interesting, well, most wireless firmware has some form of NVRAM. In something like a $2,000 laptop, you know, they have an onboard chip that has that. And in a lot of the Broadcom cases, the firmware configuration file, the NVRAM, is actually shoved into vendor-specific ACPI tables. And so, the Broadcom driver will try to find and read from an E-prom and then it will fall back to BIOS tables. And finally, it looks for a text file that it's expecting to be there. Broadcom, a couple of years ago, sold all their wireless stack off to Cyprus. I spoke to the Pi Foundation a year ago and we've had follow-up emails about it. And it's not just the Pi that has an issue here, but a number of low-end tablets and a huge amount of other ARM SBCs. And so, the Pi Foundation escalated it to Broadcom Cyprus. And there is now a conversation happening between the Pi Foundation's lawyers, Cyprus lawyers and Broadcom lawyers. So, I don't ever expect to get that resolved. And we're looking like the Pi Foundation is actively working to look at whether they can just license it themselves because it is Pi-specific. And, you know, there is other discussions going on upstream for other devices as well. We're hoping we'll get there. The Bluetooth firmware, the firmware will actually run with the built-in default firmware. It has issues and it's not particularly stable. I'm hoping we should be able to get the firmware for that into the upstream kernel real soon now. So, again, there's stuff like that that's happening and there's a bunch of other drivers and improvements that are happening. Of course, it just never happens as fast as everyone would like. Yeah, so the only thing I wanted to mention for here, I mentioned minimum viable product. A lot of the ARM devices we support, you know, a USB SD card slot, maybe SATA and serial console. In a lot of cases, the display doesn't work and things like that. And we enable them because, you know, the people that tend to use those devices are a lot more computer savvy and know what a USB TTL is and things like that. When it came to the Raspberry Pi, there was no way in hell I was going to announce that we had Raspberry Pi support if they couldn't plug in a keyboard and a mouse and basically get output like this. And a graphical display in various other bits and pieces. So just, and like the Fedora ARM team is relatively small. And so I didn't want to get into a situation where we had a million users download and try and use their Raspberry Pi. And I said, oh, you have to go and spend another three, four bucks and get a USB TTL. So when I added Pi support, I had, we have to have a minimum viable product. We have to have HDMI output. We have to have USB keyboard mouse. We have to be able to get XFCE or something like that up and running so that, you know, the user will plug in what they expect. And so Fedora 25 was when we landed what I considered the minimum viable product. One SD card, you could plug it into a Pi 2 or a Pi 3. It would automatically boot up and like literally you can just DD it out on Windows. It is the one device that the SD card for all our images is set up to boot on. And so you can literally DD it out, plug it in, boot it up and you'll get to a login prompt. So that's why, like, as I mentioned, we had support there since about Fedora 22. But it wasn't what I considered a minimal viable product for the average Raspberry Pi user. I don't think we need to go into huge amounts of detail. We use U-boot on top of the Pi firmware. Raspberry and boots the kernel directly. But U-boot gives us the ability to boot multiple kernels. In the ART64, you actually get, it boots then grub via UEFI and you get the standard grub menu. In Fedora 25, too many numbers, in Fedora 29, V7 will also have grub. So in both 32 and 64 bit, you'll get exactly the same experience as you get on your Intel. So, like, and obviously once we had a minimum viable product, we sort of extended stuff. You know, Wi-Fi support, HDMI, audio, video that was more stable. The problems we had with black screens from 25 to 26 was an order of magnitude lower. We've gone into why ARM64 support. The Raspberry Pi 3 Plus was an interesting turning point. I mentioned last year I had a meeting with Eben and the Pi Foundation. We had support in Fedora 28GA for the Raspberry Pi 3 Plus. I actually got an email from Eben about a month before the 3 Plus was announced and basically just saying, what's your shipping address? And a 3 Plus arrived in the mail about a month before. And so we had an opportunity to bootstrap this and get firmwares and various other bits and pieces into place so that when it was announced and we missed getting, I think, one bit needed into, like, the beta freeze by, I think, about 12 hours. So beta didn't actually have support for the 3 Plus, but the nightly soon after, right after when we unfroze did. And so that was incredible and actually quite cool because, like, when F28 came out, there was people going, when's the 3 Plus going to be supported? It already is. So that was fantastic. So the future. One of the biggest problems and one of the biggest support issues we have, and it's not just Fedora, but it is distributions that aren't Raspbian have, is that people go, oh, I've enabled this hat in the config.txt file and it doesn't work. We don't support it like that. Well, in Fedora 29, and I need to do some testing and we've got a few bits to sort out, but in Fedora 29, the vast majority of config.txt should work just like it does in Raspbian. Some of it does now. It won't all, like, so any of the stuff that's around the VC4 driver versus the closed driver won't work, but for the vast majority of cases, the config.txt files, and that's fantastic because the Pi Foundation and the entire ecosystem around the Raspberry Pi has all this documentation where they say enable this in the config.txt. So Fedora, Suzy, all the other distros have been working with the Pi Foundation to ensure that this happens. So it enables us to much more easily support hats and cameras and various things, but it brings us much closer to Raspbian and so we don't have to go where the exception, not the norm. And so we've got some stuff I need to deal with around Uboot and I'm hoping that that should be in place in time for Beta. Camera, there is a whole bunch of patch series that have been posted that should make the camera work. A part of that is waiting for me to test and part of that is getting the config.txt stuff working. But things around the camera have had to, like the person getting this driver upstream has had to have changes in the core V for Linux stack. And people have said, oh, why don't you just write the support? Well, one, I'm not a kernel developer of that note. Two, you need Broadcom NDAs for hardware docs and various other bits and pieces. The camera support in particular probably won't land in F29GA, but I'm hoping and will probably have a 419 kernel as a zero-day update or fairly close. And I'm looking at Laura and she's not screaming at me, so I'm pretty sure I'm not completely wrong there. And so I'm hoping very early in the F29 cycle we should be able to use the camera. My brothers will be very happy because they want to use it on the farm. Official touchscreen. It needs two bits. It has the actual touch interface. The driver for that's not quite upstream yet. It needs support for the DSI interface. Most of that is upstream but it needs a bunch of glue. Also probably needs config.txt support. But again, hopefully getting that in F29. I know Bex and the ambassadors with the Federator will be very happy about that. I have one of those sitting on my desk at home. And then beyond. Once we get the config.txt support, it enables us to enable hats. And there's a vast array of hats in the ecosystem. There's lots of people that want to use Fedora on their Raspberry Pi's for various audio stuff with the various audio file hats that you can put on top. I'm sure you could probably get a $100 audio file Ethernet cable for your Raspberry Pi. And so that's official touchscreen and official camera support is something that I get almost more queries about than I did. So in the five years it took us to support the Raspberry Pi, I think I averaged probably one query a day as to when it was going to be supported. When the Pi 2 came out that went up to probably ten queries an hour level. The touchscreen and the camera support aren't quite that. Obviously hat support. A lot of people want support for things like the sense hat. And then basically just stuff in the ecosystem. Part of the issue there is that a lot of the hat support has involved device tree overlays. And the device tree overlays that exist for the upstream Raspberry Pi Foundation kernel are good reference points for porting to the upstream model. But they can't be dropped in place. They're not going to be able to work as is. Some of them just will never be portable but most of them can be fixed and applied. I've done some basic work in theory to get this ready to go when the config file pass-through actually works. So it's totally doable and I imagine that a lot of the hats will very quickly get proper upstream support. Some of the hats will be the same and just work and again we've done a bunch of work with the Pi Foundation as to how that works. And I mean ultimately now they're more focused on getting things like the open VC4 driver working and things like that. All the work that we've done in Fedora actually ends up being a really good enabler for them. And like we've belted out a huge amounts of issues with the VC4 and the EDID detection stuff that they now don't need to do. So they're not starting from the ground zero like we were. It was very amusing to me when Fedora 25 landed where we have a five year old device and Fedora has a very old distribution. And I was having sort of early adopter problems with such a device that was five years old. I get all the luck. And you know some of the other stuff is like the GPIO stuff that the Pi Foundation uses is an end of life deprecated interface in the kernel that basically is extremely insecure. So funnily enough we've turned that off in the Fedora kernel because it's deprecated and I never want to support it. But the Python GPIO bindings if you run them on Fedora it says this is not a Raspberry Pi go away. And so there's a new library Lib GPIO and what I am and it's almost stable and well it's relatively stable. We now have enough 29 Python bindings. And what I'm hoping is that the RPI dot GPIO Python bindings we can get upstream to accept some patches where it will use that interface. And if that interface is not available it will fall back to the old interface. And people are like I've spoken to people about it and they're like oh it'll be just easier to copy it and have a clone of it. And it's like no that's not going to work because there is literally 10,000 how-to guides out there that say pip install this library. And a bit similar to the config dot text stuff or the documentation says it. So it's like if you're Raspberry and do this, if you're Fedora do that. For a 10 year old working off a worksheet in a lab that's not going to work. So we need to get support into the upstream. And like I explained that to people and they're like oh yeah that makes sense. Because ultimately we're not going to be able to go out to every like how-to guide and get them to add if raspberry and do this else do this. And so stuff like that that people don't necessarily understand oh it's just going to be quicker to recreate it. No it's not. And so it's things like that that I'm hoping we'll get support for. So even though it is Fedora all the documentation out there all the classroom guides and things like that will work just as if it was Raspberry. We're not there yet but this is what I'm looking to do. And if anyone's any good at writing path and bindings or things like that and want to get involved in that sort of stuff come and let me know. Do we have any questions? So the question is when is camera support going to be available? I believe he's trolling us. Fedora 25 go back in time. Or watch the video afterwards. You can see where I talked about it about 10 minutes ago. No one. So the question is even though the talks about Raspberry Pi what is the best ARM device to use on Fedora? My answer to that is always another question. The question is what are you going to do with it? And we support without exaggeration 200 plus devices on Fedora ARM, 32 and 64 bit. What you want to do is like turn LEDs on and off and maybe run some Bluetooth house stuff. The Pi is a perfectly fine device. If you want to do AI and machine learning the Pi is not the device for that. It just doesn't have the power. If you want to run a NAS box the Pi is not that even though hundreds and hundreds and hundreds of people do. The Orange Pi series have four USB controllers on board that run at 480 megabits a second and have proper SoC attached gigabit ethernet as opposed to US like 100 meg or in the case of the 3 plus gigabit ethernet that runs at around 300 meg. So it basically depends on what you want. The Jetson devices run known desktop fully accelerated and really, really well. The IMX6 stuff along with the Pi and the Dragon board 410 will all run fully accelerated Wayland. So they're all not bad but a gig of RAM is just really not enough. There was a patch that landed in GDN the other day which will literally reduce the shell utilization by 230 meg which when it comes to the Pi is 23% of your memory utilization. There's some patches going into package kit various other bits and pieces and again that will improve utilization across all the ARM ecosystem. And even the cheap low end X86 stuff as well. So yeah it very much depends on what you want to do. And how much you want to spend. Exactly. Till. The question is, is there anything better on Fedora than Raspberry Pi? Everything. Even the things that don't work aren't better because it's Fedora. The feedback I generally get from the people that use Fedora on the Raspberry Pi is that we have things like Kubernetes and Docker and a lot of the server ecosystem where it just works. People like the fact that because we're using the upstream Fedora kernel they can boot up their Raspberry Pi and it's running SE Linux just like their X86 desktop. You know they can use the server edition and cockpit is there and they can you know and so I think probably like if you want to use hats and blink you know leads and that sort of stuff. Razbian at the moment is by far the improved thing because they're focusing on that device. Like if you try and use Razbian on other devices it'll work but you'll have a substandard experience. So any of the server-side stuff, cloud stuff, Docker, various other bits and pieces works. Because it's Fedora we have the latest and greatest you know libraries and optimizations and measures and stuff like that. Two years ago I went to my first Lunaro Connect. Lunaro for those that don't know is a bit like OpenStack Foundation for the ARM ecosystem. It's an organization where ARM SAC manufacturers and all of that congregate to talk about the ARM ecosystem or open source in the ARM ecosystem. And I ran into a vast array of kernel developers that had been, I'd been dealing with upstream for support for everything from the Pi through to whatever. And they were like holy god you're actually here because like the ARM SAC kernel maintainer had been asking me for five years when I was coming. And they just said the thing we love about Fedora, all the latest packages, stable, it works exactly the way Fedora does on my laptop. And they were all like even the kernel where we boot our device and we have serial console, USB, Ethernet and storage is great. And they leave the default Fedora kernel in place even though they're kernel developers so that when they blow up their kernel and the device ceases to boot, they can just select Fedora from the menu, or the Fedora kernel from the menu and they can get back up and running again. I've spent vast array of time looking through all the libraries to make sure we have all the ARM optimizations in place and everything else. And so the feedback I get from a lot of people is that SE Linux just works. And you know, there are some patches that I work with upstream form where we literally tripled the throughput I think between Fedora 25 and 26 of the MMC stuff. So like we're not even the fastest but we're certainly getting there. But the general thing is that it's like Fedora and they can, like I see blog posts, oh how to build Kubernetes on your Raspberry Pi. And I reply to them or you can just run Fedora and go DNF install Kubernetes. And a lot of people do. Yeah, I think the consistency is a big part of that. I think, you know, being able to get this exact same Fedora experience no matter what hardware you're running on is a really valuable selling point for why you would consider doing that. And like I hope that we can get to a place where, you know, Fedora is a reasonable choice to use it in the classroom for all these things. You know, I do support for the Raspberry Pi primarily in my own time. Like I have a love hate relationship with the Raspberry Pi. I generally actually refer to it as the Raspberry fucking Pi because it's caused me so much stress. But at the same time, it's huge ecosystem. And I want to get all the stuff working, camera display and everything. So it is all just plug and play like Raspbian. If you could clone me three or four times or someone would like to come along and help out with the Python bindings for GPIO and things like that. You know, it's just a small project that people can work on an hour here and hour there. It will greatly help the Fedora ecosystem. And in fact, the Linux ecosystem in general on the Raspberry Pi and help expand it. And you know, I have semi-regular conversations. Sometimes we spot, sometimes we think how we can get Fedora better supported so that it can be used in classrooms on the Raspberry Pi. There is a lot of interest in doing so. And we are well over our time slot. So thank you all for coming out. Thank you.