 Welcome everyone from the break. We have Neil and Davide here to speak about a Sahira mix floor is yours Cool. Hello everyone. My name is Davide and my name is Neil We'll be talking about the Fedora's I remixed bringing Fedora to Apple Silicon Macs I'm Davide. This is Neil. We do a lot of things. I do a lot of things in Fedora and incentives I do a lot of things in Fedora sentos and other places I work in Meta, Neil works at Red Hat. These are nothing to do with either of our jobs, mostly. Yeah All right, I think you're doing this one. Yeah, so So we'll start a little bit with the Fedora Sahi over you. Can I get over? So the Fedora Sahi SIG is our special interest group that packages and maintains the software used to support Apple Silicon platforms We design we try to bring in all the software that's needed for being able to boot and run and configure and be able to lifecycle Apple Silicon hardware that includes things that are used for like say Doing rescue environments with iMobile device and or Libi and mobile device or whatever It's called as well as like being able to have M1N1 for the stage one bootloaders and having custom things managing the patch sets and changes upstream and sidestream for for being able to handle the Apple Silicon platform next and As our primary deliverable for that is the Fedora Sahi remix. So this remix is a special deliberative Derivative of the Fedora of Fedora Linux that's optimized for Apple Silicon platforms As I mentioned earlier, we maintain a bunch of software for the a Sahi Linux platform for the Apple Silicon platforms And on top of that for this remix We have a few core changes that allow it to successfully run in In in a proper fashion. So that's a custom kernel that is derived from the Fedora kernels with the a Sahi Linux patch set As well as custom Mesa that contains the latest stuff to enable hardware acceleration Custom U-boot to be able to boot the system and the Fedora Sahi remix branding the remix comes with KD Plasma and GNOME As well as server and minimal flavors These Mac mini is currently running the presentation is running the GNOME flavor that Mac book down there Is which is a Mac book for all I believe is running the KD flavor and just running GLS gears in a loop to distract you So the flagship experience for Fedora Sahi remix is Fedora Sahi KD And this variant is essentially the Fedora KD spin just tweaked and enabled to ship on Apple Silicon hardware and So the bullet points on screen basically kind of describe what we do with the collection of KD software being able to provide You know our branding and usability tweaks at Firefox SE Linux for security firewall D the whole works next and as Davida mentioned the Mac mini that we're presenting from actually runs Fedora a Sahi GNOME Which is a derivative of Fedora workstation for Apple Silicon platforms It's essentially the same experience like I said with the KD one It's it's just minor tweaks to be able to enable it to run on the platform next and For the final bits the server and minimal ones these are intended for Headless operation of Apple Silicon hardware in particular like the Mac mini that we're running GNOME on we could also use it as a server and be able to have the ability to do arm 64 based Workloads in a server configuration with this using Linux rather than trying to do weird things with Mac OS the minimal version is Essentially the same kind of environment But it's actually more stripped down because the intent is to be able to prototype for being able to eventually do something like putting it into Mac minis that are running in clouds like if you've you've seen those various providers that'll like rack up Mac minis for you and then give you Them and just in time the idea would be hopefully to be able to provide Fedora a Sahi and in a cloud environment At least enable those people to be able to provide it if they ever figure out how to you know Deal with bootstrapping it. Yeah, if you use the Fedora everything ISO that would be the closest, right? All right now, let's talk about how we got here Back in 2020 and even before one of the things that Neil and I worked on was getting butterfs in Fedora And ultimately this led to a change for Fedora 33 to make butterfs the default file system And one of the things we had to do as part of this was of course making sure the butterfs would behave well on all architectural supported by Fedora and and One of the architectures body by Fedora is AR64 and one of the things we quickly discovered back then was that there aren't that many Platforms available in the market that are widely available are reasonably cheap and can be around 24 7 Without self-destroying after a little bit So we were running things like file system stress test And if you're on file system stress test on Raspberry Pi is your Raspberry Pi will probably die After a month issue or so of operation they're just not designed for this kind of workload and While you can definitely get server-grade arm64 hardware, you probably don't want server-grade arm64 hardware in your apartment Both for power usage and for the noise and even more so during the pandemic. That was definitely not something you wanted to deal with So we were keenly interested in having a usable arm64 platform that would be Well-supporting for this kind of work at the same time Hector Martin started the SI Project to port Linux to Apple Silicon Apple had released these Apple Silicon machines a while ago. There were these new arm64 based systems They were as opposed to most arm64 systems that you can get on the market. We tend to be developer boards These are like finished products. You can buy in a big box store. They are reasonably reliable They are reasonably priced to some extent that you can buy a Mac mini and run it and it's not horrific And you can keep these in your bedroom and not hate your life And SSDs don't burn themselves out after a stress test. Yes that too and the NDSI project was a big success He was he started with a pantheon and he was funded very very quickly they always met overnight and the developers were able to work on these and Announce the project by Christmas 2020 and over the next year this matured extremely quickly to the point that he got I think he was in like six months. It got from yeah, basically nothing to a working to a working usable system So these are something that we were definitely interested and we started we started taking a look at it and seeing if we could If we could leverage this work and get involved And in early 2012 was when as I linux released their first alpha 2022, sorry We didn't go back in time. Yeah, we did not go back in time was when as I linux released their first their first alpha release which was based on arts linux Arch Linux alarm the arm port for arts linux So in about 2021 we started looking at what how we could integrate this work in Fedora We started the Fedora as I project as we called it there We did not really start doing much work on the porting itself until 2022 because we act to actually get access to the hardware and stuff By the end of 2022 mid to end of 2022 we had images that were usable I would say for the regular not regular users, but like for interested users They would not require active development work to be used We chose to use Kiwi to build these images because Kiwi allowed us to iterate very quickly on it And we were able to spin up the environment easily for this At that time we were still we were using placeholder branding for this because this wasn't an official project in any way We work with Fedora council to get trademark approval So we could actually brand this as Fedora because we wanted this to be a Fedora thing This was granted. So we were able to brand this as a Fedora as I remix We call this a remix and not a spin Because of complicated reasons, but the effectively the idea here is that this is Fedora We just the added bits necessary to provide a good experience on Apple Silicon What we work on getting this bits integrated into Fedora proper So ultimately we would like this to not exist and you to just be able to run Fedora or station or server or whatever But it's gonna take a while to get there and there's some non trivial deviations needed right now from stock Fedora And not only the user of 16k page kernel as Neil mentioned The use of a patch do boot and a few other components But ultimately we want to get to the point where you can just run or station or server or whatever and that will work And I suspect we'll get there in a couple of Fedora releases All right, right. So so where we are today We're what I would like to call slightly well oiled at this point We have a process for being able to manage the derivative components that we need to build the remix With our custom kernel downstream from Fedora's kernels As well as a Mesa build that comes from the ASAHI project upstream Davida's got patches the backports the patches on top of the Fedora you boot and The and he manages the infrastructure for the website and we now as of like a couple weeks ago now have auto build infrastructure So we were actually doing nightlies or or dailies. I don't know what we're actually running We're doing dailies and to be clear The reason he says it's only infrastructure and he's not using the Fedora one is because we can't build Kiwi images on the Fedora Right now. Yeah, so that's the main reason what we're using dedicated in front There's like some Rube Goldberg gas thing that I spun up on AWS right now for this But like long-term ideally we can get all of these in Fedora in front not not have to maintain dedicated stuff for it Right, but now that we're at a place and this is where the slightly well oiled part comes from We now are in a place where we are able to with minimal human intervention Make sure we can make releases and pump them out as we make changes to the underlying components that make up the remix So the big part of why Fedora saw he is kind of so special is that we did a lot of work actually upstream in Fedora too so As I mentioned earlier that the Fedora KDE variant is the is the flagship and that meant that a lot of work Needed to be done there to specifically make a side of Fedora saw he on with KDE to be Absolutely perfect or not perfect, but absolutely like perfect for the hardware and that meant like changing to Plasma Wayland by default Getting all the stack to move to using Wayland natively as much as possible Getting the login manager into that which was actually really really hard Get me outside sometime in the hallway And I'll tell you like the the whole horror story that led to how long it took to get that But also having pipe wire and these other innovations that came from other people within the project and their and their willingness to help Us be able to support our hardware platform Also allowed us to have a very streamlined experience for people that just wasn't possible on other platforms So and on top of all that we did a lot of work for desktop related Arm software because nobody else was really running them in any serious capacity And we were seeing lots of little quirks and bugs and stuff like that And we were capable of drilling into them and fixing them and polishing up the experience to the point where It's actually quite possible to use it in a semi sort of kind of daily driver sort of scenario If you're just doing simple stuff like a browser and some documents and whatever right like most of the things are actually in a Good working state. I use it as a daily driver for my work desktop at home, and it's fine Yeah, it's worth noting that all these changes don't just benefit the specific port But they're useful for all the other systems that run on AR64 So if you have like a pine book pro or Raspberry Pi used as a desktop all of these end up helping out there as well There were a surprising number of weird hangs that were in like browsers of all things that that caused That us narrowing down and getting those fixes and actually improved the quality of the experience for all ARM desktop platforms if any others exist like the pine book pro and some others next slide and Today, I want to you know the state of the hardware support with the fedora sci-fi remix We have good coverage on all M1 machines and M1 M2 laptops We have firmware support going up to the latest firmware at this time, which is v13.5 And as of a couple weeks ago, we switched on hardware accelerated graphics by default However, we do have a few things that are not supported yet The speaker the camera and on laptops the HDMI or deep display port connectivity is not working There's much more details about the the matrix of features and functions that are in development working Not working partial working whatever On the ASAHI Linux wiki. All right now. Let's talk about what's coming up next We are happy to announce that the fedora sci-fi remix is going to be the next flagship distribution That will be used by the ASAHI Linux project that's been developed together by as I links and the Fedora project The tentative release for this is the end of this month There are you oh, yeah, so the tentative official release is the end of this month, you know, we're working through everything It's not it's not quite done yet. We're working through it We're hoping that we'll have it done by the end of this month There are development builds available that you can play with that will have a link later Although as Neil mentioned, this isn't released yet officially. So if the developer builds eat your machine Maybe think about whether you're okay with that The at the same time the various patches on the on the various components of these are getting integrated in their respective upstream projects Because the general philosophy around as I Linux has always been to try and get as much as possible integrated upstream Because that's no point ultimately maintaining downstream force of things So slowly and methodically we're getting support for all the components in the kernel in you boot in Mesa In like X keyboard config further. We are my keyboard in all of all of these all these projects So that eventually we will be able to just run a stock kernel a stock you boot and so on and have a good working system Yeah, and a lot of this effort, you know has been driven by Hector Martin Mark Katernas Alyssa Rosentwig and a Sahelina. Sorry, Alyssa. If I didn't say your name, right? Yeah, but Russ for Linux No, like from my personal experience as as the maintainer for the Fedora Sahi kernel Russ for links tends to be like one of the the stragglers here It's it's starting to get better at keeping up with the new rust compiler versions It uses a lot of rust unstable features Which means that every time the rust compiler gets upgraded my kernel builds get broken And so that's that's actually a big part of where this is Less where I like it to be in terms of the the stability level because the guarantee of me being able to pump out new kernel builds is Not where I think it would be but it's getting there. It's workable We we saw that by effectively having a straight down rust compiler in a copper that we can use in an emergency. Yeah, yes So if we bring in a 16k variant to the Fedora kernel, which is it's definitely on the table of something to do I know that there are patches the rust stuff right now is not ready There's no way we're turning that on I get that I'm not gonna bring that in the rest of the stuff though My understanding is is is fairly well. It's being staged and merged upstream fairly quickly. Yep so if I if I were to turn on a 16k variant and Release that as a standard Fedora kernel. Is it possible to do a the Graphics driver as an out-of-tream module No, because Russ doesn't know how to build modules So like the problem so trust me this burned my soul so hard because I wanted to do it that way too Oh, so I don't think we actually said that so it's probably worse for people that aren't following this closely The reason we are talking about rust is because the GPU drivers for this platform are written in rust. It's actually the first major Kernel component that isn't a toy driver that's being developed using rust from the start right and because it's the first one It ends up leveraging a lot of Fringe unstable rust features that are in the process of being developed So it's taking a while to get it integrated Eric. You were trying to say something You might want a microphone first One thing I would like to say FDM if if rust support isn't ready for the Fedora kernel yet like For months we were running on like software rendered graphics and they're actually screaming a fast so Like even things like YouTube and all that runs pretty flawlessly on just the software Render graphics using like simple DRM. So like it's it's not a blocker You know, it's it's nice to have the hardware accelerated graphics But you don't need it for like day-to-day usage unless you're into gaming or that kind of thing Yeah, yeah, so to be to kind of add some more color here to this Up until two weeks ago. I produced two variants of the same kernel build one that had the GPU driver and one that didn't and The default that we shipped up until two weeks ago was the one that didn't have it So I actually fully okay with the idea of us having that turned on in the baseline and then you know Wait, you know, my kernel build is basically waiting until we can get to a point that that can Be integrated upstream. So like that's the reason I propose the MR upstream to get the 16k flavor in because We're getting to a point within the next couple of kernel releases that we will probably have all the basic bring up for Non hardware accelerated graphics desktop usability or I think a Mac mini you can already run it on the Yeah, I'm talking about the laptops. Oh, yeah, no for the laptops for sure But I think the Mac mini has been fine for me. He's already fine. Yeah, I mean he's been fine for about a year now But the extended power management features that are needed for the for the laptops Those are just now getting the process of being upstreamed and stuff like that Yeah, so for the most part when it comes to enabling new hardware platforms, it's device tree updates There has been I think a grand total of one driver rewrite upstream And that was for the mailbox driver because it turned out the mailbox subsystem was a Really bad fit for the for it And so the driver got rewritten to not depend on it anymore and use its own queuing model for for stuff And beyond that, I don't think we've had any like major subsystem rewrites But in general, I think like We're getting to a point where like basic functionality can be enabled for almost every use case in the upstream fedora kernel And the reason we want to have the 16 and 4k flavors is because there's some other stuff We're trying to enable where we could have you know host guests being host 16k guest VM 4k doing some interesting stuff And we we're trying to figure out what that's going to look like And so one of the things that I want to switch to in the next few weeks as soon as my flavor MR is merged is reconfigure our downstream kernel to be kernel 16k and get the infrastructure set up for some of this other stuff that We're working on the pipeline for having the the dual kernel install set up for Yeah, I hope 4k is not going away. I still need it So and yeah Is there a particular reason why speaker support is so difficult to implement which speaker support Oh, you don't want to blow up the speakers because they're not they're not there's no firmware on the speakers to actually keep them from blowing up They it's all entirely software controlled by the operating system So you can pump as much volume as you want and you can fry them on the laptop specifically You can physically damage the hardware if you if you're not careful So the idea is to do this in a way that it's safe so that users cannot Easily damage their machines. So we technically everything is plumbed in but we have hard disabled all of the access to it by default Because blowing out your speakers and then taking it to an Apple store there And it is a dicey move if they're going to repair your laptop after you've blown out the speakers like that There's Has it been hard getting the hardware working on like M1 and M2 since a lot of The stuff in my books are proprietary and don't have much documentation on it. No really it's been easy So the thing is that Apple So I talked to Hector a lot about this in the background because I'm interested as it turns out Apples like every other normal PC maker that they don't actually like everything changing all the bloody time because it makes their Own software development lives hell. So what they've done is that they've built essentially an internal standard platform And they build on top of that for every successive platform So a lot of the stuff that we built originally for M1 has worked for M1 pro max M2 and so on on top of that Apple has a pretty high tendency of reusing their hardware configurations across generations and so a lot of the driver stuff We've done has essentially been able to be one-to-one reused The main thing is that oftentimes ports change in the internal configuration for device trees and whatever But the actual drivers and enablement work Tends to not be too bad. Sometimes there's some new quirks or whatever and that has to be dealt with But that's no different than what happens on x86. Yeah, I think between M1 and M2 the main thing That change was the controller for the touchpad I believe is completely different and then new instruction that screwed up all the browser and there was a There's a new instruction in the in the CPU itself that changes things a little bit, but yeah Actually, I was kind of wondering how like the touchpad Authenticated pointers. That's what it was that broke all the browsers. Oh the touch bar There is ongoing work on that. It is not ready yet, but yeah, because it's a haptic touchpad So it's probably gonna be harder than the touch pad is the touch pad is mostly done It's a touch bar, which is the the other stuff is that was that the patches were originally written for Intel max several years ago But none of the people that were working on it really wanted to try to shuffle it to upstream The ASAHI Linux team essentially forked the patches and reworked them to submit them to clean them up for upstream use Because there are still three generations of ARM based Macs that sadly have the touch bar and so we need to deal with them So yeah I think we've covered everything on this one. Yeah, let's go so Because we also covered this quite a bit during the questions that came up But essentially like our goal is to eliminate the the delta as much as possible and try to Get everything into mainline fedora, you know with the long-term goal of actually eliminating the ASAHI Remix entirely and just have mainline fedora for all of this stuff So more to come in the future on that. Yeah, what I suspect will end up happening down there Is that we will as new generations of hardware come out We'll probably always have their emix in some form for the bleeding edge support for this new hardware to use as kind of an incubator To then get the support into fedora proper right, but we'll see it's hard. It's hard to tell in advance how things will develop And of course, you know, if you're if you've got Apple Silicon hardware and are interested in this stuff you can come join us we've got an issue tracker a mailing list and Also, if you're you know, if you're a user or or interested in in discussion and support we have support forums In partnership with the ASAHI Linux project where all ASAHI just a forum-based discussion is moving to on Discussion fedora project org and we have a matrix channel where you can all join And I assume and and I realize now we did not actually put a link on this thing to the to the actual Remix website so I would put that up now so you can You can you can you can have a reference if Firefox loads There it is. At least it's a lot easier when we can actually read the screen. It is a lot of these Yeah, if you weren't here before we're trying to set it up before we had the spy monitor here So I was typing and looking over there But yeah, it's that at that fedora dash outside the screen is dot org And that has a lovely curl pipe bash command that you can run on an Apple Silicon machine Right now that points to the daily bills that kickoff every day at the 5 p.m. UTC I believe as we said the daily bills are daily development bills that are mostly untested So they may or may not work blow up your machine Nobody knows That said that they have been pretty they were pretty well lately David Matthew wants to know if you are on a discourse. I see you have a mailing list. We got both. Oh, okay All right, was it on that page? Yes, it was on that page. Yes Can I bring that back? Yes, you can. It was here. Yeah discussion Is it live? It is live. Yes. Yes, we are. Thank you. We are actually I Don't think we actually use the mailing list that much I think we don't I locked I kept everything locked until we were ready to launch So like the mailing list was also available But like I wasn't letting anybody in or putting any messages or whatever now that we're actually launching and going live with The public presence both presences are now active You can subscribe and start having messages and people will start talking to you and whatever whichever method you actually prefer Communicating we're available on both. Yeah, and most of us hang out in that matrix room. So you can also we can also reach us there There's a link to the weekly page for the SIG for the Asahi Linux project itself and for Fedora and Yeah, I think that's all we have. Yeah, actually, I want to make a special note here that discussion Topic just like the mailing list it includes both upstream Asahi Linux discussion as well as Fedora Asahi remix discussion. So Asahi remix the Asahi Linux contributors a number of them have joined the Fedora project to work with us and Collaborate with us directly within the Asahi SIG so their members of the SIG and are available and present for People to you know, get feedback and talk with them and and everyone can benefit from that Right and that's the reason why it exists where it does in the discussion forums as a separate category and all that other fun stuff Any other questions? Yes All right. This is a bit of an ignorant question and I know nothing about any of this, but how much like Does the development on Asahi sort of impact development for other similar sort of platforms like risk 5? Like is there any sort of transitionary work that you think working on this can? I think it helps in the sense that getting better support for ARX64 Is helpful in in like breaking the x86 monoculture So when you have software that only works on x86 64 The jump from getting it to build there to getting it to build on ARM Definitely makes it easier down the road to also getting support in our risk 5 But there is five is different enough from ARCH that I don't think work directly on ARCH Usefully impact work on risk 5. I think it's mostly going to help people from a mindset perspective It's not really going to help people from a Coder technology perspective Thank you David Yes, okay, so I I want to qualify this first by saying I don't use either environment, but I was curious why KDE it was chosen over GNOME since we promote GNOME as as kind of the other Interface or the primary interface on other things and then follow up Will those other environments work on this or is it are you primarily just focused on KDE right now? so So there's a couple of there's a few major reasons for this so the the first major reason why it's KDE and GNOME at all as opposed to just all of the all the bloody variants is that The primary requirement for full full support from the ASAHI Linux community is that it's got a run on Wayland and So it needs to be a full stack well in environment bottom to top because there's enough weird bugs and edge cases and brokenness in Xorg that can't be fixed for the Apple Silicon platform that It just wasn't feasible to to do any of that sort of thing From the remix perspective as the primary maintainer of the remix My particular requirements for any other desktop environment is that somebody has to want to step up to support it and Be able to be willing to accept bug reports and and work on fixing things and it has to be a It has to be able to run in a Wayland based environment to take advantage of the enhancements and fixes We've done in the Wayland stack specifically for Apple Silicon platforms Why KDE Plasma over GNOME? Because KDE Plasma was actually willing to do a lot of work for for the upstream community The ASAHI Linux community to adapt and fix things to make the Apple Silicon hardware be responsive on the desktop One simple example is Keyboard brightness control there are no keyboard bright There's no hardware buttons for keyboard brightness control on modern Macintosh computers And there is no way to control it in GNOME without a button. They fixed that last week They did fix that last week, but for a very long time. It took a long time There's other things around like between impedance mismatches with with modern several other things where The relationship for upstream with KDE was considerably warmer than it was with GNOME and the end result was The people that are working on this stuff and daily driving and actually being able to fix things and get things working We're able to get it done faster and better With KDE Plasma than they were with GNOME now I would like for both desktops to be equally well supported But at this point in time at least, you know There are some gaps in the GNOME experience on ARM platforms just in general We're not even talking about Apple Silicon But in general there are several weaknesses in the in GNOME on ARM that are not present in KDE Plasma on ARM Because there's been a lot more work on KDE on ARM than there has been GNOME on ARM Okay. Well, so that's good to know. So if I was to go and buy an M1 MacBook now like KDE would probably be what I want to use But is GNOME aware of these like are they getting bugged? Yeah, we've been talking to them And and that's like I mean part of the reason I was we even ship both is because there has been progress on that front and Because Fedora gets new GNOME basically every release We can see the incremental improvements as they come out and they're you know My hope is that with more exposure to this and people actually actively seeing this and being able to use GNOME on a real ARM platform that any everyone cares about That we will actually see arm the quality of GNOME on ARM improve Yeah, for what is worth I run GNOME and it's mostly fine Like there's definitely little areas that aren't quite as polished But it's perfectly usable the other thing I would mention with our other KDE and GNOME Is the ASI Linux distribution they had based on ARCH That one use KDE by default and there was a mild preference from a bunch of the upstream as I developers towards KDE So that was another reason why we favored going with KDE as the flagship here. Yeah And and the thing is that between the two desktops like we support them as well as As anyone would expect within Fedora. Yeah, and if somebody wanted to support other desktops by all means there there's no Like if somebody's willing to do the work we're more than happy to build build images for other variants Yeah, provided the requirements are met. Yeah, we can do it News after step no Does that run away? So one one other question which is an idle curiosity at my point, but you probably have the answer to I want to take to build a kernel RPM on an M1 Mac 22 minutes Excellent I Do it enough times on the laptop. Yeah on the laptop. It's probably faster if you get like a They have models that are like I'm an M1 ultra on a Mac studio or something that have more cores Those are probably So I have a Lenovo X-13S which is fairly difficult to install it right now But it is the closest thing most people can get without having a jet engine in their room To build a reasonable AR-64 kernel so as a development platform The M1 would be a decent AR-64 development platform. Yeah, that was the reason for my question Yeah, I can build chromium in 36 minutes on there So it's very fast There are some workloads where these these machines are really closely as it turns out performing as it turns out It's very very very good at compiling I just wanted to add something to the comparison between Asahi and the X-13S effort Two things these the Apple Silicon is faster So you're going to get faster builds, but another thing I find super useful For development personally because this is my main development machine this M1 Mac mini is The the firmware is unlocked down to the EL-2 layer so you can use KVM and that's something we don't have on the X-13S Yes, and I heard rumors of that so that's great to hear So for the because the peanut gallery was talking here the the X-13S that That we're talking about here is supposedly maybe going to also have KVME things in the near future. I Actually have a question, and I'm sure rise it wasn't brought up is Apple contributing to us. No Apple is completely gleefully blissfully happily. Hopefully ignoring us, and I'm okay with that I will actually it brings up an interesting top interesting point, right? Like we've gotten very used to this idea that vendors are essentially going to be supporting the harbor platforms directly in the Linux kernel and that has actually led to a Little bit of an inconsistency in how well Computers are actually supported in Linux because we've kind of gotten used to not having to do it ourselves And not having to understand how the how the computers are actually working for for the Linux level the The bring up for Fedora saw he remix has actually been one of the more joyful Platform bring-ups I've done in a very long time, and I typically dislike arm So that's a that's a it's quite a bit of praise there from the x86 perspective if if a vendor isn't Involved everyone just usually gives up and just says go find something else and sometimes something else isn't a thing that you can get and It leads to a very sad situation The nature of how the Apple Silicon platform works with all the people that are actually interested in people actually caring about it We're seeing a much higher quality Linux platform Coming out of a hardware platform that the OEM just isn't Doing anything for then we see for a lot of x86 platforms That you have vendors partially or or some degree kind of supporting because the community has much more ownership over the platform Then you have for the other stuff So that's a little bit of my soapbox when people talk about x86 versus arm and Apple Silicon stuff Okay, thank you more questions Just kind of following on from that last question I'm curious if you could speak on you know thinking Towards the future, especially in the last few talks. We had your talk made a talk on risk five And the these new platforms are coming about and the work that it will take for Things like Linux or the Fedora project to support those platforms I'm thinking forward in like the next decade or two Do you have any concerns or hopes about like what that support means for our Communities if you think it will be an additional stressor or if you think it will open up new opportunities Or just like general thoughts, I guess I think in general You're likely to see a wider variety of use cases stem out from this because I think especially That you can see these on arm outside of the desktop and server armies very properly in the example in the embedded space But you will see this we as five as well Where I suspect that would be more more examples of non-traditional use cases that will be that would be well enabled where something like Fedora Or like a major distribution would be a usable platform there like in the past on the embedded You often had people build their own because the system was so constrained These days you have systems that are massive compared to what you had even a few years ago where it is actually viable to say Oh, I'll just run like Fedora workstation Plus my software on top and it's gonna be fine for for some use cases not for everything of course So you would probably see more of that In terms of support ability and workload. It's hard to tell but I think if we continue to see the kind of interest We have seen for example with the as I project where there's a very excited community doing work Doing actively work and doing work the right way like getting the work upstream in there in the projects upstream So it can be maintained in the future that I think both very very well Where I think this breaks up a little bit is when you have say a vendor like making a tree dropping it dropping a BSP on the floor and running away and Then that kind of withers because it's sometimes people pick that up and I go to the table of upstreaming and get It was supported but it can take like a decade for that to happen And then the hardware isn't shipped anymore. Maybe or it's just obsolete or it doesn't make a ton of sense You see these a lot with phones where like yeah, you can run Linux on a pixel from I don't know five years. Yeah, it's Things like that. Yeah, and and this adds a little bit of extra context to my comments earlier is that one of the one of the more interesting challenges that we're gonna start seeing is Where is the balance between community versus vendor enablement for? For platforms whether it's in risk five or arm or or even you know x86 or whatever My personal desire is to see more of the balance tip towards the community Rather than towards the vendors because the vendors seem to not Be that responsible with it? We've let the vendors kind of drive How they're going to consider support for Linux and it hasn't paid off nearly as well as I would have liked in Across the different spaces some of them have been better than others And it kind of depends on who you're working with and what they're doing but when it comes to community stuff people don't have the bandwidth to be Willing to just say well We're gonna do this and then drop it on the floor and then go do something else or whatever Most people don't have the money to do that much less anything else a much less the time So you tend to see the much more methodical careful correct way of doing things when you have the community-centric approach Towards these kinds of enablements It's slower for sure. It's sometimes frustrating, but it tends to lead to a more correct solution over time He's got a I would actually like to just strongly Counter that statement with we've worked for decades to get vendors to do as much as they can to try to push this Stuff upstream I get there are some that are horrible. I'm not going to name names There are some that will do nothing. There are some that will write a driver maybe Lob it upstream and they leave it that there's various levels of but The more the vendors get involved the better it is for everybody I'm not saying the community-centric approach. I agree. Yes, but but even if they don't We need to be pushing them in that direction rather than saying don't just half-ass it and let the community Take care of it instead We need to say you need to do better as a mender now I get the Apple Silicon case is a completely different case and there they They have no commercial interest in doing it, but as a whole as a blanket statement I think we need to be pushing hard for vendors to continue to do the right thing and move in the right direction. I Agree with you in principle. The problem is that I'm so burned out from it Like having experienced this, you know in in the private space in previous jobs in previous experiences Like, you know, I used to do arm-bring-up stuff as part of my job for some things I used to do, you know, platform stuff. I used to debugging. I've had to write drivers before I've had to debug drivers with vendors before I Am all burned out from most of that and that and and I imagine that a lot of other people are and the problem is Most of us that are out there that are working in this community space who want to make that who want to pressure the vendors do They don't actually have a voice. They don't have a way to talk to anybody and And this kind of kind of comes to the larger problem of like We don't have commercial Linux desktop not really and without the commercial Linux desktop thing to like create this virtuous cycle of enablement and support and and and communication because at the Business-to-business level it's a lot easier to do these kinds of things whereas at the community level it's way harder and so With with what I have and what's available to me. I can't really do anything about it I agree. I mean it's been a problem for a long time. I get the burnout I mean, you know, we've been doing this since 1995 now. It's it's it's frustrating But that doesn't mean that we should give up and just say don't do anything because we need to be moving I mean, how long does it take into I realized we were a long way still from having an up-screen Fully working in video driver. How long did it take though to even get them to? Do you know do what they did and open up their specs and and open source their driver almost 20 years exactly? Yep, exactly what I'm saying there. Don't give up because it can get better. Yeah The only reason we have working cameras in our laptops is because of Chromebooks Otherwise, it would be a binary which also comes back to that there was a commercial That gave it right like I Don't okay I'm gonna say this Google is probably the best thing that ever happened to Linux desktop because they drove a ton of open source Upstreaming effort for both x86 and ARM platforms that made the infrastructure for desktop Linux way better There's a lot of pieces under the hood that actually came from them for building Chrome OS that we rely on today Or we are about to rely on today or that we are integrating and adapting from And that's because they They felt the need the desire and the stuff wasn't there before and they had the power to pull it off The the part that makes it difficult for me And this is one of the reasons why I really liked working on this Apple Silicon platform stuff Versus everything else I've ever done in the past was that When as a person who is working on something like this Everything I'm doing is actually impactful and actually improving the space for everyone whereas when I'm kind of shouting into the void to some of these you know nameless People or whatever that like are supposed to you know care about some of these things they don't and There's only so much I as an individual can do with no real Backing or scale or support or any of that kind of stuff again. I'm just a guy in fedora trying to make Desktop Linux work. What am I supposed to do? Yeah, there's a big difference when you approach these from the point of view a company with a commercial relationship And you have certain levers you can pull if you're approaching this from the point of view of a volunteer that Either wants to do work in a specific area or as a user that would like to see development in a specific area And doesn't really have a way to do it themselves besides. I don't know donating to a project that knows they're doing it It's definitely more difficult to see what is the approach to getting viable support for X No, I'd like said I get it. I've done that and you know, I've done it with all I did with x8664 And ended up with with personal contracts with vendors because nobody else knew how to Seriously, it's weird But it's what it doesn't mean it's worth giving up on so I realize it's frustrating You still push forward you still work around you work around it and make things work as long as you have to But in the meantime, you still let's not give up. Yeah, I'll be clear. I'm not giving up I just I have for the most part stepped out of the fight for a little bit But I'm happy to encourage other people do it and it's not entirely hopeless as you said and video opened up Other people have been doing it. We've got Lenovo doing stuff upstream, you know, occasionally Dell does some stuff and you see HP sometimes now I've been seeing a lot of improvement on that space heck I think even aces and acer are starting to do stuff like it is improving But it's still a giant patchwork and there's still no community there's no avenue for community pressure to make an impact and that's something that we do need to solve because Almost all of these things are not They're not company-driven. They're the like the needs and the usage is by individuals And so if that's the case, we need an avenue for their voice to matter And I don't know of one right now with the Apple Silicon one with the Asahi Linux project The individual voice has mattered and that's what's led to like tremendous improvement so quickly The bring up to make this a fully usable platform for his uses. It only took four months. Yeah Like that's impressive and that's because being able to work with the people and get tight feedback loops and actually having people like Care about what people are experiencing that makes a huge impact whether it's a company or an individual it does But you also had years of Lenaro working behind the scenes getting getting AR 64 to the way it was to begin with I mean, it's not just that there's more to it and We've got to keep applying that pressure in any way that we can Yeah, sure, it's just I'm just saying like from an individual from the computers that people are actually using Like the fact that we're able to bring this up so quickly and now have a wonderful experience to share with everyone With in partnership with an upstream community this phenomenal. We haven't done anything like this before Yeah, no, but I think it is a good point to to raise that this this isn't something that came up in a vacuum It definitely built on the work that was done for like for that like at least that decade and change From Lenaro from other partners that try to develop the arts I mean hell I'm space and then the airspace in a viable one. That's that's for sure something that's worth noticing I mean hell arm for links started in 1999. So like, you know, it's it's not like it's new It's it's a very old thing that's incrementally been built over the literal decades This might be a silly question, but for the Apple hardware and And maybe for other arm hardware as well. Is there a Firmware update story at all for Yes, and it's hard to have this conversation in general So I for for Apple hardware specifically the way the firmware works is that the firmware is distributed alongside MacOS The firmware itself isn't Re-distributable meaning that you can't copy it and serve it from your website, but it is on the machine So the way the process the provisioning process for the machine right now works Is that the installer will grab the firmware out of MacOS? And that's how it's available on Linux and then on Linux is installed like any other firmware Using the Linux firmware interface, right? Because most of this is Like non volatile firmware. Yeah, like firmware that's loaded around time There's also firmware that is updated on the machine itself and that's updated as part of the MacOS update cycle But that is something that is specific to this platform in general for ARM This tends to be something that is very vendor specific That you would need to see and it's also component specific because oftentimes you'll have like a mini PCIe Wi-Fi card or whatever that comes with its own firmware concerns You the firmware for the main system you might a firmware for like weird additional co processors and stuff so it's very hard to Answer these in general terms for Apple Silicon Silicon platforms. The main cases to think of is Whenever you update your Mac OS operating system container environment, which is the boot environment volume not Linux containers But like volume container when it gets upgraded the firmware is downloaded installed onto the system and the system DFUs into that environment Going forward then it reflashes itself and boots up with with that new version of the firmware when we boot into Linux every time We poke at what's available from that Mac OS has put into the boot environment and we copy it in and load it to Initialize the hardware. So the only thing we actually have to maintain is the draket module That was written to be able to pull that stuff out and boot and initialize it and the device trees Which actually know where the where the hardware Configuration actually is to connect the kernel to the hardware to be able to do things and the parser to extract the firmware Fuzz from the like proprietary format that I use for that right You are required to maintain doable. Yeah with the ish. So you are required to have There's two parts of this right now the way these are deployed. Yes They're designed so that you have Mac OS alone. You have a full Mac OS alongside your Linux install The full Mac OS isn't strictly necessary But right now it is the easiest way to do firmware updates. So that's why we're recommending people to keep it the way the Linux side is provisioned is that you the installer makes a straight-down Mac OS container and then Effectively repons the kernel from the Mac OS kernel to the Linux kernel to boot it So you have like a skeleton Mac OS that is used as scaffolding to boot Linux And that one sticks around effectively forever, but it's completely inert once you boot the Linux It just wastes like a couple of gigabytes of this space. Yeah, it's essentially Going to be what we do in the PC world with UEFI Yeah, it's like Mac OS has a giant bootloader, which by the way, that's also how it works on Intel machines Yeah, giant weird Mac. It's a special weird Mac OS running it firmware. There's a question back there It's a kind of like how I think Apple's Unix Worked it where it would like boot into Mac OS and then there are a number of disturbing parallels Yes, that was even more fun because if I remember right that went from a live Mac system and and you ran Unix from there Yeah, kind of that's what kind of sounded like it is it is definitely it's definitely fun in that sense I don't you can't quite Kexac from Mac OS to Linux although that would be entertaining Okay, although the pre-bootive environment is a technically a derivative of Mac OS. So yeah, I say you basically are doing These things have a couple of variants because Apple really likes building things in-house So they they have variants of their own core operating system that they use for various things So there's like there's a little Mac OS. That's the bootloader. There's a little Mac OS in the damn dungal for doing video output It's it's a very Interesting example of vertical integration in a company So the bootloader on these things is actually a stripped down Well, the state to bootloader is a stripped down Mac OS the recovery environment Which has a GUI and everything is a stripped down Mac OS and he runs the the bootpeaker thingy is a Mac OS up Yeah, and that also means by the way you have full hardware services You have Wi-Fi working at early boot and you can do Bluetooth and have audio going through an early boot. It's weird What did you put it in full-screen? That's why it was weird. Well, given that we are I can show you one quick thing. Oh Yeah, that's right. Yeah. All right. So while he's doing this I'll explain so the way that we actually boot into this system is we start with the weird Mac OS environment which then goes to to exact m1n1 or mini which then loads you boot which then load which creates a UEFI issue environment that then loads grub that loads That execs the Fedora kernel that then boots the Linux environment and you're about to see it So we will see if these works because I don't know if HDMI is actually going to sync unless you just invited the gods against Because these these may not sync but I will say there you go. Yeah, your first Mac OS environment Yeah, so this is the this is the boot loader And in the bootpeaker you can pick whether you want Mac OS or Fedora or if you do options He would put you in the recovery environment Which is like a full system where you can run like they who don't have this scan the terminal and things like that So if you pick federally, it says 37 because he's an old install We we changed the thing to actually say something more descriptive now and then it will chain the And then it will do this Now it's booting and when it boosts it copies it copies the partition into the boot partition And now it boots into into mini which you didn't see because it's invisible now It's putting you boot right now you boot is going to boot the e the UEFI emulation environment That's booting grab grab is booting the kernel. That's the prime of Thing that's going on the one on the back the f you see on the back comes from meaning actually Yeah, and then you get a desktop after a while. Oh and many apparently provides the The fake background thingy that we do on x86 like it can do it too Although Plymouth doesn't know that that's there so it doesn't load it correctly We just did it five boot environments in one go. Yeah Okay, perfect. Thank you. We are out of time. Thank you. Perfect. Alrighty. Oh And if you will have if you want to play with this There's a Macbook there that doesn't have any personal data on it So if you want to like play with a machine is there or you can find me and I'm happy to end it to you for five minutes Someday there we go