 Let's get started. Hi, I'm Robert Foss. I work for Collabra, as you may have guessed. And my name is actually Foss. It's not some sort of an alias. It's my mother's name as well. And she does not do any kind of software, really. So working for Collabra means that I work on open source for a living. I mostly do Linux graphics work. And this is what we're going to talk about today, the embedded GPU ecosystem and how it's doing and what's going on. So specifically, we're going to cover the history, what the open source support is for most of the vendors. And we're also going to cover what comes next and the bigger picture of what the industry looks like and how open source drivers seem to come about. So we're not going to talk about the GPU you're making in your garage. We're not going to talk about very old architectures like the Viya Chrome GPU, for example, that Kevin Brace Bravely is working on. And the emphasis is also going to be on the embedded ecosystem, because it's what's interesting to us. So let's start off with the history. This is Intel. They've contributed for a very long time. They've been a very good citizen for a very long time. Their current driver, the i9-15 driver, was first merged in 2004. And they've been very active in supporting it since then. They support OpenGL 4.5, Vulkan 1.1, and all the applicable hardware anyway. So if there's a gold standard, I think Intel is it. There's also MD, which has been around for quite a while in the open source space anyway. And they're also a good citizen. They've contributed a lot, especially recently. But they've been very open towards the community since at least 2009, where they released the documentation for their then-current high-end GPUs. And they also support OpenGL 4.5 and Vulkan 1.1, which almost is as current as it gets. There's OpenGL 4.6, but let's ignore that. No one supports it at the moment, not in the open source space anyway. And then there's NVIDIA, which is very interesting. The Nouveau driver has been around since 2010. So that's a relatively long history. It supports OpenGL 4.5, so it has very good support. But NVIDIA itself has never really backed it. They have on occasion backed it, but not through and through. So they could be a better citizen. I'm saying that. And there are very specific things they could be doing, which would be very simple for them, like releasing the firmware for their GPUs, so that an open source driver could just load it and run it in the same way that the proprietary one does. Something interesting, though, about NVIDIA and their support is the support for their Tegra platforms, which was partly sponsored by an aircraft supplier. So they upstreamed, I think, the initial support for the Tegra platform. And that's something we'll get back to. There's also Qualcomm, which has an even shorter history in the open source space. And as you may be noticing, we're going through this in a sort of vaguely chronological order. And Qualcomm is becoming a rather good citizen in the sense that they're contributing code through the Coderora project. They didn't start the driver. They didn't start the initial development. That was Rob Clark, mostly by himself, I think. And their current driver supports all of their hardware, which is rather good. That's more than we can say for most vendors. And interestingly, there's a fun fact about the Qualcomm Adreno drivers. It was bought by or from AMD. The Qualcomm bought the AMD mobile handset division, 2009, I believe. And Adreno is a wordplay on Radeon. So it's hidden in plain sight, if you will. There's also Broadcom, which has been a very good citizen, but since very recently. They hired Eric Anholt in 2015, I believe, who has since been going on a solo mission to provide very good support for the VC4 platform. And the VC4 platform has very good support now. And there's more newer, higher-end hardware coming along, the VC5, which has partial support. But it's not really even available to the public in DevKits anymore, or already so. We're kind of ahead of us ourselves. There's also Vivante and the Etnaviv driver, which is fairly recent. It's from 2015 in Mesa, or maybe the kernel. The first patches were accepted. Reverse engineering of it has been going on since about 2012. And more recently, in the past few years, reverse engineering and development has been sponsored by another aircraft supplier, which is very interesting. I'm seeing a trend here. Apparently, aircraft suppliers care a lot about open source, which was very surprising to me. And support for their hardware is now very good. They're also coming out with newer fancier hardware, which has initial support. But it seems to be working out quite well, that too. And then there's imagination. Imagination is interesting in that they have some patches and some support floating around for their hardware, for the kernel and for Mesa. But none of these patches are acceptable to the upstream community. And that's mostly due to the fact that their patches are very partial. Like they support a very slim use case, or a slim set of use cases for their hardware. So they can't be accepted due to not offering full broad support. So there's, for example, a kernel driver for the PowerGPUs. And it's unacceptable due to there being no user space, the user of them or it. So Mesa has no support for it. There's also some Mesa patches floating around for it. But it's very limited to non-3D operations. So it's not interesting to the wider public, maybe, in some special use cases. And then there's ARM, which is fully unsupported. They have never contributed anything, as far as I know. And not to the GPUs, anyway. They bet very heavy on OpenCL for their previous generation of GPUs, which is interesting, the Mali TE something-somethings. But they wanted to sell the OpenCL support. So I think that that's descriptive of their perspective on open source. They would like to have their driver be a paid service. So they would like to make some money off of it, which is fair, but not really congruent with the values of the open source community. So with the broad, hand-wavy history out of the way, here's the open source support today, what it looks like. For practical reasons, I'm not going to have a look at anything before 2009. And the practical reasons is that my diagrams would be very wide, and I wouldn't be able to fit stuff in them. That's why. So 10 years is plenty. Let's have a look at NVIDIA first. And this is what the reverse engineering process looked like. This is specifically looking at a reverse engineering tool called NVTools. And around 2010, early 2010, the first support for the current modern-day NVJ GPUs was merged. That's Fermi for, let's see, it's also known as the NVC0. But since then, there's been a steady pace of support added for the NVIDIA GPUs. And the support comes in the form of adding register definitions so that you can see what the proprietary driver is actually doing when it's talking to the GPUs. You can decode it in a reasonable fashion. So about every two years, there seems to be a new GPU added. And let's see. Sorry, that was the wrong way. And this is what the reverse engineering work ends up looking like when it's actually producing a driver. So you can see that there's a little bit of a delay, generally speaking, from the reverse engineering effort. I think that's maybe six months. From the first Fermi support to the later Fermi support in Mesa. And there's also a corresponding graph for the kernel when stuff is merged. But it basically says the same thing, so it's not very interesting. So I just opted for the Mesa one. So Pascal is their latest GPU. So the open-source support goes all the way to current day to whatever you can buy in stores today, which is nice. If we continue with Intel, there's no reverse engineering timeline. Although there has been, I've heard rumors floating around that internally, Intel actually does some reverse engineering. Or the Linux team does some reverse engineering of their Windows driver to get better documentation. So that's kind of interesting, I guess. This timeline is very much cut short. It's of the i9-15 driver. It itself was merged in 2004. So yeah, we're missing a fair bit of time. So in 2004, a company called Dunkstan wrote the initial open-source driver for the i9-15 platform. And since then, Intel has been actively contributing towards it, and still is very much so. So if we look at the Gen10 hardware, that's hardware that isn't even released yet. So when you actually get a hold of that hardware, you'll probably be pretty pleasantly surprised about the level of support. It'll work out of the box, hopefully pretty well. So let's look at AMD. There's no reverse engineering timeline here either. I could probably dig one up. But since their entry into the open-source community with the R600 GPU, they have been providing documentation. So little reverse engineering efforts have been required, or at least fewer than normal, which is very nice. And yeah, it has, I think, helped the community greatly. So this is what their timeline looks like. The R600 around, let's see, 2010 was merged into Mesa. And the Vega 10 is the latest hardware that's available. So it covers basically the entire recent span. Something interesting about AMD specifically is that they have a lot of drivers, like a lot, lot. On the kernel side, they have two drivers. They have the Radeon driver and also the AMD GPU driver. So that's two. On the Mesa side, they have three drivers. They have the R600 for that very early generation of drivers. There's the Radeon. The Radeon driver covers a span from the Evergreen to Southern Islands, I believe. And then there's the Radeon SI, which covers Sea Islands and two today. So that's the stuff that lives in Mesa. But then there's more. Then there's drivers that don't live in Mesa. As specifically, there's the AMD Vulkan driver, which is open source and available to anyone. But it's not really hosted within the normal, like, everyday projects like Mesa. So it's living somewhere else, I think, on GitHub. So if you're interested, you can have a look. And then we have Broadcom, which is interesting. They have no reverse engineering chart, because much of the development has happened in-house by Broadcom, and specifically Eric Anhalt. He's the one guy dev team inside of Broadcom doing all of this stuff. And his work has resulted in very good support for the VC4 platform. So if we have a look at the graph, it's pretty empty. There's not a lot of hardware in there. But the VC4 hardware has been supported for a good while, and it's still being actively developed on. And coming down the pipeline, there's also the VC5. And the VC5 isn't really available quite yet. So it not being totally ready yet doesn't matter. That's fine. If you look at the dev kits you can get a hold of for this stuff, I would recommend maybe a Raspberry Pi 3. They're cheap, fast, available, and would let you try out the VC4 stuff. And then there's the VC5 hardware. There's an SOC number. I don't think it's available anywhere. So who knows? Maybe it'll be in the Raspberry Pi 4. No idea, but maybe. Continuing on, there's Qualcomm. And this is what their reverse engineering chart looks like. So it started a few years ago with the A200, A300. They were added basically simultaneously. The A200 and A300 hardware was very much available at the time, or both were available at the time of the first reverse engineering. They were added pretty much simultaneously. And this hardware is what you'll find probably in your Android phone, most likely. And it also covers the entire span of available hardware from Qualcomm. The A500, I believe, is the very latest hardware you can get a hold of. So it is well supported. And much of this work has been done by Rob Clark. And this is what the support in Mesa looks like. So it's almost like same day, at least for some of the hardware targets. The A500, the most recent hardware, was reverse engineered a little bit before the actual support was added to Mesa. But it's all in there. And it's in good order too. The driver is very mature, so that's nice. And if you want to have a look at the hardware and develop it for yourself, perhaps, or develop the drivers for yourself, this is the first or the most reasonable A200 development kit I could find. It's the NXP IMX53. It should be pretty cheap by now. It's somewhat old. It's three generations old, I think. If you want to have a look at something more reasonable, there's the Dragon board 410C, which is a pretty current dev board. I use it every now and then. And it's based on the Qualcomm 410E SoC. And if you want to look at the A400, there's the Inforce board here. It has a Qualcomm 805 SoC. And lastly, there's the most recent board. I'm not even sure how available this is, but I think it's starting to trickle out of Qualcomm now. There's the Dragon board 810C, which is based on the 820C, which is based on the 820E SoC. It's standing for embedded, I would presume. I don't know. Yeah. And support for this is pretty okay, I believe. There may be some kernel stuff that's still being worked on, but it should be usable for most use cases. So that brings us to Vivante, which I've been sort of working with a fair amount recently, which it's pretty interesting the way it was developed. So the reverse engineering work has been, I think it started in 2012. And the Vivante GPUs are very common. They're very low priced, so you'll find them in pretty much anything that's cheap. And, yeah, this is what the reverse engineering effort actually ended up in. So the driver was introduced in 2017, 16, yeah. And was done in large parts by Vladimir van der Laan. He did much of the reverse engineering. Christian Geminer also did. Lukas Stack and Pengatronics also did. And the support for this hardware, I think the entire span is the GC200 to the GC3000. It's very mature. It's pretty good. And then lastly, there's the GC7000, which just has the initial support added now. And you can sort of maybe get a hold of a dev board. Maybe not. It's a little bit early days for it. So if you want to develop stuff for the GC2000, there's the IMX Saberboards or IMX6 Saberboards. For the 3000, there's the IMX6 QP Saberboards. The QP is basically just a higher end version of the normal IMX6 with a little bit fancier hardware. And then there's this dev board, which is barely available. I don't think I could get a hold of one if I wanted to. So hopefully it'll be easier to get a hold of. But the developers of the Vivante or EthnaVib driver have some hardware. So development is proceeding. So everything is good in that respect. So let's have a look at ARM. There's actually something to say about ARM, which is pretty pleasant. Some stuff is happening. The Mali-T series of GPUs just drew its very first triangles. Maybe I think it was like a month ago, which is very recent. And also there's more of our continuous, ongoing reverse engineering effort for the Mali-G series of GPUs. There's not much of a driver, and none of it has been upstreamed. So you won't find any support for this in either the kernel or the mesa. So it's all based on tooling. So it's still the very early days for ARM support. But if you want to have a look at this stuff, here's a dashboard that you could use. It has the ROCKSHIP RK3288 SoC. I think it's like $70 or something like that. It's not too bad. And if you want to look at the more recent G-Series of GPUs, there's the high-key 960 board, which is being used for development of all kinds of things, actually. It's pretty interesting. And support for it is pretty good upstream, I believe. So that brings us to imagination. And I don't have a lot to say about imagination. There's not a lot of support. There's not really any reverse engineering going on. And they don't seem to be too interested in actually contributing towards the stuff. So I don't have any dev kits to recommend. I don't know. If you feel very brave, you can look this stuff up and see if there's anything that seems like low-hanging fruit to you. Yeah. So that brings us to what comes next. So if this is where we are now and where we've been before, we're also going places. And OpenGL support is being continuously improved. OpenGL 4.6 isn't really supported by any of the open source drivers quite yet. So most of the drivers with good support are at 4.5. So there are a few features still missing. There's also OpenCL support. Or rather, there isn't any, not in the normal open source ecosystem anyway. There's the Intel Neon support for the Intel GPUs. So this doesn't live in Mesa. It lives somewhere else on GitHub, I would presume, or Intel open source, oh, 1.0.org. And I think it supports the latest OpenCL standards, which is very nice. It's not an easy target to meet, but they've done it. There's also the AMD Rockum, which doesn't live inside of Mesa or any of the traditional repositories either, but supplies us with a pretty decent open source support for OpenCL. But for the community-only drivers, this support is very poor. There is none, basically. So if you wanted to run OpenCL on your Qualcomm Madrino GPU, you'd be out of luck, basically. And there's no such support today. But there is support coming along for the NVIDIA driver, the new VODE driver. There's also support coming along for the Fridrino Qualcomm driver. So what this looks like today is that there is a need for an OpenCL compiler or front-end. And there is LLVM, and it supports OpenCL to some extent. However, in order to use it in the context of a driver, we needed to emit something that is parsable by our drivers. So we need LLVM to emit SpearV, which is a compiler-intermediate format. SpearV isn't natively supported by the Open Source drivers, but you can convert SpearV to something that is supported by some of the drivers. There's more work to be done in that space, too, for sure. So most of the current-day Open Source drivers are using a format called TGSI. And TGSI is not very easy to convert SpearV into. However, there's an alternative. There's a format called NEAR, which is a bit more modern than TGSI. TGSI was introduced, I believe, with the i9-15 driver in 2004. It stands for Tangsten-something-intermediate, I would presume. I don't actually know. But it's very old. It's been around for a long time. So it's well-supported, but it's not well-suited for the purpose. It's, for example, not SSA-based, which is what you really want your intermediate formats to be, at least for modern-day compilers. So the idea is to move the Open Source drivers to support NEAR, which would make it very easy or relatively easy to ingest SpearV into the driver. So that concludes the OpenCL support. Colabra is also working on this in terms of making LLVM support emitting SpearV better. There is a branch of LLVM for this, but it's not upstreamed into the mainline LLVM, which is what we want and what we need in order to support this stuff. There's also Vulkan support. Most vendors don't have Vulkan support. AMD does. They have lots of support, in fact. They have RADV, which lives in Mesa, which is the community driver. And there's also AMD Vulkan, which is the corporate driver. Both are Open Source and available, of course. And I believe both support Vulkan 1.1. At least RADV does. AMD Vulkan may not, but it should be soon anyway. And for Intel, there's the ANV driver, and it supports Vulkan 1.1 and also lives inside of Mesa. So it's as community-oriented as it gets. So that concludes the Vulkan support stuff. The big picture. This is the interesting part to me. If we look at the development perspective of the big picture, there are a few conclusions we can draw. For example, some drivers are very mature. The Intel ones, the AMD ones, the Qualcomm ones, they're doing very well, and they're very reasonable to use in a production device. And if you find any bugs, you know, just report them and someone will actually attend to them. So that's very nice to see. There's also lots of supported... There are lots of vendor-supported drivers that don't live inside of the usual repositories, specifically AMD, seems to be guilty of doing this, which is certainly okay, but maybe not ideal, especially not in the sense that if AMD used the same code paths as the other drivers did, there would be a lot to gain for the other drivers, but also less work for AMD. But presumably they see the value of having the same code base run on multiple operating systems as a bigger net gain for them. Yeah, and then there's this. This is pretty cool. Just looking at the graphs or the timelines I had, there seems to be about between zero and three years, seems to be the time it takes to reverse, from reverse engineering something to the day that you actually include the first support for a device in the kernel or in Mesa. So if we look at the ARM reverse engineering, which started relatively recently, that happened basically now. If we're very lucky, maybe we'll have some drivers in three years, maybe. It would be nice. And yeah, this is sort of reiterating what I already covered. Vendors support APIs outside of the usual repos, so that's what AMD does to a large extent. Also OpenCL is hard. There's lots of work that needs to be done and the ecosystem for doing it in the open source is not very mature. Lots of tooling is needed and lots of the very basic infrastructure is just lacking. For example, the compiler infrastructure with spear V not being really in place. And there's the vendor aspect to the big picture. Some vendor support is really awesome. Some vendors are really, really good. And some are not so good. Open source drivers come in a few different shapes and the way they're structured. They're the ones with vendor support and those that don't. For example, the Vivante driver has no vendor support, yet the driver is pretty damn good. So that's cool. There are small vendors. For example, Broadcom has the one-man Eric team, which is doing really awesome work. And there are big teams like the Intel team or the AMD team. Some vendors are more motivated to control their software stack. That would be maybe the arms, the imagination, the Nvidia. They are more motivated to own the software than they are to contribute to the open source. And then there's the Nvidia, which is just plain weird. They have a great driver, but they're not allowing us to really use their GPUs to the fullest extent by not publishing the firmware that is needed to actually have them work. If we have a look at the manufacturers, here's some interesting stuff too. Some industries really need open source. And apparently that's the aircraft industry. They care in the sense that they really have to support a device for 20 years. They can't half-ass it. But the Android product manufacturers are doing, where they sort of release a device, and it's supported to some extent for a year to three years, maybe. But their devices are installed, and they're going to be used for 20 years. And if a plane ever goes down because of a faulty driver, that would be pretty unacceptable. So they actually care about open source support in a very real way. So anyone who really cares about having good long-term support for their drivers should have a serious look into the open source drivers. It's one of those huge things that isn't maybe obvious. If you do run a proprietary driver, how are you ever going to move to a new kernel in five years? Probably aren't, or maybe you are, but your vendor will tell you to give them a bunch of money to do work. It's hard to make guarantees about using hardware that is unsupported or poorly supported. So it's hard to deliver products. Yeah, and that's it for me. That's all of the conclusions. Does anyone have any questions? In your opinion, do you have a chip manufacturer in mind that is doing the best in terms of open source software development? There are some that are really good. Like Intel, AMD are doing really well. Qualcomm seems to be doing, or they seem to start caring, which is very nice. They're actually contributing support for new GPUs. There are some that are too small or unmotivated, or some combination of that. And is there like any development boards that you could really recommend in terms of the embedded socks that are kind of leading the way? Some that are really good. Yeah, I think depending on your use case, like what kind of performance you want, maybe grab a Raspberry Pi. They have good support. They're cheap. They're available everywhere. They're a pretty decent option, I'd say. Thank you. Sorry, what about risk five? Risk five. So we're more GPU focused. I don't know. I don't think. So Mesa is independent of the architecture. As long as the kernel supports whatever CPUs happen to be using, that should be fine. No, it should be fine. So you haven't mentioned anything about virtualization. Yes. A lot of projects I'm looking at have use cases where a lot of people are doing hypervisors or other sort of testing and having, is there anybody with good virtualizations? So currently there's a fair amount of work being done to a project called Virgil 3D, which is rather interesting. It's used on the host, and the way it works is that the guest runs an open source graphic stack that has a Mesa driver called Virgil. The Virgil Mesa driver takes stuff, sends it through the VRT IO paths in the kernel to the host where an application called Virgil Renderer takes that very raw GPU state from the guest and pulls out OpenGL commands out of it and then runs that on whatever driver the host happens to be running. So you could run an open source host with a proprietary NVIDIA driver on the host. Sorry, an open source guest and a proprietary host if you want. So that's kind of what is currently being worked on. Any more questions? All right, that's it. Thanks.