 Who am I? I come from Belgium. I have a small consultancy company. We do a lot of Linux and some BSD also. In BSD, we do mostly free BSD and open BSD. Last year I was talking to the guys of NetBSD to see if there's any interest from them to get on power. For the moment, they don't have the resources and we also don't have the resources and with we I don't mean only just my company, but the other function that I have as the chair of the Open Power Foundation. So we will focus for now on free BSD and open BSD. This talk will specifically talk about free BSD, but open BSD also works on it. So yeah, like I said, a small introduction then specifically about free BSD. Where you can get resources, where you can start playing with things, and then some back to the future elements, where I will give highlights what will come by end of this year and end of next year. So introduction, when we talk about power and open power, you're talking from everything from the power eight and onwards. So even on free BSD, you have power PC 64 LE, so the little Indian version. And that's the one that is currently being developed. That's the one that's currently being pushed forward. That's based on the power ISA 2.07. And you can get these machines still today on eBay at a reasonable price. I know people who bought recently at like 1000 euros a machine and you can start playing with them. So that's still fairly okay from price. Of course, this is much older technology and it does not have all of the acceleration features that newer versions have. But you see a whole bunch of manufacturers there. IBM and Tyen being the two main ones and it's mostly the Tyen ones that you find on eBay very easily. Then the power nine, which is a complete update of the ISA. That's why you see 3.0. That one actually has a whole bunch of virtualization updates in there and also made some changes to how the allocation tables and the hash tables work. You see again a whole bunch of manufacturers there. IBM and Raptor being the two main ones. Raptor being a company that produces fully open source hardware. So you can actually go to their website, find all the specifications. They also made a power eight version. I don't think that one is available anymore, but the power nine is available. So the Talos two, which is the most common board, even now has a BMC card, which is fully open source. So you don't use the A speed and then you can bifurcate it and have the run BMC version from Raptor. So you basically have full control over your stack. This is the current version, so 3.1, also power 10. At the moment, IBM is the only one who's manufacturing this. We actually have Inspur, Raptor, yeah, I forgot Raptor on this slide, but Raptor will announce a power 10 very soon. Inspur has announced them already and there are a few other companies there. You see there, for instance, the LibreSock project, which is more of a single board computing project, so smaller type of devices they are targeting, which we hopefully will start seeing next year in the dev version come up at least. That's also the last line, power pi, power SBC. These are projects that we are trying to start up so that you could get smaller developer type of machines. They would have the same capability, they would just be less performant. So there's no open power machines. There are machines that use open power technology, which means that what the foundation does as, for instance, the power ISA is fully open source. Now a company like IBM can take that and make their power 10 chip. LibreSock could make that and make that fully open source. So under the LibreSock project, we actually want to build a power ISA compliant core that would be fully open source. You can go to the LibreSock foundation website and find the entire code there with all the instructions, the implementations and everything. And that's why I say this because while power used to be a one manufacturer CPU, we are moving towards multiple manufacturers and this will mean that you would have different type of power implementations because some people like LibreSock might not implement MMA instructions because it's not a requirement. Some of them would have accelerated memory instructions. Some might not have that. So that's the reason why I mentioned this here. Power in the past has been associated mostly with enterprises. This has to do with AIX and SAP HANA, which mostly run on proprietary power, so IBM's version of power. That's again outside of the scope of what I'm discussing, but I just want you to be aware that those also exist. People talk about power not being really fully open source. That isn't true today. So you will see that the complete ISA set is open. You can go to the website, find it. If you find a bug in it, you can actually report it through the website. And then the ISA working group, which manages the document, will actually give you feedback on, well, this is a minor correction, thank you. Or this is something more complex, so let's discuss this further. You do not need to be a member of the foundation to do that. Obviously, we as the foundation like companies becoming member, but there's no requirement for that. So that's also why it's important to understand that this is really fully open source with a feedback loop. You can give feedback on any part of the technology that's being developed. And yeah, then the most common mistake is power today is not just IBM. So IBM is still, of course, the major company behind it, the major contributor, because they invented the ISA, and they will still continue to drive a whole bunch of things, but they will not be the only company behind it anymore. And yeah, power is ancient. Yes, it is. It was developed in the 1960s, but it's still functional. It has evolved over time, and that evolution you've seen in the way that it is being put forward. So we don't like breaking backward compatibility, which is why certain features take a long time to get in, and others take a long time to get deprecated. Then the biggest confusion and the biggest thing that changed is when IBM started open sourcing the platform, they made a conscious decision to go little Indian. This was mainly driven by the Linux people, so it's not something that was 100% their choice. They were advised that little Indian would be easier to develop, and so they pushed everything to little Indian. For now, I think FreeBSD still has support for big Indian. The main problem will be desktop applications. So a lot of desktop applications only work in little Indian, and with desktop applications, I mean Firefox, JavaScript, and all of those type of applications you want if you're running a workstation. So that's why you will see a lot of the work is migrating to little Indian. If you're using it for server purposes, you could as well run big Indian. And on the network stack, if you're running for routing, or even for high-end storage, it makes sense to keep it in big Indian, because it's faster. So on the other architectures you have threading, typically most other architectures only have two threads per core. On power, we typically have four or eight, depends which type of system you have. So on the power eights, it was always eight threads, which means that on a small machine, you could get up to 192 threads, and in the latest one, it's 480 threads. So you'll see that the small and the enterprise are the same on the P10, and that's because a small machine is a single unit. An enterprise is typically four sockets per node, but you have four nodes that are grouped together. So that would mean that you have 1,920 threads in one keg, as IBM calls it. So it's a lot of threading. Because they can share memory, they actually have a shared backend infrastructure on it. You'll see the P9 small is the only exception where you have four threads that was done for marketing purposes, because technically there was no reason to do it. It was done for marketing purposes. IBM sometimes says it was done for thermal purposes, but that isn't entirely true. But yeah, most of it is eight threads nowadays. So I was mentioning that everything is fully open source. The device is completely open source. Now you will find back that a power machine is not a machine, you can actually press a button and boot up. So it does have a power button on the front, but the power button is not the power button like you traditionally know. So what happens is once you put in the power on the back of the machine, it actually starts the BMC. And that BMC is a computer in a computer. And so when you press the button, it's actually the BMC IPL in the machine. And so IPL is initial program loader, which is how the machine gets booted up. At that point, host boot takes over and host boot will actually initialize all of the devices in the machine. So it will do a memory check, it will do PCI bring up, it will do memory training and all of that. Once it has completed, ski boot takes over and ski boot is actually the Linux kernel, an embedded Linux kernel that runs on the machine that will initialize all the processors. So once that's done, then the actual processors and everything has been initialized and you get petite boot. If you were in the talk of Netflix before me in this room, they were also talking about petite boot because of the PlayStation 3. Petite boot is nothing more than just a TUI, so a text-based interface where you can select what you're going to do and how you're going to do, whether you want to pixie boot. But it's running a Linux kernel at the back end. At that point, if you would say I want to load FreeBSD, what it does is it KXX the FreeBSD kernel and then boot FreeBSD on that. This is the same on the Talos machines, so it uses exactly the same. Sometimes you'll see it referred to as OPAL, so that's the open power abstraction loader. OPAL is an open source project, so you can go to GitHub and you can find all the code. If you actually just give it, let's say, like Talos 2 and you build it, it will build the image completely for you. If you want to add there for other pre-loaders, you can actually do that. So let's say you have a GPU card or a special SATA card that was not traditionally foreseen. You can add those drivers in and it would actually boot that and give you the ability to boot from there. So this is also a fully open source. There's a device tree listing that needs to be filled in by the manufacturer. Every manufacturer is required to put that online and you will find them all available somewhere. So like I said, you typically do not boot a machine by pressing the button. There's a BMC in there. So the BMC is your open baseboard manager, like you know on other server architectures. It's a computer in a computer, so it actually loads up and when you press the button, it will boot up the machine. On power-based machines, that's running open BMC. If you have older machines that will still be on the ASD, so the ARM architecture. If you have newer machines that will actually be based on Libre BMC, which is an FPGA implementation of the power isa on a device that you put in a board. So in that way, you actually have a power device booting up another power device so that you can actually work. That one is only available on newer machines. It's something that was developed last year. You can upgrade specific Power9 machines to use this. So the Talos 2 with that upgrade card is there or the developer board from IBM also has that. So that's a special machine where you take out the ARM one and replace it with the FPGA implementation. The interact between the two typically is referred to the self-boot engine. So sometimes you might see the self-boot engine that's nothing more than these two BMCs interface to the actual machine. The interface is there only to boot up the machine to verify temperatures. It's not like it has access to anything of the kernel. It just needs to control temperatures, make sure that you don't blow up your machine and sometimes it can initialize or reset PCI slots. So for FreeBSD, what does that actually mean? We have support for open power from FreeBSD 12. It's the PowerPC64LE. At the moment, it's still tier two. So we're working on getting that in tier one. There are some bugs that still need to be fixed. For instance, if you have a power nine, recently the bug that got fixed is using the hash table completely and using the red X tables. So you can actually get acceleration and full acceleration. One other thing is the NX engine within power nine isn't being utilized for FreeBSD. The NX engine is the compression engine. So it's a special co-processor within a power nine machine that actually can do compression offloading. For the moment, that's not supported. That would make a huge difference for GZIP, LZMA, BZIP two and maybe one or two other ones. But that's something that we need to look at. Like I say, it's still work needs to be done. A lot of the parts are also not fully supported. That's another thing I want to look at. The foundation has been helping us. And I mean the FreeBSD foundation. There's also people from Raptor and from the open power foundation that have been helping out. So I'm not claiming I've done all of the work. But we did get a loaner machine from IBM specifically for FreeBSD additionally. Which means that if you have time or the ability to do this, you can actually help out and use that machine. And like I said, I'm not definitely the most important person in this project. I'm just the one advocating it. Alfredo and Leandro are the two main developers. They actually did most of the parting in the beginning. And yeah, we're still looking for people to help out. There's a whole bunch of parts that need to be supported. There's a whole bunch of other things that do not work fully yet. For instance, Firefox now works on their Linux. Nobody's actually tested if that works on the FreeBSD. From my perspective, I'm more of a server person. So I'm more interested in that part being available. But from a complete ecosystem, we need to be able to support all of the parts as well as on other architectures. So I wanted to give a small demo if I can find the place. So this is a Power 8 machine. And if I run, so I have created an auto installer script which basically will start the installation. It starts of course on a serial console. It's a VM. I don't have a spare machine to do a complete install on bare metal and show you, but it's the same idea behind it. So in this case, I have an auto installer file that will boot up. So you see here the kernel is booting up. It will start the traditional FreeBSD installer. It will then boot up the, well actually now it's only booting up the kernel itself. So it uses the open firmware booter and then goes into the FreeBSD kernel. It will then start the installer and do the auto installer like on any other machine and system. The complete installation for this will take about one and a half minutes. Again, this is just a small little installation. I'm not doing anything complicated here. But if I now say, okay, start installation, you'll see that, yeah, the internet isn't that fast, but you'll see that it's basically doing an installation and it's only installing the FreeBSD base and FreeBSD kernel. That's about it. Like I said, I'm not doing too much in this presentation, but you get the idea. It's like on other architecture. So here it's using a USB installer image to actually do it so it doesn't need internet because otherwise you need to go and fetch it. It would take a little bit longer, but you can do that the same way. Typically any power machine can be installed through a USB installer. So the one that's available on the website of FreeBSD, but also some machines may not be able to do USB installers and you would need a Pixi boot. It depends which type of machine you have. Mostly the proprietary machines require you to Pixi boot while all the open machines will boot from a USB. So you see it takes about one minute and 21 seconds to install. Let me find out. So what I did here was just take out the auto installer because my script doesn't take that out automatically and then we'll just keep on auto installing all the time. But if I now boot up without the auto installer, it will actually basically start the open firmware. And then after doing this, it initializes all the, in this case, QMU, but it's the same if it would be a physical machine. So this is the part which the boot loader does and then you have the actual kernel from FreeBSD booting up at this point. This is running on a Linux machine with QMU because that's what is currently supported for virtualization and at this point I'm in the machine and you see it's a PowerPC machine. I can do PKG and install a whole bunch of applications. Like I said, there's a lot of ports that are already supported. There's also a whole bunch of stuff that is missing and we need to work on. Yeah, for the moment, the current machines, if you want to play around with it are not that cheap, which is why I will show you later how you can actually get access to machines without having to purchase them. But you will see if you want to spend, I mean, if you want to get a decent machine, you end up spending 5,000 euros minimum. Again, these are developer machines and not the entire machines. One of the things which we started about two years ago is looking at manufacturers that can build smaller type of machines with a target of 3,000 euros for an entry server and about 1,000 euros for a workstation. I'm not saying we are there yet, but that's what we are working towards. It's still going to be more pricey than an X86 or an ARM machine, but you will also get much more performance for that. If you are able to work with FPJs, you can use MicroWatt. MicroWatt with a decent FPJ, and I'm talking here about an RT-A7, so that's typically 250 euros type of hardware, so it's not like you need to buy one of the huge FPJs. They can run MicroWatt or Libre BMC. Sorry, Libre Sock. MicroWatt and Libre Sock are nearly the same type of car. Libre Sock has a few more extensions that it's working on, but this means that it's fully open source, you can program it. So the upper one is a DCSCM board, and this is the open BMC, Libre BMC implementation. The lower one is an RT-A700, which is one of the smaller FPJs it works on. The upper one is a bigger one, but those are the type of machines you can also run it. Again, you will not get the performance like on an actual machine, but if you're doing dev work, it's something you can look at and work with. Then the other way of getting access to machines is using the OpenPower Hub. So this is from the foundation, resources we make available. It's cloud-based access. You can get bare metal, you can get a virtual machine. They have a whole bunch of accelerators, GPUs, anything else that's available. At the moment, we have four providers, Oregon State University, which has the biggest setup, and has FPJs, GPUs, University of Oregon, which also has a whole bunch of them, and then Raptor has, obviously, their own machines, and my company also provides some services in there. So if you want temporary access, or even if you want long-term access, you can request it. So if you go through the form, traditionally we'll give you three weeks. If you add in the form that, okay, I'm going to continuously keep on updating this and work on this permanently, you will get permanent access. It may be that you don't get it in the one that you request, but one of the four will give you permanent access. Yeah, I put the email address there of the hub. If you have any problems, you can always mail me, and I can figure out where you can get access to. Then the next part is more about, yeah, powering back to the future, and with this, I presume everybody knows PowerPC, the Apple devices. We want to go back to, yeah, smaller formats, desktops, single-board computers. There has been an initiative for a laptop. I'm not saying that's something that's coming very soon, because that's going to be very complicated for now, but these are things that we're definitely thinking of. One other thing is the power isa today is fully open source, but when IBM donated it and made it fully open source, they also made it more structurally divisible. So previously, if you wanted to be power compliant, it meant that you had to implement 1099 instructions. That doesn't keep account with how many intrinsics and hash tables. That's 1099 instructions just to be power compliant. With the new subset in 3.1, you can actually go as low as SFFS, which is a viable working option for embedded devices with only about 214 instructions. We are actually working on reducing that to 200 instructions and looking if we can even create a smaller subset for microcontrollers or other type of devices so that you do not need to implement that many instructions. So for people trying to manufacture it will be easier, but also for us as maintainers, it's going to be much easier to divide this. So we have started work with GCC on this. We haven't moved to other ones yet, but we will move to LLVM and other ones over time. But we want to go through the same logic like x86 and ARM has, whereby you define instructions in groups and then you can just say that okay, this CPU is this combination of groups of instructions. Because now it's limited to IBM Power 9, which is good if you have one manufacturer, but if you want to go more independent of IBM and have different manufacturers with different features, we will move to that type of setup. So that's something that's in the play. That's why I added also the subsets here and the names. So you have PowerPC64, which typically is referred to as Big Indian, and then PowerPC64LE, which is the little Indian. Those are the ones that FreeBSD supports today. But we have been playing with PPC64 SFFS. For now only with Linux, because the amount of resources we have to play with and the people that are available to help out are all Linux users. But once we get that a little bit further, I would like to have BSD support also on that and make it available like we have today for FreeBSD and OpenBSD. So future hardware, this is ranked in order of expectation to come to market. So entry level, PowerISO 3.1 or Power10 based rack servers will come by the beginning of next year. Again, target price will be still around 3,000 euros. So it's not cheap, but it's not as expensive as before. And three level workstations will be coming. Again, target price will be about 1,000, 1,500. And then the single boards we hope to have by end of next year, most probably more 2025, but by the time they actually become available. And microcontroller systems and FPGA devices are today already available. So the FPGAs are the easiest way for the moment to get access, because you can buy them, you can download the images and just get working on that. Yeah, PowerPie was the initiative I mentioned last year. So I said I will mention it again here. So the ideas of having an open embedded, sorry, an open source embedded single board computer, it would be a dual core Power 3.1 based machine, most probably some kind of OMI acceleration, because this is the memory bus that IBM has developed and is today part of CXL, but it has a lot of advantages in how it connects to other memory buses. Given that we now have Libre BMC, we will most probably put some kind of small FPGA on the board and have it as a BMC, which you can then program and use. And the idea is to have a platform that people can actually use at home. I think everybody knows the Raspberry Pi and how popular that became and how much development work is done on that. People buy it, play with it. I'm definitely not saying that we will hit that prize of 50 euros, it might be 10 times more expensive but it will still be a magnitude less than what you today have. The idea is with the more people that we can give this to, the more people that buy this, the better the platform will become. So that's why we keep on advocating for this and are working towards building this. And yeah, my point which I tried to make last year and which I'm trying to make this year is that there are a lot of people involved in making this work on power, we need more people. So if you have time, if you have the ability, please get involved. I'll repeat the question. So one, I'm not a RISC-V expert. So let me be honest on that. So the problem that mostly comes and this is what I keep on telling most people is RISC-V says it's open source but isn't actually fully open source. That's one aspect. RISC-V has done a, or is doing a bottom top approach while power is doing a top bottom approach. So it has the enterprise level type of machines already. It doesn't have any of the lower sets. While RISC-V today you will find a lot of microcontrollers, a lot of embedded devices. You might even start seeing some desktop environments but most of the server machines that they're coming out with have failed. I've not heard of anyone that is really successful or any implementation anywhere in the world that is large. And the second part is while RISC-V is very much hyped because they say they are open source, they are not open source. If you find a mistake in the ISA, you cannot report it to RISC-V. We need to be a member of RISC-V to do that. If you want to build a chip on RISC-V, we need to be a member of RISC-V. In the power ecosystem you can actually build a power chip without being a member of the Open Power Foundation. You can use it, you can do anything with it. You can even get it certified that it would be open power ready. You don't need to be a member for that. So that's where the difference lies. I still believe that RISC-V in the smaller devices will be much better. And if you're talking accelerators and very specific applications, RISC-V is better. If you're talking more general computing and more of a mini to maxis machine, power will be a better solution. So the question is, this subsets, does it make more work and more complexity? Coming to the point more, yes and no. So the thing with having these subsets is that the subsets are fixed. So these are the four subsets we have today. These will not change. We might add one more and make a fifth one, but we're definitely not going to touch these in any way. So if a manufacturer comes and creates an SFFS compliant CPU and let's take a specific example, Libra makes this car and is actually working on that, and tomorrow company XYZ makes that same car with that same instruction set, your code should run seamlessly. There's nothing you need to do from that perspective. So that's the idea behind it. Having the different subsets is because not all people will want the same device. ARM does this also, Intel does this also. So that's where the idea has come from. So I don't think it will fragmentate the ecosystem that much. Yes, you will get these three or four. To be fairly honest, I only think the BELE SFFS will really work. All the other ones will be for embedded or very specific uses. Those will be the three general ones, and that's what we are aiming towards. And it just means that you have different subsets and different architecture codes for them. If you look from a perspective of, oh, I only want to maintain one, that's going to be difficult because then you need to take the lowest common denominator, in this case, SFFS, and then it would work even on LE because that same binary will work on an LE. It's not that that won't work. You might not get all of the acceleration and all of the features, but it must work on that. So a core that is LE subset compliant is automatically SFFS compliant. So in that way, I don't think it makes a fragmentation that big. But this is, of course, a risk in a way. So for OpenBSD, I think it's PowerPC-64. It's big Indian, if I remember well. So you basically can support all of them below it. The server ones will be the BE subset. So the highest one, the workstations will be the LE subset. We have no direct plans for SFFS that will most probably be next year. So late 2024, beginning 2025, I suspect. Not at the moment, no, unfortunately, no. That's something that we have discussed, but we haven't figured out how to do that. Because for instance, OpenBSD is big Indian. FreeBSD is little Indian and big Indian. Some of them provide only one or the other. So for us, it's very difficult to figure out how we do this benchmarking. If you have an idea, we can discuss that, and we are definitely thinking about how we can do that. We don't have an answer for that for now. So the question is Raptor and Power10. That they weren't going to develop a Power10 because the OMI connector, sorry, the OMI bus wasn't fully open source, which isn't true, by the way. It is fully open source. The problem is the training wasn't published. So the training program is published on how you train it, but the training data, the blob itself hasn't been published. Microchip, which is the co-author of OMI, doesn't want to release the binary blob, but has released how you build it so you can build the binary yourself. And they say, oh, this is open source and it's true. I mean, they don't technically need to publish the binary. Raptor has found a solution on that. That's all I can tell you for now. Last one then. It's a human problem. So the question is we don't have that many parts supported. It's a human resource problem. There are not enough people involved to keep it up to date, one, and then test it, two, and there are not enough people using it. So it's more a human problem, I think, for the moment than an actual technical problem. For most of these, the patches are available online or they are available for Linux. You can port them to BSD. For some of them, like Firefox, they will be exactly the same. So I think it's a very human problem for the moment where we don't have the resources because there are not enough people interested and we need to create that.