 So, as you can see, we ran out of qualified and competent people to do the speaker announcement, so you have to cope with me now. It is my pleasure to announce Michael Dexter speaking about things I don't understand. I'm going to learn something. I hope you're going to learn something too. Have fun. Thank you, Michael. Henning. How are you doing? So, hello. My name is Michael Dexter. The full official title of my talk is the free BSD build option OpenZFS. We have compatlinux and jail.conf nexus where all of these intersect and in general about how the whole is greater than the sum of its parts with these. But I think my key goal is to get you out of your food coma and conference coma from four wonderful days and to leave you with ideas and inspirations for the coming weeks, months and years. I will rant at times, but I hope to wrap it up nicely. So thank you for your patience. My very first conference talk was at EuroBSD Constrasberg 2008. I have colleagues to blame for nudging me into it, twisting my arm. And my themes have been virtualization, multiplicity, you name it. And in many of these talks, I've talked about building blocks, be it the monitoring that might go with such systems, be it storage, be it smart monitoring utilities and all these little bits that slowly come together towards a greater whole. I am 49, 95 years old and it's better late than never to kind of bring some of these parts together that I've been talking about for about 14 years. I trust you're all a bunch of BSD fanatics who will be spreading the word among your colleagues and friends over the next coming weeks, months and years. And I will turbo through some topics because they came up over and over from the keynote onward and in this room with many of the talks about jails and such. So I'm so glad to see that people are thinking like-mindedly. That is inspiring. And so as with any proper talk, I've been working on this until the very last minutes. I've shifted a few things accordingly to accommodate breaking news, including software developed today. So, I want to talk about five key points, the importance of standards and compliance with them, the power of explain it to me like I'm five, the power of owning the stack and virtualizing the rest, the Venn diagram from hell, this should be fun, and BSD used to the rescue as their wonderful new things across the different platforms and you will get some thanks out of this good man. So, I will put on a different hat that many of you have never seen over all these years of conferences. Putting on my SNEA storage networking industry association data protection and privacy committee hat, the committee is working on updating a best practices white paper. Yes, three, two, one backups are a good thing, but things are getting a little more complicated and formalized, which I'll touch on in a sec. The committee gets involved with working on standards and seeing how the sausage is made and that is a fascinating process and I'll touch on what it looks like and how you can get involved. And I do promote things like ZFS at the event, which some attendees have major issues with the rampant layering violations and open ZFS, but it's a very useful tool for a number of purposes that I'll touch on later, so we can sort of have nice things. Now, in the course of this participation, leafing through, I found this text, which is the software development guidelines for the organization as a public benefit organization where actually it's a vendor organization, but people leave their guns at the door and hopefully develop sane standards. In developing reference implementations, point six here, in general, commercially-friendly software licenses such as BSD are preferred as inbound and outbound licenses, and I'm like, I had that Jurassic Park moment, wait, this is a Unix system, I know Unix, this is BSD. You're speaking my language in the least likely environment. So I took that to heart and I do what I normally do is quickly give a talk on the subject, so I did a talk a few years back on by the book. Here are open source and key point production ready reference implementations of these technologies and terms. So the organization publishes its own, the LTFS format for tape format, NDMP, which hasn't quite gotten the love it deserves. Redfish, Swordfish, OpenIPI type management, it's getting some traction. But in the course of that, obviously, they're the RFCs that run the internet and run everything we do. There's NFS, there's ISCAS, there's Fibre Channel, and it's just a long list of just open standards. And I'll get into the role of those in BSD things. So has anyone been involved with standards development? No, no? Okay, go ahead, can't blame you. Oh, I see a hand in back. Which standards have you worked on? Architecture standards, excellent. So I work mostly with storage and on the theme I'll get to of owning the stack. I really want to own the stack such that I want high quality part, partitioning GPT, partitioning tools across the board on every OS. And they are stunningly bad once you start comparing them. And the matrix I get if not here, it's like it is scary bad. So in an act of desperation, I looked at the spec. It's like, what's going on there? Let's take a quick little peek at it. And it's a great computer science refresher. It's actually delightful. And there are words like reserve, which is your friends, which means do nothing. So in storage, most things are like a really long number, like a ruler. You start at zero and you go up to offsets. And in meters, a few meters in, you do something. A few meters in, from there you do something else. And the GPT spec is like, step one, hop over a bunch of stuff. A few shims, we say EFI part. So just blast in a few characters that are those characters literally lined up in various forms of Indian, this big Indian, little Indian binary. And I found that you can punch those in there with a combination of printf and dd and just start building up a partition table. Step one, validating a table. You start here, hop over, hop over some more. Reserved, hop over, you just increment a number from like 512 to plus 18 or to jump ahead. So I hope to share some tools that are just a simple little CS education on just stepping through this scary mystery of partitioning that vendors handle and third party software handles, but the spec is actually really straightforward. You just manage a bunch of offsets, do a checksum on a little block of it, and off you go. But in looking at the spec, you find things like typos. The official spec has simple typos in it and is ambiguous about Indianness and other stuff. So as many of you who just have an itch and jump into projects, well, there are opportunities that are remarkably low hanging fruit for standards that everyone in this room and this building in the city use. Everything is GPT partitioned at this point except for a few embedded devices. Maybe the little flash recorder on my audio recorder here. So this stuff matters and the barriers of entry aren't always that big. Naturally, some standards are goofy. I don't know if anyone in the room has any opinions on IPv6, but some standards need refresh and love and really careful thinking before they're implemented. So standards range from ad hoc to strict. Validation tools are rather important and sometimes they're plentiful. Sometimes they're absolutely absent. Hopefully there's a one-to-one relationship between standard and the tools to check it and that tool is accessible and affordable. And you just test the darn thing as you develop it. That's what I thought and continuous validation is important. As a community NVMe developer pointed out, the test suite for the NVMe spec is the price of a medium sized luxury car per year. Hopefully there are discounts for organizations and it's a key role for the foundations to play is hop in and say, hey look, we want to provide the best implementations. And for the record, for a while possibly to this day, the beehive NVMe emulation spec was the only available software implementation to pass all tests. Well, so someone's doing something, right? But if those tests are not accessible, we are limited in our progress. So, as for compliance, has anyone heard of the ISO 27040 spec? Security in storage, the refresh will arrive in October. And a great many industries will be subject to it. So it touches on things such as what to do in case of data breaches and organizational controls and people controls and physical controls and two factor authentication and removing software that shouldn't be there. Because, hey, it's a threat vector. So get ready for that, watch for it, you've been warned. It's probably a good thing and it's a shame so many industries have been flying blindly without guidance. And it's wild out there. And one shocker to me was that there are no universities that teach storage. Perhaps this one does, I sure hope if you look at most universities, they talk about storage ethics and privacy. No, how about SAS fabrics and record sizes and all that? So shifting gears to another point, there is a power of ELI-5, explain it to me like I'm five or 50 as the case may be. So I believe it originated in the US version of the office, no comment. And prior to that, Albert Einstein said if you can't explain it to a six-year-old, you don't understand it yourself. And there is a natural education nature to all things BSD, be it back to Berkeley, be it to this very event in this university. And there's a role for all of us in this education. So the tools themselves can literally have typos. They can literally be a bit complex or ambiguous. But it's the documentation that surrounds them that needs countless love. They need translations and literally anything you can do to help out with that both locally and globally is appreciated. Bless their hearts, my SQL had the 15 minute rule that you could set the thing up in about 15 minutes and show you that you're punching data in for your boss and he's like, we'll use this. Yes, it just go away, make it work. So discovery is always a challenge in free software, finding a feature. I mean, the classic in Microsoft terms was people requesting features that are already in the software, well, they failed at discovery. And in that is a lot of mentorship, find local and global mentorship like this very event to help communicate all these wonderful things we have to offer. It's a very non-technical function in many ways. Please don't confuse the need for a technical solution to non-technical problems, please, please, please, please, please. And so in the BSD circles, well, we have our OS and ports. Well, maybe there and docs. Maybe there's another layer to that that's needed, be it solutions, policies or something where the wisdom of the team can be brought into things like a bit. Like turnkey linux where they have sets of functionality and an appliance that is all standard off the shelf stuff, but it's just supplanted with a little wisdom and a little attention to detail. So shifting gears once again, owning the stack, I'll explain what that means. So if it's proprietary, you can't fix it. Full start, I realize that in the 90s, but how to actually achieve that is a challenge, unfortunately. The fewer repos you have, the fewer rat holes you have, in my opinion. Appliances are exciting, I've spent ten years with one, but enough time spent with an appliance allows you to cut right through it and actually find that the appliance is a barrier to getting a job done. And I like scripting. I find that if the operating system has all the hooks to do things with the logical syntax and things as simple as return values on everything. I've been advocating for a test that IF config can simply report if an interface exists, nothing more, nothing less. Not parse all the output and send it to Dev and I'll just yes or no. Is it there? And main tools get that right, fortunately. And things like I virtualized the rest pun intended. If your REST API is bigger than your OS, maybe just from a sheer bug count perspective alone, you're kind of off the rails. And so when it comes to owning a stack, our choices are the BSDs, Illumos, maybe some of the independent stuff. I guess Google has a new OS. I do not know if we'll all be proudly using it ten years from now. Couldn't tell you, but I know where we are here and now. So next step, the Venn diagram from hell. Yeah, I'll need it, you need it. So, my fancy graphics. If you squint and look at the phone in your pocket, my little travel router, a cloud server, a PBX used in the university, this contraption that runs all that, you name it, you squint, you see a Unix kernel. It's like, Unix one, it's like, this is great. We totally took over the world, except for a few little minor things. So, I don't know, NetApp with a proprietary file system or 1FS. Okay, they do their thing, little secret sauce, they do their thing. ASICs, especially network hardware. It's like, oh, it's running Linux, BSD, you name it, but then there's the actual chip that is a switching and then that's under NDA, ouch, ouch, ouch. Maybe a pretty gooey, maybe a command line route, Cisco-like interface, you name it, GPUs, wow, pain and suffering there, and of course support. Well, we won, but we didn't, we didn't, we didn't. So you've got all these collections of things we use every day with Unix, that we all help make. You in this room helped produce, but we're now like trapped by them. Okay, so in practice we have these venture capital funded appliances, like let's say, let's make a new router. They're virtually all GPL violating, rarely are they updated, it's inconsistent. Sometimes it's really consistent to a point that, well, you pay very clearly for the right to update your software. Support varies, I've had clients say wait, please don't take away the old model three par or something, the support was great. What's this new crap that you're ramming down our throats, et cetera, et cetera? Naturally they're expensive because the motivation is the money. And often, especially in storage, they're just setting up for either the IPO or the acquisition by, in the case of storage, by DDN or Dell EMC, and it's like wait, we not only won, the solutions are there, but we're not like benefiting, it's a little frustrating. So I don't think that Richard Stallman sat on his mountaintop and thought the future of software freedom is venture capital funded, unsupported, GPL violating, unupdated devices that will go out of date and switch off on you. It's like I don't think that's what they meant, I couldn't find that paragraph in the GNU manifesto or the open source definition or any of that. So is this what we asked for? And the cloud only compounds this. It's like, well, you want a black box, let's give you a one way black box. And computer science, yeah, going from like Pascal to C to Java to Python to JavaScript to Azure credentials. Like wait, hold on, that's not quite what we signed up for. And again, not a single school teaches storage on planet Earth. So you've probably heard this feel good comment from Theodorat. Linux people do what they do because they hate Microsoft. We do what we do because of Unix. We love Unix. Now, there's one level that would just feel good. Then there's a terrifying level, which is that what happens when a whole bunch of bitter Windows haters who aren't familiar with Linux start developing products that are venture capital funded and black boxes and all that. You get this whole momentum of like, no, no, no, stop, stop, stop, please. Don't drive the industry with that. Anyone worked with command.com on a Windows machine? It's like, it is a command line. And it's like someone described it over the phone. It's like, yeah, you can enter commands and you can do stuff in a file name. It's like, okay, I'll implement that. And then we get this mess we work with every day. So case in point, open GPU. So AMD probably opened up. Oh, great. We'll share our cards and our interfaces and all that. And you can do SRIOV. So you hand that graphics card to virtual machines and off you go and life is good. Well, they have a repo that hasn't been updated since like 2019. There's one person who has updated it and made it work with like eight commits ahead of upstream. Bless his heart. Whoever this cost bar is, thank you. But that shouldn't be necessary. When someone proudly announces an open GPU initiative and all that, this crap doesn't work. Maybe the pandemic played a role, but hey, we're pretty late in that game. And increasingly, if you want the future of this back with these cards, the cloud vendors get them. High-end servers have them for a fee for the license to go with the card. You just said open. Where did we go wrong? The other one is all these open networking routers like white box routers or ON, you can get the Dell, ON variation with all these Marvel reference implementations and such. Everyone ran Cumulus Linux. Now it's X Cumulus Linux and it's now on the Oni and others. And the open compute project, wow, a big unifying project came together and produced the software and you go to the released versions. And you go and it's like, wait, there aren't any. But yeah, how did we get here? Is it the old habits die hard? The capitalists just have to screw everything, even when they get the framework of the orgs and the open standards and the GitHub and whatever, correct? That's very frustrating. And now here's the just stab. The BSE appliances, be it PFSense, OpenSense, whatever, OpenBSD itself, whatever, they're better at GPL compliance than the GPL based products. They have no obligation, but they're better at developing in the open, sharing in the open and just like, okay, so what do we do about this? First off, I'll say hardware always plays a critical role. You have to like boot stuff. You need like switches with open ASICs you can interface with. The foundations, all of them need to be first in line before the vendor is interfacing with that new nifty GPU. They will probably have software faster than a vendor and the vendors are claiming they're open. Well, just keep at their door pounding on it. Please, I beg of you. And the foundations need to be there before the cloud vendors. There are some success stories, but we're still like hurting and losing. And question for you, what does a secure web-based management API look like for a Unix system? Because it's been reinvented a thousand times and they're all still kind of limited. Maybe we can come together and think about that as a group. I don't know. I, for one, love scriptability of a well-thought-out system because the hard part's done. You do the easy part and you make it work to meet your needs. And I still wonder about what a web GUI properly looks like that Theo would be happy with. So the ultimate question here, can we have nice things? BSD is hopefully to the rescue because I don't think we'd be in this room if we didn't believe that. So to their credit, and your credit, literally in the room here, credit, like decades of production quality open standard implementation. The BSD networking stack has, for the most part, run the internet from day one, and that's a good thing. And if you were to break that standard, you wouldn't participate on the internet. That's kind of required that you actually behave with regard to the rules. Decades of ties to academia as we sit here in a university, that's great. We've got to communicate what's available, what can be done. It's good for your career. It's good for your mind. Consistency is great. The BSD standout is just where, if a command works one way in one corner of the system, it probably works similarly another corner of it. The virtualization options are growing rapidly, be it VMM on OpenBSD, and VMM and Hacksam on NetBSD, Beehive on FreeBSD, and all that. And then, and the progress is coming along nicely, so let's shift gears one last time into very technical things. So PF, the human readable firewall, that is what success looks like. OpenSSH ships in Windows. You go to Options, and you can have like the Solitaire and OpenSSH and pick from a list. That's success. That is victory. Olivier with FreeBSDRP and FreeNAS, it's like, well, it does not pretend not to be the underlying OS. It simply adds some tools to make it streamlined and friendly. And all of these have their rough edges, and I know them well, but hey, PF sends an open sense. Well, they're doing a concerted effort to say, this is a preconfigured BSD with a GUI, nothing more, nothing less. Some, often with a hardware play, which you gotta boot stuff, that's cool. Just don't make it too proprietary, please. And so, one thing that has really, really stood out for me, and has kept me very busy for years now, is the build options in FreeBSD. Is anyone in the room familiar with build options in FreeBSD? Four hands? Five hands? Okay, good, about eight hands. Thank you very much. So, let's get our feet wet with them. So, if you go to the mansource.com manual page, you will see the list of the explained with and without switches, and you can turn stuff on, turn it off. And some OSes will say, you never want to do that because that should be the one thing all the time. I'm like, yes, yes, yes, yes, I get it. But, what really stood out to me is that, it's like, well, there's without jail. Well, let's forget about the underlying OS for a moment. It's a nested ABI. And without iSCSI? Well, no, no, no, no. We're talking actually without RFC 3720. It's an implementation of that RFC, and it's a pretty good one. And it can be turned on or off as you build the OS. Some of them are libc-level, and that's pretty intrusive. Some of them, and now combined with, say, reproducible builds, means that you can turn stuff off. The remaining things are identical, hopefully, to the shipped release OS, and it's still the OS. It's just less of it. I'll probably hit that point later. But to me, that's a toy chest of open standards. And you start with nothing and just add the ones you want, and things get kind of fun doing that. So, for years, I discovered them in like 2003, the moment I discovered jail, thinking, this is great. How do I cut everything down? Well, those options have been broken for decades. And I've been begging, encouraging, pleading that they get fixed. And with a bunch of bug reports, they were finally rather serviceable come recent releases. Now, in celebration of this event, someone went and broke three branches of them. So I'm working to find out tools to make that absolutely and institutionally CI tested so that doesn't happen again, because I'm finally able to do cool things with these. So there's a talk of package based on FreeBSD. They are a similar goal. They achieve different things. Hopefully that will happen also, because that allows for that similar modularity. I personally test them every time a snapshot arrives from FreeBSD. And when you remove everything in the OS and you don't build Clang and LLVM, you build quite quickly. Like on this laptop I'm presenting from, like under five minutes. The whole OS in a bootable sense, a state. And some of you may have heard of my little OccamBSD project. I've completely refactored it the last few days, and it's available on GitHub, which I'll have a link to later. But just for those who aren't aware, there are 234 options as of 13.1. There was from Paul Henning Kemp, the build option survey, which would step through each option and build the OS with or without that component. That would take a week or two. And I am not worried about without VI breaking. It's a VI text editor. You can build without it, and I'm not losing sleep over that. So that is not a great approach. It's out there if you want to just poke at it, and then you slam the electricity into that. So I've been working with a light option, which enables one, two, three required options required to build the rest without them, and you get to exercise all the make options and find out what someone broke this week. So late last night at the event, a social event I cornered Lee when the cluster testing was free BSD, and hopefully we'll have some official tests in really soon. And hopefully people will start testing all of these before commit. So I hope I'm not the only one who cares about these. And of course you can interrupt me with any questions anytime. But I will tie this up, rest assured. That's where you can find it. It lets you pair down free BSD to without compression, ZFS rooted 240 megabytes with an actual log-inable system. And one fun thing there is that I thought the result would be unrecognizable once you got the thing. It's like, I don't know, BSD44. It's like, wait, in college I sat down to this, but it's that plus ZFS, which is kind of cool, thank you very much. So if you've ever tried, I don't know if anyone's tried Occam BSD in here, but now I let you keep the binaries because you're probably sending to different places and now send to an external hard drive if you want. Just, hey, I'm trying to make it more user-friendly. But five minutes to get through a build world, which means that for any kind of regression testing, you name it, especially regression testing and bisection. Five minutes rocks compared to like 40 minutes. So I've now done a configuration file for lack of a better term where I simply out-specify the build options, the kernel modules, the kernel options in BSD kernel configuration terms, and the devices where, yep, you need like a PCI device to boot, you need a few things to literally boot the system, and then it's whatever you want on top of that. By default, there's no networking. Well, if you want to say be a student and learn an OS, I suggest you start with the very smallest core of the OS, the stuff that absolutely has to be there as opposed to the gigs upon gigs of other stuff. And again, the complaint that, oh, if you change anything, it's not free BSD. Well, with reproducible builds coming along, that binary might be identical to the standard release, just it happens to be only that binary, not a whole bunch of other stuff. So in support of such a thing, OpenZFS, love it or hate it, a point I've been emphasizing in the SNEA circles is that read-only and unmounting is free of charge. Remember the default activity of like you unmount the entire file system, the entire OS, and remount it, read-write if you happen to be booted, single user mode and read-only mode. You can have fine-grained like selective free of charge, unmounting, unmounting, read-only in the era of ransomware. I mean, I want my system to react to a hostile threat and go read-only in an instant, and if it's really hostile, just vanish, unmounted, and let that storm pass. Are we doing that? I have not seen that implemented, but that's one of those tools in this tool chest that get really exciting for guys like me. Native encryption. We can argue separately about layers of things on file system, but that's exciting. It has a few bugs that they're working on. It's institutional. And when it comes to compliance with things like 27.040 and all these regular story compliances, well, encryption is one of those big old check boxes. So having it absolutely fully institutionalized is your friend. And one thing I absolutely love in this thing is delegated permissions. Since the 90s, when I first spun up my own web server on the internet and self-hosted, since then, how do you back it up? You don't want root access in any way remotely, but having snapshots of the file system travel out the door as an unprivileged user, even though it's root-level content, is great in my book. So I personally think OpenZFS is a great regulatory compliance tool. Love it or hate it. And compared to, say, proprietary options, I'd rather be in these circles. I have a few talks out there that aren't too hard to find if you're curious about such things. Moving on, there are hypervisors coming out of the walls right now on the BSDs. And there have been a number talked about this week. Again, own the stack from that disk partitioning on up and virtualize the rest. In Beehive Circles, my number one request, hire VCPU accounts so that you can truly have isolated build environments, throw all your cores in a VM, run a future version of the OS, and build it efficiently. John Baldwin's doing great on that, bless his hark. You can go check out my talk on isolated build environments. Again, the building blocks to get to where we should be to not be in this vicious circle of hell of these proprietary appliances. In reducing the host OS, you can approach a type 1 hypervisor insofar as the OS is tiny and things are handled by Viet containers and virtual machines. That's been a narrative since Popek and Goldberg explained it in 64. So that's desirable. You can shift resources to a VM. And one neat trick is, yeah, one neat trick is dev control. Anyone use it on free BSD? You can live strip off PCI devices. And quite a few of them. To a point in my lab, SSHD, and I'm like, I'm going to nuke my VGA, nuke the network, though I could have a serial connection, nuke all of USB, nuke all these components, and then suddenly my PCI list on the hardware looks a lot like a VM PCI list. Audio, get rid of it. And then you can distribute these like candy to virtual machines. That is flexible. That is cool. The computers have done that nifty ways behind the scenes, but you can have it today free of charge. I believe Linux can do that with a special patch. That's nice. They're catching up. That's cool. But, yeah, you can strip off devices. Just say, okay, I'm going to turn off this whole subsystem, and it doesn't panic very often. Another neat feature bringing all these components together from different directions. Beehive has a config file. Finally, the number one requested feature from day one on GPU pass-through for the gamers was the config file. And a neat trick that Jan Brankoff pointed out is that you can slip your own parameters in there that it just ignores. So if you want to add additional metadata for whatever purpose or management into the configuration, just do it. And hopefully this will be the, that's intended to be the platform for all the things you want. Be it JSON, YAML, other higher level stuff. Well, we had to start low level. That's what they did, as opposed to one high level format that no one could agree on. So, that's exciting. As came up in a talk earlier, the Linux compatibility in FreeBSD is becoming quite usable. Yes, it has its constraints to this OS I once heard about, CentOS. Have anyone heard of that, Grandpa? That was then, this is now, that whole community has to figure out what their standards are, because even when they have standards, it's like hard to find one. Did OpenBSD rip out Linux Compat and other compats? Yes, we did. We found out that our Linux Compat is compatible to Linux. ABI that Linux isn't compatible to anyone. That, my friend, is the problem with moving targets and that OpenBSD found that their Linux Compat so old that Linux didn't support that subset. Which is, they're all moving targets. Every single thing I've touched on today is a moving target. So, embrace that and focus on tools that accommodate that and make your life easier. And especially tools with muscle memory that last decades and just in the last session there's complaint. Well, why is it Linux is going from IF config to IP? It's like, well, just to find out your IP address is not like a controversial at question. And similarly, Wine and Proton, Wine for Gamers is coming along such that there are not super comprehensive, but usable for some circumstances, options for Windows and Linux compatibility. Now, that puts the onus on us to document them. What works? What doesn't? With no emotion, just here's a test suite that verifies if certain applications run or don't run. And years and years in the making for those using jails, jail.conf.d. You drop a configuration file in a common directory. If it's in there, it gets booted. If it's not, it doesn't. There's a great talk from Mark Atwood about the greatest no SQL database is a file system. It's reliable, it's fast, it's flexible, it's object oriented. There's a whole bunch of things. And there are times that dropping a text file into a directory with heavy lifting meaning is a good thing. I personally prefer that to a massive administration tool built in some funny language. And I just learned this week there are a few bugs to address in it, but I'm talking to them about what if we had nested data sets and you could, using ZFS, make a data set disappear to make a jail disappear. You are using 100% in-base tools to achieve what is often solved with massive lifestyle solutions. I don't like lifestyle solutions. The BSD OS is pick your BSD. In this case, free BSD is the lifestyle, not the stuff you bolt on top of it. I am impressed with Bastille BSD, but there's a comment in the last session, or second to last in this very room, that the virtual machines are driving a bit too much policy up the stack, and that's a good observation. So these conversations have to take place to figure out what is the right and wrong or better and worse solution. I do have a tool that's in its infancy called jail hive, which is a jail with the bare minimum resources to contain a beehive virtual machine. That's what Illumos is doing institutionally. Bless their hearts. In free BSD circles, if you are to use beehive load, which simply calls some Lewis script, you could have a very hostile VM and not know it, and you run it as root and you've got a problem. So I hope VMM on all the platforms are taking precautions there. So a general theme here is the near base tools. The base OS's are great. I mean, that's what it takes a while to appreciate, but you sit down at a BSD and things are consistent. Documentation is consistent, behavior is consistent. So I for one am not sold on the notion of the easy button, the one button solution because the more time you spend with those, the more you just have to drill through to the underlying OS and get the crutch out of the way. Pounded that point repeatedly. The human readability of these mechanical things is critical, and again, that's what PF gets right. It's like, well, where's the web GUI? It's human readable. Where's the web? Just try it. Just sit down. It's like you can get started in a few lines. There are great books out there. Lean into that. I mean, we really have to lean into that on all these components and at the academic level. It's in schools, in your community. You name it. One thing that keeps coming up in this very room this very day is like, well, proper process supervision. If any given BSD needs a proper process supervisor, let's have that discussion. If all of them are terrible in some way, this couldn't... Yes, sir? Five minutes. Five minutes? Oh, I wasn't... Oh, okay. You don't have a process supervisor of choice to propose for inclusion in your base OS. He is a process supervisor. Yes, he is. And I love him for it. So let's have that discussion. So bringing this home in an attempt to inspire your actions coming weeks and months because you will go to somewhere other than Vienna statistically. Please talk to your local communities and figure out what resources are there, how you can help, what schools you can give a talk at, you name it. So to recap those initial points I wanted to cover and instill in you standards and their compliance provide us guidance because if we don't at least start there, we will just end up with anarchy. And when vendors do it, it's painful anarchy and expensive anarchy. The power of explaining it to me like I'm five or 50. This is on all of us and thank you for organizing an event that brings people together. Thank you for whatever you're working on because that's what it looks like. It's a messy, wet-wear process, humans. It's not a technology, it's not a new, you name it, technology. I think you're all here for some visceral understanding of owning a stack. You can fix the OS and contribute at some point. Whatever crazy point you want from the starting at zero GPT partitioning, you jump 512 bytes and 512 more, check it out, it's great, it's awesome. And these features are getting really good. There's a notion of actual genuine Kaizen revision in the BSDs where you just progressively increment and make a little better whereas they are still trying to unify GNU and Linux and someday they will. I see a single task there but others see a thousand tasks there. Bless their hearts. That's fascinating. And so with all of this and everything you do, let's get rid of that Venn diagram from hell. So, thank you so much. I'm not sure that was what you were expecting but I hope it leaves you inspired for the coming weeks, months and years. Any questions? Ah, yes. And Paul, we miss you. Maybe rest in peace. I'm easy to find. I guess in our five minutes I can't show you a 240 meg version of FreeBSD booting with ZFS but that will have to wait. Thank you all. Thanks to Michael for the 1000 presentations.