 All right, shall we begin? Hello, I'm Vagran Kaskedian. I would like to present to you my latest infatuation, my crush on Gnu Geeks. I'll start off with it's kind of awkward calling it Gnu Geeks, especially when you're talking about crushes. But for the longest time, pronounced it Gwix. It's a bit awkward in conversations talking to people. They don't know what you mean. Are you the intersection of people interested in Gnu Geeks who are geeks interested in Gnu is pretty high. But anyway, despite awkward naming conventions, I'm pretty interested in this new system. It's got its quirks. It's a bit of a curiosity. And I think there's also a really interesting, a really solid community alignment with a lot of the values that Debian shares. So a bit about myself. I'll just reduce it down to some basic numbers. I've been a Debian user since probably around 2001. I got deeply involved in Debian in Debian Comp4 back in 2004. I became a Debian maintainer around 2008 and a developer in 2010. And my stats with Gnu Geeks are a bit fresher, which is a bit more exciting. I started submitting bugs and kind of hanging out on the channel kind of late last year and started getting some patches into the Geeks Git repository just earlier this year. So I'm kind of doing this as a compare and contrast of Debian. So I just figure I will briefly touch on at least some small piece of Debian, most notably that Debian is a volunteer effort and they want to focus on a free, high-quality Unix operating system. And this I just cut from the Debian fact. Of course, Debian is much more than just these few simple lines. But interestingly enough, Gnu Geeks, the Gnu System distribution, and the Gnu Geeks package manager are free software projects developed by volunteers. So there's a pretty strong alignment there. The free software values, the volunteer driven, community driven focus, that's pretty similar. And I think, bears in mind, some value commonality there. What, where it kind of differs a little bit more is Gnu Geeks is a re-implementation of the Nix build system, and the re-implementation is done in Gnu Gile. It also operates as a standalone package manager, interestingly enough, on top of other distributions. Maybe you know of some. And it also has a full operating system designed entirely around the Geeks build system. I pretty much got interested in Gnu Geeks when I caught that twinkle in the eye across the room when I was involved with some reproducible builds. And I started meeting some of the Gnu Geeks folks. And with reproducible builds, I'm pretty excited about reproducible builds. And a lot of things that we're doing with reproducible builds in Debian is kind of trying to shoehorn some new practices into a well-established system that didn't really have that built in mind. But Gnu Geeks has a lot of the premises of reproducible builds built into it directly. It's by intention not something we're trying to bolt on later or reintegrate. So it has a number of things where it normalizes the build environment. It even builds in containers. It has standardization of the build path. And it has built in mechanisms for actually verifying the reproducibility. In Debian, a lot of these processes are pretty hard to actually integrate. We're doing the best we can. But it's going to take some time. Whereas this, it has much more potential to be reproducible right out of the gate. So one of the bizarre things where it really differs is in the file system layout. The file system hierarchy standard is, for the most part, thrown out the window. So all of the software on the system goes into subdirectories under the Gnu Store directory. And then it will have a hash of the tool chain used to build that particular version of that package and a package and a version. There are other things also in the store that are not specifically packages, such as upstream tarballs, but they all basically follow the same basic premise. So since it doesn't follow the file system hierarchy standard, things like your path are going to be different. So a lot of what it accomplishes in order to get you to use the binaries from Gnu Store, blah, blah, blah, is it sets the path, sets up in a specific order so that you typically will get the binary that your particular configuration is set up to run. So it uses a lot of sim links to accomplish this. So first, you'll have your GUIX profile, which is a sim link to var GUIX profiles, which is a sim link to another profile within that directory, which is a sim link to another actual file, I think. And then in there is a sim link to the file that you're actually running, say, you want to run Emacs, for example. So it's a whole big mess of sim links, which has some really interesting properties, but we'll get into some of that more later. So where exactly does a binary live? Typically with bash, you might want to run bash from the system. So it'll be in the system profile path. And then you'll find the actual bash is located in the Gnu Store, hash of toolchain, bash, the version, and bin bash. So what will be interesting here is when you list all of the libraries that bash uses, they all point directly to other files in the store. This has some really interesting properties when we get into the transactional upgrades and transactional rollbacks. So basically, because essentially all that a package is is a built thing shoved into a directory, really the only way you can use it is through setting up your sim link tree to actually point to that one preferred over some older version. So this way, when you upgrade a set of packages, first it will build them. And then once the build completes and is successful, then it will update your sim links. And so the act of updating a sim link can be an almost atomic operation. So the likelihood that you are ever in an inconsistent state is really quite low. It's just a bunch of file system operations. And you can, in a desperate situation, actually manually re-implement those to get yourself back into a consistent state. I've only once had to do that in the last year, but it has a mechanism where you can actually switch back to any generation currently available in your profile. Users can install packages without requiring a system administrator permission, which is a double-edged sword. You can, every single user has an entirely independent copy of their profile. So the users upgrade their software when they want to upgrade. They're not waiting on the system administrator. And I'll talk more about the implications of that later. It's also really fairly trivial for an individual user to install a customized version of a package. So this was, this is kind of a cut and paste. The formatting didn't quite turn out great, and a lot of it's scrolled off the screen. But basically, if you're familiar with a scheme, all of the packaging is basically done in a scheme dialogue called Guile. And essentially, you can run a custom version of Geeks by using the pre-installation environment, and then just asking the build daemon to build this custom package for you. GUIC's environment is the ability to, it's kind of like if you were to call, say, S Cherute, and then it will, or S build, and then it'll drop you into a shell with all the dependencies necessary for actually building that package, tweaking that package, customizing it. And so this can be really useful if you're just kind of developing a new upstream version or something. You just get the build tool chain, and one really cool thing about it is because it's just linking a bunch of sim, it's creating a bunch of sim links to files that if they're already built, it's a trivial, quick operation. It has some importers to pull package definitions from other sites such as PyPy, or I think there are even some Haskell ones, and basically it will dump a definition much like the one earlier, so it'll spit out the scheme that will give you a baseline to start hacking on a new package. And kind of back to the reproducible builds issue here. So by design, it's intended to be able to challenge the builds. So basically, if you do a local build of bash, you can compare against the servers on which you might download that version from, and you can even compare against multiple servers. So by design, it already has a fairly simple system for checking and seeing that you're getting the same results as produced by the upstream servers. And here's an example checking bash from my local system, and they were identical. Here's an example with the Linux Libre kernel, and for whatever reason, it's currently not reproducible, so you can see that they differed. And it gets a little more verbose, because it's a more alarming situation. Upgrading a package in Canoe Geeks is often as simple as updating two lines. So I've been the Uboot maintainer in Debian for many years, and so naturally, some of the first patches I submitted to Canoe Geeks were to update the Uboot versions. And so here, you're just updating the version number, and this is the expected hash of the upstream tarball that you use for the new version. And literally, for that version bump, that was the only thing I needed to change, which I think at the time I actually used Canoe Geeks to test Uboot before I tested it on Debian because it was just so simple to just do a few tests, install Uboot on a few machines, rather than going through all of the steps to update the Debian packaging, update the patch sets, those kind of things. The Canoe system distribution is the full operating system level. So it includes an entirely new init system, which I know people love to talk about init systems. That init system is, of course, written in Canoe Geil. And this init system is just a few lines of Geil, which of course includes much larger libraries, but this is a particular exact init for this particular instantiation of this operating system. So every time you get a new init system for every time you update your system, obviously the differences vary, but so to configure a system, you pass it a configuration defined in scheme, and then rather than editing a bunch of configuration files in Etsy, you actually generate your configuration files. So it's kind of like configuration management by design. So the running SSHD, for example, here is actually passed the configuration file in the store. You can fairly trivially instantiate virtual machines. You can build a virtual machine that's basically identical to your running system configuration, and it can pass the contents of the store or possibly a limited set of contents of the store to only include the things that match the same system. So this can be done quite quickly. In bug reporting, this will be familiar to a lot of people in Debian. It uses Debugs. What's maybe a little unfamiliar is there are only two packages with which to submit bugs. Maybe there are more than that. Somebody might be able to correct me on that, but it uses Debugs. So you can use your favorite, just email it, describe your problem. You can get a list of bugs at the URLs. And here's where we really get into some interesting things. So you can run GUIX SD as a full-blown operating system, but you can also run GNU Geeks as your own as an installation on top of another operating system such as Debian. You can download binary tar balls from the upstream GNU Geeks site. You can install from source, and there's been a little bit of work gone into actually packaging GNU Geeks for Debian. So one of my biggest frustrations with GNU Geeks is there's a long outstanding bug where there's not really a built-in chain of trust integrated to the system. So the entirety of GNU Geeks is defined as a single get repository, and each individual commit may be signed. In fact, I think they're required to be signed, but unfortunately there's no built-in system for the end user to go through and verify those signatures. So there's no trust key ring. There's nothing like that. There's no built-in system. You can go through and manually pick, oh, I see this commit is signed by somebody who's got a strong enough connection in my web of trust, but you can't, there's no built-in system where, oh, this is the expected set of keys that should sign things and will automatically only build versions that have been properly signed. Another really interesting effect is because every single user on a system can have their own sets of packages installed. There's a system-wide profile. Even Root has its own set of packages installed separate from the system profile, or in addition to, rather. Security update consistency across the whole machine is a little hard to audit. It has some strengths. You don't force any users to upgrade at a time that's inconvenient to them, but obviously if there are lingering security updates out there, that's a bit of a concern. So those are some of the big issues that have turned me away from using GNU Geeks in any serious capacity. I just have a stronger concern about the level of security there. The updates, basically Geeks pull is essentially roughly equivalent to App Get Update, but it actually recompiles all of the available package definitions, and this takes some time. Even on a fairly modern computer, it can take a few minutes sometimes depending on how many package updates there were, and that doesn't even include installing the packages. So the fact that when you try to update the system, it's going to take a long time, can really hinder people from doing those security updates that they keep putting off. So that's a big concern for me as well, and it's also just practically kind of annoying to sit there watching a little spinner going on. So those are some of the big concerns I have with it as a system. I know there's some work to improve the speed on that, and I've even seen some of those improvements in my brief involvement. But, and then another thing that can sometimes drive you crazy is a minor update in the tool chain can have a cascading effect on which packages get rebuilt. So it's hard to know when you run an update, like just how many things are actually going to be updated. You know, I want a new version of X, but along with that comes like an entire tool chain dependency on countless other packages. So sometimes updates can be, even for what seems like a relatively minor update, it can be an entire, almost an entire rebuild of your system. They have some features that kind of can work around some of those limitations that are reasonably safe to use, but in my experience every once in a while, you'll just get a huge update. And additionally, it's both a binary and a source-based distribution. So individual users, individual users could choose to rebuild everything from source and to some degree that's encouraged so you can compare against the official servers. And that gives greater assurances with regards to reproducibility. But there are also binary servers that are basically rebuilding all of the packages, but get pushes can go a lot faster than the binary servers may be up for rebuilding. And so at any given time, you might do an update and you can download a lot of the binaries or you end up rebuilding the world. And that can be a little awkward and challenging at times. Another thing is I don't have much background in scheme or lists or anything like that. And the parenthesis for updating package definitions, it kind of always has this feel like it looks very elegant and straightforward and then I actually try to do it and you miss a parent somewhere and you get a very explosive trace back trying to figure out how it goes. Your mileage may vary on that but sometimes I find it a little overwhelming. Just you define something and boom and you don't know where it was and maybe you didn't fastidiously commit to get while you were hacking on it. That's basically, that's the gist of things. I really have, I've been using Debian for ages. I love it. I do sometimes wonder if some of the things that GNU Geeks are doing might better handle things like PyPy and RubyGems and things like that where the entire ecosystem is just moving a lot faster than Debian can keep track. So that's one of the things I've been interested in this GNU Geeks is it's much easier to do updates to individual packages without necessarily having to do an entire transition. Those kind of things, you can do individual packages to just test updates without needing to build the entire ecosystem around that. You can just build it for your little test. And yeah, so a huge thanks to both Debian which has really built me into a free software developer on some level and GNU Geeks for being a small welcoming community. That's it for my slides but I have a number of demos I could get into and why don't we take a few questions and then maybe I can respond with some demos or something like that. Speaking of PyPy and RubyGems, is there any prospects for some auto translation of the language specific package manager recipe into the Geeks format? Yeah, there's an importer which can basically, in my experience, I've done a few PyPy packages with it and with some of them it just worked out of the box. You say Geeks import PyPy and then the package name and then it'll output the package definition in the Geil format and some of those have worked and some of them needed some additional tweaking, pulling in a few more PyPy modules, things like that. So there's work there. I don't know how fully automated it is. There's always probably gonna be some degree of manual checking and it's code so it can be improved. What's that? It's Sol's package. Ah, that gets more interesting. But yeah. Oh, also for Debian source packages? Yeah, I racked my brains a bit at how I could more closely bring some of the features of GNU Geeks into Debian and it's a bit hard. But yeah, it's a very different paradigm. Not to say it's impossible but I think the Debian project might move a bit slow to make that a reality. Yeah. So you mentioned updates sometimes needing to rebuild a lot of stuff. That presumably applies to security updates as well. So if you get a Lib C security update, you then have to rebuild absolutely everything. To on near, very close. Yeah, at times. Right, and my understanding of this, the way that the system is set up with this whole functional model, that's kind of baked in. Do they have a bodge for that? They do. I believe it's a system called Graphs and it kind of fudges the like, well, so if you remember every package, you might have 10 different versions of Bash 4.0. And each version is basically the tool chain used to build it. I believe, and I don't understand in great detail, but I believe the Graph system will take, whatever the most recent Bash is, and use that or Lib C is probably a better example, will take whatever the matching, as long as the version matches, we'll kind of pretend that that is actually built with this other tool chain and do some tricks like that. So that helps a bit with that. That's actually, that kind of reminds me also of, I neglected to put it in my slides, but one of the biggest frustrations I have with GNU Geeks as somebody who kind of came into the free software world as a system administrator is they have no concept of stable updates. It's all, it's basically all a rolling release at this point. And so, yeah, security updates are just normal package updates essentially. Does that answer your question? Hi. Okay. Any other questions or I can just start to dive into some examples. Yeah, I had a quick question. You mentioned that each user, including Root, has its own environment and its own set of more or less home-built packages that are not system-wide. Is there any particular reason why when a user wants to have a package built, it cannot be part in a GNU store to have the application with also users? Yes, so when another user, so you have user A, they install a package and then user B installs the same package, all they have to do is make the SimLinks. So the second user or system or whatever to install it, they share the same store. Okay. But then it's just a trivial SimLink operation. So that also gets into things like how about Geeks environment. So here, basically, I've probably already run it. So it was a trivial operation. It built a charoute that essentially has only the stuff necessary to build the hello package. So this, because it didn't really show you much, why don't I try with another one? So here, it's basically building a charoute kind of like what P-Builder or S-Build would do, except because many of the dependencies might already be available because I've installed other packages which have this in the store. It doesn't actually have to go through, unpack the packages, do all of those operations. They're basically already on the system and so it can just do a nice quick SimLink tree to instantiate that. Yeah. Sometimes you need to rebuild the packages and all the dependencies and that you can have several versions of the same library on a system. So is it working on old machines? So if you have an old hardware, is it going to be working? For example, if you have a small HDD or not very efficient processor? Right. So naturally one of the things I started doing with Geeks is to, since I work on ARM systems a lot, I basically added support for a number of ARM boards and one of them was a single core ARM processor with half a gig of RAM and it sometimes took a day to do the most basic operations but there are some really nice features. Let's see. So there are some really cool features where I would build the entire system on a faster machine and then you can actually publish all of your build results. Everything in the store is basically something that should be publicly available and so you can have kind of peer to peer setup where each, you know, this machine, trust that machine to build packages and it can just download them from a faster machine that's already built them. So that's one of the really nice features. But yeah, but even then there are some things that still need to be run locally just as part of the process or if they don't build reproducibly enough or something, then it's like, no, no, I gotta rebuild this myself, I'll see you in a few days. So yes, on slow machines it can be a bit challenging. There, I believe there's a way where you can rebuild everything from source and not trust any binaries but I don't know about the, I haven't found any way to do the converse where it's like, if a binary is not available don't bother installing it. So, yeah. For these build machines, I can imagine that the C flags or the LD flags will affect the binary. So is that taking into account into the reproducibility? Yes, so since it's functional package definition, let's see if I've got this right. Because it's functional, so here basically the input is the entire scheme information that produces that package and so you'll see it'll pass cross compile arguments and all of the inputs that go into the file are there. I wonder, do we have C flags specified anywhere? So one really exciting thing is the entire repository is just one git repo. I mean, so, and realistically we've got a distribution here that has maybe a third of the packages in Debian, give or take a little and those packages are represented in, you know, an 89 megabyte git repository. So it has references to the tar balls and hashes of what it expects to find or references to other git repositories and it has a hash to make sure you get the correct thing. But so you can, to look for things like that, you can do things like git grep. Oh, okay, in various packages, so we could look in GNU packages Android and we can see it sets the C flags for Android lib cuutils. Never seen this package before but so basically if you change anything in the package definition, it's going to result in a new definition of the package. So any kind of C flags type changes are going to propagate into any of the things that use that as an input. Otherwise in Debian terms, kind of like a build dependency, it'll cascade down the whole system and everything that used that will be rebuilt. Okay, oh-ho. Can I interrupt briefly and say that seems to imply that I'm forbidden now from just setting C flags in the environment because that will totally mess everything up? Yes, forbidden, well it'll just be ignored. So you can set it all you want. But it's fairly simple to edit individual packages but I guess if you're wanting to set a global change or something like that, that's more what you were getting at. Yeah, yeah you can't, you actually need to specify, because it's functional, you need to specify all of your inputs in order to produce the output. So all of the inputs are defined in the given package. So here, here I've got the arm trusted firmware package I worked on briefly to get an upstream version of it. Currently packaged in GNU Geeks, they have arm, they have arm trusted firmware. All of the targets are actually built with, they're all built with custom vendor repositories. This is kind of a thing in the arm world where just about everything is built with a custom vendor repository. But finally, they added some support in upstream arm trusted firmware. So I started working on a packaging definition for that. And you'll notice there's a function kind of right at the top there, make arm trusted firmware. So you can use functions to define packages that they then inherit all the baseline for it and you only specify how they differ from the main package. So if you did want one or two packages that shared kind of a common build system or something, you could define that in one of these function definitions and then you could create a local version of that package. So here I'll try this one. So here I'm running a build. So when you use the pre-inst environment, that's kind of a hack to work around updating all of the package definitions. It kind of just, I'm not even sure exactly how it works, but basically you build a particular implementation of GNU Geeks and then it will kind of reuse as much as it can and spit out a bunch of warnings. But here I've just built a new version of arm trusted firmware, which is now located in GNU Store, blah, blah, blah, blah. Another question. Yes, so you just mentioned that we can basically, oh, people can generally more or less write arbitrary will code as for package definitions. Do you have some particularly cool users for that and do you also have some cases where it just makes package definition hard to analyze either by hand or automatically? On the latter case, I have a harder time defining that, but I think, I don't know if you've ever looked at the UFoot packages in Debian, but the UBoot, like for example, here I've got UBoot one board and it's a two line change and now I support an entirely new board in the Debian packaging that usually takes editing four or five files and making sure you get it right. Someone that's maybe my own fault and it could be done more elegantly, but things like this I find to be tremendously useful. Any time you have a project in which you might have multiple versions in the archive, say a console only version and an X version and a Wayland version or something like that, this kind of approach really makes it easy to define multiple different versions. You can also say the Wayland version isn't ready yet, but you wanna add support for this package. You can start by submitting a version that only does the console utilities first and then it's already there and maybe somebody else comes along and adds another variant and it's pretty trivial to add. And there was a second part to your question. Yeah, do you have cases where basically you would want to, for instance, automatically grab for whatever package information, like find all packages that have this or that feature and just it's hard because now you have to more or less execute arbitrary code to understand it. Yeah. Oh, Quicks already runs the code, so. Great. It's got, I mean, you can search for packages, kind of like apcache search. Oh boy. Let's see how that does. Right, so it puts out a boatload of stuff sorted by relevance. Probably looks fairly similar to your typical Debian packaging format. So you can kind of search, there are package definitions. I do find get grep to be extremely useful for searching for common things or searching for a package that's doing something that I might need to cargo cult into another package definition. But just the fact that the entire operating system is in one get repository makes some of that analysis possible, I think, much easier than with Debian, where all of the packages are scattered between countless tar balls and archives and sources.debian.net simplifies it somewhat, but obviously using common utilities with get on a 89 megabyte repository is going to be a lot faster. And I think the answer to, or to put it, my specific question was, I guess, Quicks already turns like that arbitrary code into package definition we can search, it seems. Yeah, let's see if I can find one for you, oh boy. So also, it drives me crazy that Garugik's puts the hash. I really wish it had been package version hash. Makes tab completion a lot easier, but that's not the way it is and to change that would be a lot of work or a lot of rebuilding everything. So it would require reboot strapping the whole thing. But anyway, I was trying to, ah, yes, the GNU Store is essentially read-only. There's a daemon running. Yeah, I'll try and explain that. That's a, so somebody mentioned they don't want you to touch GNU Store. So there's essentially a daemon running. So one thing about all of your binaries being in this one directory with GNU Store, big long hash, package name. Processes that are running tend to have very long names and are a bit hard to sort out. But basically, here's an example. This is the Geeks daemon running. Currently, I believe it's essentially a fork of, it's a fork of the Nix daemon. There's work in progress to rewrite it in scheme. But for now, essentially this is running where it's, when you do a package installation, it's basically you call out to the daemon and you say, hey, I'd like this package. And then the daemon will go and it will check all of the essentially inputs for that package. And in this case, it will look for substitute servers on berlin.guix, berlin.guixxd.org and mirrorhydra.gnu.org. And it will check for, do I have any binaries? I can just download and save you a bunch of time. And then anything it doesn't find in the entire chain of whatever it is you're trying to install, it will then download the sources, start building it. And where was I going with this? Ah, yes. So the daemon then can, has some special privilege to remount the GNU Store as needed, as writable, and then inject any of the objects that either downloaded or built into the store. So, so one of the workflows that I really liked about GNU Geeks was in Debian, I'm packaging bootloaders and with the ARM64 systems, they pretty much all require ARM Trusted Firmware, but almost every ARM Trusted Firmware is some vendor fork somewhere, and to get it into Debian right now, we only have a single vendor fork of ARM Trusted Firmware for all winner systems, and it doesn't even cover all all winner systems. So in GNU Geeks, it's fairly simple to say, to add just one more package definition that shares a bit of the same system as the other ones, because you can inherit the existing packaging, and then add, well, actually I wanna use this for upstream for this particular variant. And that was a lot easier to actually get a Uboot package that has kind of everything ready to build out of the box. So, yes, question. Yes, question from IRC. Is GNU Geeks suitable for self-contained cross-compiling? Yes, well, I don't know about self-contained per se, but it does have a pretty good support for cross-compiling. I've been cross-compiling Uboot, a kernel, ARM Trusted Firmware, those sorts of things, and I think it even uses a bit of functional magic in order to actually derive the cross-compiler rather than having to build an entirely new package out of it. So I was pretty excited to see the cross-compiling support, which in Debian is only kind of a recent addition. So there's a bug report to do a request for packaging on Debian, and I don't know if anybody else is, if this is sparked their interest in that, could sure use a lot of help to actually get anywhere. It will probably also require a Debian policy exception because of the file system hierarchy standard, not really being a thing for Geeks, but because you can install it on a foreign distribution, I think it can be really useful for end users to be able to update to newer versions of software where it's not really appropriate for Debian. You can run Debian as the base system, and then all the user packages that you actually find that you actually interact with or maybe a select set. I think you might have a bit quicker uptake than say with the back ports. It's got its quirks because you'll have to set your path appropriately to actually be able to use these things, but yeah, that's about it. All right, well, thanks for hearing me ramble on a bit, and thanks to both the Debian and GNU Geeks projects. You've shown me a lot of great things, and I hope to contribute back more.