 Felly, hella y rhoed o'r ffrwng. Ym ymdwy yw Chris. Mae'r ffrwng i'ch gweithio'r gweithio gan gynwg, ond o'r gweithgo'r cyhoed yw'r gweithio gan gynwg gweithio rhoed o'r ffrwng ffosden. Mae yna tîm i gael. O, mae'n ffosden. OK. Felly, e'r gweithio'r gweithio gweithio gan gynwg gweithio, rhoed o'r ffosden yn rhaid i gwyno'r rhaid o'r rhaid i gael, i gyd yn 2016. I remember installing it the night before, which is how I can remember about it. So some of the concepts involved in geeks take a bit of time to get used to. So I've tried to break down the benefits and properties into three different areas, dependability, flexibility and geeks' position as a general purpose package manager. So, starting with dependability, I consider geeks to be very dependable. The main building block for that dependability is the mutable store. So the store is a directory where data can be added, but it can't be changed. With geeks, the location for that is kind of fixed, so it's at slash gnu slash store. When you build packages with geeks, you can fix the behaviour of those packages at the build time, so it doesn't change later on when you're using it on different systems. I'll go some concrete examples of how to do this later on. The other part of dependability is kind of coping with inevitable failures. So sometimes things go wrong and the best thing to do is just go back to when it was working before you did anything. And the architecture geeks uses kind of elegantly supports that. So having spent a couple of years now doing odd things of geeks, I can say for sure that it's quite a flexible tool. Geeks supports downloading packages like binary packages from the internet. It also supports building those same packages from source. That building from source can come in really useful when you're building your own packages or tweaking the definitions for ones you already have. Geeks also elegantly supports building, having multiple versions of the same package and having multiple versions of the same package even on the same system. You can even have multiple different variants of a single version of a single bit of software, perhaps built with different compile time arguments. Finally, geeks is a general purpose package manager. Personally, this is something I have a big preference for. I think the kind of problems of package management are best tackled without fracturing the effort along lines like programming language or fields of use. While geeks is a general purpose package manager though, it has specific tooling. So if you want a docker image, for example, there are ways of getting from geeks to that. Geeks is a package manager because of this, works on many systems. So you can use it on Debian, but also there is a system called Geeks SD which is built around geeks itself, so it kind of goes across that traditional boundary between package managers linked to an operating system and other package managers. On to some practical stuff. You might be able to guess what this command does. It will look at the available packages, find one called hello, and then install it to the current user's profile. In case you haven't encountered hello, its purpose is pretty simple. Provides a binary called hello that prints out hello world when you run it. The profile bit is key here and I'll try and explain this now. So let's take another example. Say you've installed two packages, samtools and bowtie, you would end up with something that looks like this. So you have a dot geeks profile simlink shown at the top in your home directory. It would point at another simlink in the slash var slash geeks directory. That simlink would then point at another simlink named with a number and that simlink would then point into the store. So it's a directory ending in dash profile in the gnu slash store directory. The number of the simlink here 42 is the generation of the profile. So it's a 42nd version of your profile. The profile directory doesn't actually store the files for those packages. It's just linking to those files in the packages in the store. All this complexity is actually hidden from the user. If you run the geeks package command, you'll usually just get the environment variable set up for you to use the packages you've installed. So here the path will be set up to include the profile and that would put, you can use samtools and bowtie too. So what happens if you make a change? Say you remove the bowtie package, so just like you can do dash dash install, you can do geeks package dash r and then the package you want to remove. So a new generation of your profile will be created. So there's a 43 that's appeared and the simlinks are now linking through that to the store. The directory in the store for the profile at the bottom here has been updated and it no longer contains the bowtie to package. It's just pointing at the samtools package. A critical thing that's happened here or rather not happened is you still have your old profile. You still have all the stuff you had before. You're just not using it anymore. What this allows you to do is to roll back. So you can just switch back to using your 42nd generation of your profile and you're back with the same things you had before. Because the stores are mutable, you know that this is the same stuff. It can't have changed. So yeah, to recap, the immutable nature of the store combined with the way we're using profiles here means like you've got that dependability. Even if stuff goes wrong, you can roll back. So on to some flexibility now. So the geeks package command deals with profiles but there's another command that also does that as well, the geeks environment command. The purpose of this command is to create development environments. So you want to work on a package but you don't want to install the dependencies using the geeks package command. So you can run the geeks environment hello and it will set you up with an environment which contains the things you need to build that. So like GCC in this case. So if you were to run that command, look at your fine GCC on your path, you'd see it appearing off the end of the screen here. So looking past the purpose of the geeks environment command it's a very flexible way of dealing with arbitrary profiles. So here's how you would generate a profile containing the hello package and the dial package. If you were to look at the profile in the store you'd see something like this. So we're looking here at the bin directory of that profile and it's composing those two packages by synlinking to the files in the bin directories of those two packages. So to recap very quickly. Installing packages to your user profile can do by geeks package-i. Entering a development environment you can use the geeks environment command and to enter an environment with like specific packages you can use the dash-add-hoc flag to the geeks environment command. Right. Let's look at some more interesting ways of using geeks packages. So usually geeks packages explicitly rely on the dependencies by pointing them where they like lie in the store. So this is really useful for reliability because the stores are useful if you're pointing at something in it you hope that it doesn't, if you have some degree of reliability that's not going to change. So the behaviour of your package is kind of on that stable bedrock. It also provides some flexibility because as you know what's used you can remove all the unnecessary stuff around the side. So this is what this command does. Now the bash shell started in this environment will be started in an isolated mount namespace with only access to the parts of the store it needs. Some certain bits of the root file system and your current working directory. It's also in a different pid namespace and several other namespaces. So if you were to look at, say, the contents of your store you only see what you need for running hello here and bash. You also don't have like ls so that's why you have to use echo. So note here that I'm not talking about containers and well I am talking about containers in the sense of just Linux namespaces. So like somehow the term containers has kind of been conflated with managing state and package management. So Geeks kind of you've got some separation between these two concepts. You've got the information about what's using what at the package manager level and that enables you to kind of add on container stuff on top. So this is another way of looking at that information. So here's a graph generated with Geeks for the hello package. It shows the references of that package and the things that you're referencing, their references as well. So this along with another command called Geeks size can be really useful and understanding kind of what this graph looks like in the store. So this, if you actually look at what those references mean one of the references that the whole package has is in the way that the binary is linking to the shared libraries it uses. So you might not be able to read this very well because the text is quite small. But on the left here we have calling the LDD command on the hello binary in the store. And it's printing out these long, long paths to show which shared libraries it's referencing by their absolute location in the store. Now this gives you some reliability as like the whole binary won't change in its behaviour or at least not from these things changing because they're not going to change in the store. So this is a concrete example of how you fix the behavioural program at build time. So you build it to use these exact shared libraries and then that won't change even if you deploy it to a different system. So this, you might know static linking and dynamic linking. I mean this is a kind of hybrid approach where you're doing dynamic linking but you're getting some of the benefits and the stability from static linking. It's quite cool. So back to some flexibility so we've looked at geeks package-install you can tell it to use a different profile. So if having one profile per user, which is how geeks works by default isn't enough you can have arbitrary numbers of profiles. What that will do is it will create some sim links where you put the profile. So here in the current directory that points to the relevant bits in the store. Another neat way that I've come to use geeks is to integrate it with a program called DRM. So DRM is a completely different thing. It's an environment switcher for your shell. So you can use a file called .nvrc push it in a directory and when you enter that directory DRM will evaluate that file and set up your environment and make some changes to it. You can get geeks to say which environment variables you need to change to activate a profile. So you can hook the two things together by putting something like this in your .nvrc file. So use geeks-add-hot GCC toolchain, Python, Python, Alex and L the temp food directory so that when I enter it geeks finds out what environment variables I need to change to use these and then DRM will adjust the environment to match. So this is quite a cool way of managing dependencies on your system. You kind of just change into the directory and you get GCC appearing or Python appearing. Okay, another cool thing. So geeks pack is really practical for the benefits of kind of bundling software together but also using geeks at the same time. So some quick examples. Here packing up Gile, Emacs and Geyser so into a table that contains not only those packages but also all of their indirect and direct dependencies in the store. You can also generate a table that's kind of of a format compatible with Docker using the same approach. You just add dash f Docker onto the command. Right. Geeks has around 6,500 possibly more packages but you might want to use something that's missing. One thing you could do is look at packaging it. So geeks contain several importers so programs that can use existing metadata say on sites like PyPI, Python or rubygems.org for Ruby and create a package definition using that information. Now depending on the importer and depending on how complete the information is that has available, you might get a package that builds or you might need to do some tweaks but there are very quick way importers are a very quick way of building packages. So that's kind of readable. Here is an example package definition. So this is for the hello package that we looked at earlier. It's pretty simple. It's not, it's literally just this bit of code. You don't have multiple files. The entire build process is encapsulated into the build system which is around the middle kind of definition. But this is a simple example of something that kind of follows some conventions. Geeks is like you, for some stuff some stuff's more complicated and you have to build to package definitions to handle that. Now one part of why Geeks packages are so dependable is what this kind of represents. Even though we're not declaring any additional inputs beyond what the build system provides this package represents kind of all of that information. So as well as the source table. So the source table is kind of a bit towards the top. What I'm getting towards is that you can pull out the information and graph it which gives you something like this. So this is a representation of that package who are graffing the derivations for the package. I don't expect you to read this but I've coloured some of the parts differently. If you notice the data the source has this hash in the bottom represented in base 32. That means it's fixed data going into this kind of process. So the fixed data parts of this process are coloured in black and they kind of lie around the edge as you'd expect. The bits coloured in purple towards the middle are the derivations. So the bits of the process in building hello that take some data and or some output, some other derivations combine it together to build something. So hello is one of the he's purple nodes towards the left I think. Now you might have noticed in some of the examples I had earlier that there was a hash showing up. So that hash represents this. This graph all the bits of data that are going into it. That means if you change any part of this graph the hash will change and the package will be different. This is how you kind of get different packages for different build steps. The hash describes all parts of the process. So if you change a single part of the process you get a different hash, a different entry in the store. Right. So thank you for listening. I think we're doing very well on time. I didn't put any demos in but I'm quite open for going back and trying to look at some of the things in more specifics. This was kind of a like I before using geeks for a couple of years. Now I was quite familiar with doing Debian package things and it took me at least a year to come around to the idea that this was kind of even sensible. So it's not something I'm expecting to convince anyone kind of overnight but if you want to talk to other people there's a quite active RC channel on FreeNode and there are a blog post and mailing list on the website. There's also a talk, a couple of talks actually in the HBC track tomorrow including one by Ludovic Cortez who kind of started the geeks project. Any questions? Yes, so the question was does geeks have some way of clearing out old packages? And the answer is yes. So in the same way that actually I had an example I didn't have an example for this. So there's a command called GeeksGC which is apt to name garbage collector. It works in the same way as a kind of you might be familiar with garbage collectors from like pro-going languages. You have a graph, the graph of items in the store which have kind of dependencies on each other. You can traverse you can start at the like known kind of routes, the generations of your profile and the other bits of software that are installed on your system. You can find everything that is used in the store and then you can cut out everything else and get rid of it. That's a really, really safe process because you've got those absolute dependency links you can remove stuff without worrying about someone using it. Providing that they've kind of registered their use for it which is like Geeks package and Geeks environment make it easy to do. So yes it does you can get quite big stuff and especially on small systems you can run out of space but you can work out what's taking up your space and get rid of it from the store. Two questions. I'll do both together. I have my own particular software that I'm trying to package for HPC purposes and so package management is always an issue. Currently supporting, directly supporting this back because it works for us. Easy build is a bit of an issue still but what I want to be able to do is to build my version of software with different variants. To build a version which has say 32 bit labels, integers 64 bit floats or 32 bit floats with a particular version of boost with a different version of Seagull with an older version of Flex and have like six different combinations coexisting to be able to swap them. Second part of the question is I'd like to have this deployed in such a way that you can say this is going to be my installation path and then make that available on say generally available for everybody on the cluster. And not one machine but say 5000, 10,000 cores something like that. We're not everyone is going to have geeks installed there but then I'd like to then wrap it with a module system to then address what's going on. Could any of this fit with geeks? Yes, so to summarize two questions. First, variants secondly cluster deployments so I don't personally use any HPC stuff maybe some of the people who do because I know they're in the room can join in afterwards but yes, variants are a possible thing so I was in the SPAC talk earlier so I know what was said in that talk about variants and geeks has some you can add some things on the command line for rewriting the kind of directed acyclic graph packages but that sounds like it might not be quite sufficient for your case but you can write out the package definitions and write out some functions to make the transformations you want to and it should all be possible including using multiple versions of things and managing it elegantly. The second part about profiles deploying this so it sounds to me like the geeks kind of package command where you tell it to use a profile that might fit conveniently you can have a name for that that collection of software that you kind of curated and you can put that somewhere on the system and then a user can source the slashetc slash profile file within that and that will set up the whole environment Can I load that up as a TCL or a Lua module? So could you load that up as a TCL or a Lua module so could you have a Lua module within that profile which you load up using Lua or what other people are using modules? I'm not familiar very much with the modules thing for HPC but like if you wanted to if you had Lua software in that profile then that should work if you had a Lua program to load that profile then that should work as well Leger From the discussion of your own because I do it with mix.geeks you can generate a module with it you can use it as a specific language and generate a HPC module so that's an explanation Cool, so I'll repeat your answer I think that was a someone uses Nix which is very similar to Geeks and it's possible with Nix so yes Cool, yep About the distributions that I can use Geeks on top of is one of them on Windows? So I don't know if it's working but I know that's the bashal ganu thing on Windows I guess it should work I mean you might have some issues getting the build demon to work because the build demon uses namespaces Linux namespaces so you'd have to see how well Windows this kind of kernel supports those to do the isolation but maybe it might be a question of how many compromises you make So do you know of anyone in the geeks community that has tried to to run the package management of the package? Not sure if I repeat the question but yes, this is about Windows and now I don't know of anyone who's tried it I've tried Nix on Windows with bash like you said and it worked with recent updates of Windows then because with the first ones that's the right system I'm not asking because I want to use Windows if I adopt something and then I have users using Windows that want to use it then Right, so the answer was with recent versions of Windows next works so that should be good. Right, I think time's up so I have some stickers if people want stickers and you can come and ask me questions afterwards as well Yeah