 OK, so up next, we have Corbin with the NetBSD project talking about Dev Setup, a local development environment for any platform. Yeah, thank you very much. So keep your mind, then let's get started. Thank you for joining. As was said, I'm Pierre Poucheray, Corbin. We're going to introduce package source, actually. This is the plan for this session. We're going to use package source in a particular way. I have to make some changes to make it fully work. And then Dev Setup happened. And hopefully, you will get to see how it came in and how it happened, what it can be useful for. And hopefully, get to use it for yourself. Let's see. We'll start with package source. So to introduce package source, I have to speak about NetBSD. NetBSD, as you know here, is an operating system based on the 3DSD originally. It's been created on 16 CPU types. So productivity is really a stronger aspect here, one of the key aspects of the system. As has been as a tradition with BSD systems, we have the base systems developed as a set, independently from the packages. And the packages, the third-party software, actually comes from the package source project. Another noteworthy aspect is that NetBSD is still using CVS for development, just like package source does, actually. So like mentioned, package source is the official source of packages for NetBSD. But it's also available for other operating systems. NetBSD is very portable, like I mentioned. So 16 architectures are supported by NetBSD and also package source. But on top of that, 23 other operating systems are supported by package source, even if you count Linux as just one operating system. So this is a great aspect of the system. It actually works for much more than just NetBSD. And I mentioned CVS, but the project is also mirrored on GitHub. It's really automatic conversion project, used with care, but basically it's there on GitHub if you want, inside the NetBSD organization. So if you actually clone it, which we are doing here, we use a GitHub account, in this case, we can use SSH. And we're gonna have a look at the project structure, how it's organized. So we basically cloned it, we can change folder into it and have your directory listing. We initially have a bunch of folders, but among that, thankfully, we have documentation, a readme file. So let's open it with my favorite editor, in this case. So we are inside Vim. We can see package source is the default package manager for NetBSD and smartOS, among others actually. And not only that, but we also have, there's a note that we can use it in unprivileged mode and can install it this way. So let's see what this means. We keep scrolling and it actually mentions to use package source on other operating systems than NetBSD, you actually have to bootstrap it. And this is the procedure. So this raises a question, can we actually bootstrap package source but also in unprivileged modes, like without any privileges? Can we use this to use package source everywhere? So let's try that. We're going to bootstrap package source. So we go back to the base folder and we notice just like documented, we have the bootstrap folder here. So we're going to change folder into it. Here we are, we list the files and we see we have, as mentioned, documentation for most, if not all operating systems supported. And there is a generic readme file, so we're going to open it. Are we going to open every other readme file? Don't worry, we don't have time for that today. So let's do that, open Veeam again. So what do we have? The bootstrap program, some short usage information. So to try to get package source working on a system, we have to try this, but it mentions a detail, we have to do it as a route. But that's not what I wanted. So let's see if we can actually use it in a different way. So we check bootstrap directly, we run it. We have the actual usage screen, not the simplified one from the documentation, and we can see there is an unprivileged flag here. And so let's just try that. So we run the command, we have a few CPU cores, we can do that with multiple jobs. And when this is done, we actually have a nice message, sending us, thank you for using package source, yeah, thank you for providing it. We have to add a few variables, whatnot. So actually, let's be curious and look at the source code for package source and see how the unprivileged mode is actually implemented. What does it change? What does it do? This is actually implemented in the subfolder make, unprivileged.mk. We see right away that some variables from package source changed the behavior. In this case, we have the unprivileged modes which can be enabled with yes and no. This is basically the make.conf configuration file from package source. We also have an unprivileged user which defaults to your own user, logically same for the group, same for the list of groups that you have access to. Then we continue, we have a bunch of variables, which is just two here. The package users files and the package groups files which restrict which users and groups can be created. It makes sense since we have no privileges, we're not supposed to be able to create users and groups. And so this basically sets, when this value is empty, the package source knows it cannot create any user. And if you happen to be able to create one, you can add the value here. So we continue into the unprivileged implementation. We can see here that it also disables user and group creation here by, yes, this variable here, package create is a group. Then additionally, it overrides the CH group and CH own access, basically forcing them to always succeed even if they are not able to. So this is quite transparent for us. And in addition, it will not try to register any shells. So even if you install bash as unprivileged mode, it will not try to modify your thresholds to see shells, for instance, on a BSD, or affect the system in any way like that. Okay, so we've gone this far and now why not run services with package source, which it can do on a normal system. But in this case, let's try it without privileges and on any system actually. Thankfully, there is a package doing supposedly just that. In package tools, it's called RC.subber, which is coming from NetBSD actually. It's part of the unit system to boot NetBSD and it's responsible for running services. Just as mentioned, if we open the description file for the package, it allows us to use, to take advantage of the NetBSD RC system anywhere. So can we use this package to run unprivileged services then? It should work, because normally you can do that. But if we try on a NetBSD system in this case, we can see that this package is only available for these other platforms, but not in this case with actually Darwin, but actually it didn't work for NetBSD. Initially, when I started looking into that, but also unprivileged mode is not supported. This package is not available in unprivileged mode, which I thought this is a shame. So we have a challenge on our hands and let's change a few things, see if we can actually fix it. And the first part is actually very easy. If we open the make file for this package, for package tools, RC.subber, we can actually add NetBSD to the list of platforms supported, okay, check. Then we can just remove this variable, not for unprivileged. This should already help a lot. You can see here that I added the variable, I'm going to introduce that in a moment. Where are we? Okay, I showed that. We continue and I also added a function to the subber, the copy of the RC subber file into this package to have the load RC config bar, which allows us to read etch.sc.conf in package source instead of the system itself. And like I mentioned, I had to introduce the sysconf.base variable, which points to the configuration files from the base system. It's typically set to slashetc, but in unprivileged mode, we want to have everything provided by package source. We cannot rely on the system to provide anything because we don't know if we are on NetBSD or not. We don't know what's there or not. And this is actually important for RC scripts. We're going to provide our own RC scripts from the packages in unprivileged mode. So we need to be able to tell package source where to find the RC scripts instead of hard coding them in slashetc. So I had to introduce that and make a few changes in different parts. One part where I had to do that was the bootstrap. So adding it here, adding it to the make.conf configuration file, adding it, of course, to the documentation and the default configuration file here. And this meant that I had to update RC scripts in order to remove the hard coding slashetc and replace it with sysconf.base. So then if you're on package source privilege in NetBSD, it defaults to slashetc, it doesn't affect the behavior. But if you're in unprivileged mode on any system and you want to run services from package source, this then allows the system to find RC server and the RC scripts and so on. For this example, which is PHP-FPM, it was just a matter of changing that. So in practice, I looked at code a few packages, I already converted this many packages, I think it's about 20. I'm still missing a lot. Obviously, code package source has a lot more packages than just that. Notably, I have to look at PostgreSQL, MySQL, MyIDB and so on. So now I want to dive into what we can do with this. Now that we can run unprivileged local services with package source, if we combine the possibility of package source with the ability to run services on any system, I think we can actually enable one new use case for package source. This is what led me to create Dev setup for local, as a local development environment. Right now, this is hosted on GitHub in my Corbin namespace. So as you can see, it's fairly simple so far. We can do the same as before, just clone it, see what happens. We see that, thankfully, there is a bit of documentation and we can do just like before the writing and open it, see what I wrote earlier if I don't remember. So we have, like I mentioned, the development environment. In this case, everything is built from source. To get started, we simply run Dev setup in it. All right, I don't know about you, but I usually don't pipe curl into my shells. So let's have a short look at Dev setup, what's inside, even if I wrote this, it's always a good practice. We can see that we're going to use Git or we can probably guess it from that. Then we continue, skip some lines, obviously. We can see that by default, it's probably going to run fast CGI somewhere on localhost, install a bunch of packages, maybe even run an app server with the default password here, and a bunch of services. So here we have Dovcott, Nginx, Postgres, a bunch of others. But we have to notice also that the default upstream in this case is from HPSD. So what's that? It's a fork of NetBSD, actually. I didn't add the orange, so I made it blue. I'm just kidding. Actually, NetBSD, as I mentioned, is still stuck to CVS for the main development, and I like to work with Git locally, play with branches, do a bunch of work that Git allows me to do and not CVS, especially when working offline. So I've done that a few years ago. The project lost a bit of momentum, but on the other hand, some HBC developers became NetBSD developers, so it's still a success. But this is on the site, actually. Let's go back into depth setup. We want to look at the file. We continue, bunch of variables. We can notice at the top that there is a configuration file that's going to be read. If it's available, it is documented too. So you can change the local behavior without any different script. This is usually useful. We scroll down. We can guess what the usage screen is going to look like. So this is the main function. We're going to have different parameters. In ETC here, we're going to be able to probably install packages, start and stop services, and so on. And this is actually the case. We have the usage screen here. We can guess what it's going to look like, and there you go. So a bit of information for each command, actually. Start and stop services here is at the bottom. Installing packages, indeed. There's also something about the environment, so the commands are highlighted here. So let's just see how this works. In my vision of things, this is actually package source simplified, which is usually a good thing to have. So I mentioned the environment command. It actually typically outputs this. You can directly source it into your shell. And so you're going to have, if you install Go, directly the Go pass being set, the man pass, the pass to have the command from this environment directly added to your shell. So basically, Dev setup provides every package from package source. Can be managed from a single script. You benefit from package sources release cycle with the releases every three months. You also have security updates, and it supports every major and most niche platforms. So I think this is pretty awesome. It enables a lot of things, including, for instance, cross-platform development, meaning you can have a team with someone using Windows, someone using MacOS, someone else using Linux, doesn't matter. Everybody has the same environment, the same versions. It works with the same tooling. And Dev setup actually runs local services and configures them by default. So you can have an open-ended app service working for everybody, email setup working for everybody for local tests. And also importantly, it deploys consistent versions across every system. So you don't have to worry, am I going to use Python 3.8 or 3.9 or whatnot. Whatever is the default here can work for everyone. We said that indeed package source offers an entry-relage mode, so you don't need to have administrative rights. Everybody can run everything inside their own user, create a new user for that if necessary. This is very flexible. You also benefit from further security features from package source, like the auditing system. So basically inside the package tools, package install package, which is the defaults, to manage the packages. Actually, you have the package admin command, which can fetch the list of vulnerabilities as maintained by NetBSD. And then the audit packages command will tell you all of the vulnerabilities which are known on your system. Unfortunately, right now there are lots, but that's also my full-time job actually. But speaking of jobs, this is actually part of the daily current jobs in NetBSD. So you can actually do the same through the setup on your own system and for your team to make sure that everybody also is not vulnerable to anything. By default, DevSetup also runs a local web server. I think it's nginx that I configured and it even provides links to the documentation installed by package source. This is not dynamic yet. So if you modify the list of packages pre-installed, it will not match, but that's something that I could improve. So there are a few more things that I could improve. So let's look at the challenges that I encountered or maybe the shortcomings which are still on now. So package source, unlike its name, also supports binary packages and unfortunately DevSetup only knows about sources. This is also for good reasons. Since we don't know where it's going to be installed, some people will use Linux, some people will use Windows, different architectures. We would have to provide binary packages for every system, but on top of that, every user has a different location where they installed the software. So the binary packages would have to know about that. So basically this is not really possible except if you abstract all of that away. However, if you master your own environment, you can also create binary packages with DevSetup and then spread them and use them with the tooling from package source as usual. So this is one maybe important issue. However, there are more things that could be looked into. Like I mentioned, there are more services which has to be converted to the sysconf base variable. Then we could consider maybe importing DevSetup into package source itself if it's useful enough, if people like it enough. Could improve the documentation always, especially the one which is generated as part of the web interface right now. Maybe there could be a web interface to manage the software or install more things and so on, which should be straightforward since there's no need for privileges, no need for root or anything. So binary packages were mentioned. Then also maybe we could use DevSetup to generate containers or generally do this also with package source, improve the tooling to recreate images and so on and so forth. So then I will welcome your suggestions about what should be coming next. Now, before we get to the Q&A, I would like to give a few pointers. So we have to find everything. So obviously NetBSD is here at netbesi.org. As I mentioned, it's also available in GitHub right now with the automated conversion. The same goes for package source, package source.org. Then there is also an excellent package browser online. If you want to know where it's what, you can search for package names and then know which sub-folder it's going to find it to find in with package source.tpc. Then HBSD is also at hbsd.org. I still maintain it somehow, even if I don't really advertise it. Then DevSetup is on GitHub as well under Corbin. I also just imported it into HBSD as one more mirror. Usually it doesn't hurt. Then to conclude, I hope you have enjoyed the presentation. I'm curious to know what you will do with DevSetup. And if I have to summarize it right now, you can enjoy most of package source, thousands of packages, security updates, consistent environments across platforms and across possibly distributed teams. So now this is up to you. And in the meantime, thank you for listening. And if you want to reach me in any way, you have my email address is here, my Twitter handle, and links to my company before Networks, HBSD and so on. Yeah, so that's for DevSetup. Do you have any questions? Yeah, can you go to the microphone? You mentioned multi-platform support. Yeah. When I tried our package source on previous Linux and Lumos, last December or so, I ran into a lot of breakage, including free BSD where it won't even fully bootstrap. Have you done some work on that too? I myself use usually macOS or NetBSD native. So I didn't run into these issues, unfortunately. I run into some issues on macOS where I had to do fixes to bootstrap, for instance. And actually right now, to be honest, the bootstrap is broken on package source current. I don't know why. But yes, indeed, NetBSD is the focus, let's say, let's call it tier one. And then even though other platforms support it, it's sometimes like a bit behind. So the best you can do is actually to report the issues you have so we can see them and maybe developers using free BSD or Elements and so on can look into it. Yorg, you want to add to that? To be honest, support was pretty much that effort. I can't say that anyone in the party of the package source developer would like to have free BSD on a Mac database. In the malls, it has a very active community. I mean, one of the most active in the exercise developer is in the malls. In the malls, it's a bit different but in the malls, it's not a consistent package. So you can have different versions of it. You can have different policies and things going over there on a regular basis. There are also a couple of people that use it for works or it's a reasonable thing. And otherwise, for parts of the package. And what would be an e-mail when you talk about it? Are there any other questions? If not, I can let you enjoy the lunch before everyone else. All right, thank you. So please introduce yourself to the... To be five, six active developers but then most of them went into NetBSD proper, which is great, actually. It was a way for them to get a name for themselves and then be invited by NetBSD because it's a bit more tedious to become a NetBSD member than HBSD. Basically, if you give me like a PGP key and SSH key, I just add you to the... I just give you a commit bit and that's it. Yeah. So, yeah, it allowed me to bring more people faster, which was great. Yeah. That's the biggest problem that we've got with BSD. Yeah, yeah. We use free BSD. Yeah. I'm actually trying to start now a new initiative to get every BSD major project to talk to each other about drivers and see if we can have regular meetings and... There's a lot of things that... NetBSD has this, so both BSD and that, and that previous BSD has the other. Yeah. There's a lot of bits that all of them would benefit and maybe get some more crosstalk. Exactly, yeah. Okay, thank you.