 Hi, my name is Adam Shomarik. I'm a Fedora contributor and also a software engineer in Red Hat. I'm mostly focusing on the Fedora modularity objective. I'd like to give a talk. I was originally planning to do it for them, but I couldn't do that because some health issues, but anyway, I'm back I'm okay, and this is my talk, modularity 2, the rise, the fall and the happy future. So the problem we see with Linux distributions is that they are either moving too fast or too slow. And this is all about different people having different needs. So for example, developers usually want the latest features possible of their libraries and language tags and tools, while on the other hand, CIS admins don't want to have many breaking changes. They only want to install security updates on the system they manage, and they want their applications to keep running without breaking. And there are many different Linux distributions out there each targeting a different user group like Fedora, which is a relatively fast release. It releases every six months. And by releasing, I mean that every six months there is an opportunity to introduce new versions of software distribution. With the new release every six months, Fedora is great for developers. It might be a little too fast to use on a server, even though many people use Fedora on a server. But in general, Fedora is considered to be a fast moving release, and that's great for developers. But if you need stability for a longer period of time, there are slow release distributions such as CentOS, which releases every three years. And that's great for server. Again, I know people who use CentOS on a workstation doing most of their work in containers because some parts of the distro such as language stack can become old over time. But in general, CentOS is really stable and great for server. There are also some super fast distributions such as Arch Linux, which basically don't have releases. And that's referred to as a rolling release, which means new versions can be introduced at any time. So there are many distributions on different speeds, but it's always the whole thing being either fast or slow. So what we see here is that there is a need for different parts of the distributions to be moving at different speeds. It could be a longer maintained version of a database for Fedora server, or a newer language stack for CentOS. And there are many existing solutions to that problem such as Linux containers, which are virtual environments that contain your application, plus all the dependencies so it can run in the isolated environment basically on any Linux distribution. The tricky thing about containers is running them in production, because you need to make sure that you keep all the dependencies in the container up to date. So there are no security issues and so on. And that can be a lot of work. So containers are great for running applications, but maintaining them and building them can be a lot of work. Besides containers, there are also many other solutions out there, such as software collections, copper in Fedora, or you can even go to the upstream and build the versions yourself and maintain them yourself. But that's a lot of work too. So could we have a distribution that is fast and slow at the same time? Can we make it fast for developers and slow for CIS admins? And also, because this is an open source project that we care about our developers the same way we care about our users, can we make it simple for developers and packages to maintain the distribution? And I think the answer is yes, we can. And that's what we do in modularity. We have introduced a new concept called modules. But before we dive deep into it, let's talk about software delivery in Linux distributions. So it's mostly packages, many, many packages, and they represent tiny pieces of the distribution such as individual libraries, various runtimes, etc. But if we talk about application or the whole language tags, it's almost always multiple packages. Often there are package groups or meta packages that help you with installation of those bigger things, but that's not always the case. And that's where modules step in. Modules are kind of like package groups, but a little bit better. Compared to package groups, modules have three pretty important benefits. Modules are independent, they have multiple streams, and installation is use case driven. So let me go through each of these points. When I say independence, I'm talking mostly about the lifecycle. So for example, in Fedora, even though the distribution releases every six months, modules can live in Fedora even for years. And the same module can be built for all the releases like Fedora 28, Fedora 29, Fedora 30 in the future. They can even be built for Epo, which stands for extra packages for enterprise Linux. When I say multiple streams, I mean there are multiple flavors of each module, typically multiple major versions available. So for example, if I have a Node.js module, I can have streams like version eight, version six or version nine. And because these streams are independent from the release, I can have all these streams available to all the releases. And finally, when I say use case driven installation. So I can compare packages to package groups. If I have a package group, I always need to install the whole thing, but modules don't have to be installed like that. There are many different ways how to install the application or the language stack. You might be able to install it for a developer use case or for production use case. A database module can be installed as a client or as a server or both. It's always the same thing, but different set of packages. And we call it installation profiles. And because installation profiles are defined by use cases, that's why I talk about use case driven installation. So let's talk about how it works. If you want to build so many versions, how do we maintain that? We have something called stream expansion. And this is all about building one source into multiple binaries. In most cases, this will be used for building one source for multiple releases. So there is no duplication of maintenance for packages. But stream expansion goes beyond that. Let's have an example. On this picture, you can see two main sections, source code and binaries. In the source code, I have two federal releases, federal 28 and federal 29, two versions of a runtime. I named them runtime 36 and runtime 37, and two versions of an application. And if I want to build it, stream expansion builds all the combinations for me. So I end up with two versions of the runtime for each release and two versions of the application for each runtime. So as you can see, I got four different binaries from two sources. And when user wants to install it, they don't have to go and choose the binary themselves. We have client tooling that figures this out for them. The user only needs to see the two versions of the application. The right binary and the dependencies are chosen automatically by the client tooling. So that's stream expansion, making it hard for computers and simple for humans. And I hope that's not going to be remembered when AI takes over the world. Anyway, stream expansion is all about building one source into multiple binaries, but having it simple for users. And this takes me to another interesting thing, system profiles. When there are so many versions available, you probably don't want to choose the version for each component, right, for each module. That's why we have system profiles, which are sets of defaults that allow you to make choices only when you need to. So you can either say, I want the default or I want this specific stream or even I want this specific stream with this installation profile. And when you choose a specific stream, it's very important that system updates won't break it. One of the features of modularity is that conscious choice is required to change streams. So when you update your system or even upgrade the major version of the whole distribution, the streams you chose will stay on your system. And when you are ready for a new version, you can just change the stream by yourself. So let's have a recap. What are the benefits of modularity? The first step is that thanks to stream expansion, packages can maintain more content with less effort. And this goes very well together with the Federa CI objective, because if we have so many versions, we want someone or something to test that. And let's make robots do all the work so humans can improve the world. That takes me to the second step. Thanks to modules, distributions can move fast and slow at the same time. And this is all about choosing the right stream for your use case. For example, Federa keeps releasing every six months. But if you run Federa on a server, as many people do, you can say that this runtime keeps stable for longer. Or if you're a developer, you can choose even newer runtime and libraries if they are available. And everything is up to you. It's both fast and slow. So we've been doing this for quite a while. So let's have a very long and detailed history lesson. No, don't worry, I'll make it quick. Federa 26, the bolt-on server. Our first release, which was kind of important to us, we wanted to validate if it's interesting for potential users. And also we did a lot of user experience testing. And here are some interesting highlights from the UX testing. I have four questions in total here. The first is about the installation command. And even though users will be able to install packages as the same way as they could before, they will be also able to install modules. So we presented them with three different variants of this installation command. First was the package syntax, like dnf install httpd. The second was the package group syntax, like dnf install add httpd. And finally a new syntax, dnf module install httpd. And that was the most popular choice. In the comments, people say that they want to mix the syntax of installing modules and packages. And that makes perfect sense. So we decided that this would be the main syntax we will use. In other words, our installation profile is important to you. And yeah, people liked it. People like to choose the use case. We also asked them about the system profile, which is the set of defaults for your system. And people like that as well. But my personal favorite is this. How do you like that updates keep the stream you choose? And this is all about moving fast and slow at the same time. You can always have the distribution default, but if you choose the stream, it will stay with you technically forever, or at least for the time it is maintained. So thanks to all of this feedback we have collected, it looks like it's a good idea to continue our work. And that's great. All right, another release was Federa 27, the modular server edition. Or should I say, wasn't Federa 27? Well, we kind of haven't shipped this one. And let's get straight to the point. Revolution is sometimes best achieved through change, that by setting things on fire. And yeah, there are three main things we have learned. Lesson one. Everything as a module is hard for the low level pieces. So in the beginning we were trying to build the whole distribution out of modules, which was great for the application and for the language stacks, but it was kind of hard to do it for the lower level pieces. So we had to change something with that. Lesson two. Changes are better done incrementally. So if everything is a module, in order to transform the distribution from a traditional approach to the modular one, all the changes would need to happen at the same time. So we basically required everyone to add up the new workflows at the same time. And that caused a lot of fire all over the place. Lesson three. Optional gives people time to adapt. That goes back to the previous point. If everything is a module, then, well, everything is a module and not everyone was happy about that. So three lessons learned. Let's have a look at what we did in Federa 28. We call it the application stream repository. And this is how it looks like. We have kept the releases of Federa as they are. So we can see Federa 28 and Federa 29 here. And all the content is still available there as standard packages. So we call it Federa 28 base and Federa 29 base. What's new here is this new application stream repository, which is sometimes referred to as a Federa upstream. And it contains all the modules with all the additional versions that we shipped with Federa. So I have no GS6, no GS8, no GS10 in this example, etc. And what's great about this change is that everything is optional for both packages and users. So that allows us to make the change incrementally and have people adopt this as they want to. So what's next for you? You can learn more. We are building new documentation website. If you go to this URL docs.bagger.org slash modularity, you can see the landing page and links to the new documentation. You can experiment. Federa 28 is coming. But if you can't wait and want to try it now, you can pull this Docker image and it has Federa 28 and the application stream repository in it so you can try the DNF commands. And please get in touch. Are you a developer? As is admin. Here you Federa package. Do you like modularity? Are you missing a feature you would like to help with? Talk to us. We have an IRC channel. It's called bound Federa dash modularity. It's a free note and we are there basically all the time because we have people in multiple time zones. And also you can follow me on Twitter. It's at AD Sama Lake. I post modularity updates at the time and you can also ask me questions and I will point you to the right places. So yeah, that's modularity. Making great distributions fast and slow at the same time. Thanks for watching.