 Hello. Welcome to Budapest. Welcome to Flock. Welcome to my talk. My name is Merlin Matasius and I'm a software developer at Red Hat. And this talk is for Fedora package maintainers who are already familiar with the RPM packaging tools and are interested in learning about some of the Fedora tooling that is available to help in creating and building modules. And if this doesn't match your demographic, you might want to just take this opportunity to go to another talk or take a nap or get some caffeine. Otherwise, welcome aboard. You can stick your carry-on luggage wherever there's room. Make sure your seat backs are up. And yeah, cool safety features in the room here, there. Rogic. And but I would like to ask, try to hold questions to the end and we'll see what we get. So the next next 20 minutes or so I plan to give you a brief overview of Fedora modules and the terminology we use when dealing with them, give you some ideas about when you might want to create one of these modules and highlight a few tools available in Fedora to create and build them. But I am specifically not going to talk about the options and the way Fedora modularity is currently designed and the cases when you need to avoid creating a module that was discussed an hour ago in this room with Adam Shamalik's talk. But Adam, take off. Okay, let's get started with a brief introduction to Fedora modules. Modules are simply collections of packages with some extra metadata. That's the special sauce. And you can imagine that our modules are virtual repositories of packages that are embedded within a master repository that also carries this metadata describing the module to package relationships and other per module data such as defaults. And then there's the DNF module commands that know how to handle these package collections with the special sauce. And now to make sure we're all on the same page and using the same terminology, let's go through some of the basic terms that I'll keep using in this talk and we'll be you're going to hear them if you use modules. First is just the term module. That's the logical collection of packages representing something such as an application or language runtime. And examples of existing Fedora modules are known JS, MySQL, and Pearl. And the stream is the particular version of a module's content, typically locked to a major upstream lease of the packages that the module provides. Examples are stream names are like six and eight, which is applicable to Node.js. And the latest and develop are commonly used by all sorts of modules. And then lastly, the version. Version. There's a lot of confusion around this term. It's overloaded. Streams are referred to as versions, specifically in the module terms. It is the specific version that's assigned by the build system in the form of a concatenated Fedora release and a time and date stamp. So just be aware that that term is overloaded when you hear module and version together. Make sure you have the right context. I don't want to use the word context because that has meanings in modules, too, which only cause more confusion. Then the module installation profile is a list of packages representing a specific use case of the module stream. DNF knows about these profiles and it allows you to install a set of packages in a particular module profile just by referring to that profile name. And some commonly used profile names are client and server and minimal. And then here in defaults, there's yet another overloaded term. Modules can have default streams, which is the stream that's used if you don't specify a particular stream. And then each stream can have a default profile, which is the same thing with profiles. If you don't specify a particular profile, you get the default profile. And now is also a good time to point out that there is a wonderful Fedora modularity homepage that has all sorts of information, links to documentation, links to the issue tracker. I'll give you the URL at the end of the talk. And I'm going to keep referring to that documentation website. Go back there, a lot of information. Okay. So when is it a good time to package things using modules? A little bit what was discussed last hour with Adam's talk. But basically, when they're rather than being stuck with a single version of a package that was shipped with a release of Fedora, modularity solves this too fast, too slow problem that you keep hearing about, allowing package maintainers to provide parallel availability of multiple versions of a package in the form of module streams for the same Fedora release. This is useful for developers who need to maintain older versions of software as well as those that are trying to provide the latest and greatest newer versions. And it also allows the same versions to be available across multiple Fedora releases. That's a little bit of a change from the traditional forward marching versions in the straight packaging world. However, only one version of a packet or module can be installed at a time. There is no parallel installation of multiple streams. And if you have decided that creating a module is the right thing to do, it's good to know what already exists before you start making your own module. And at the package level, DNF repo query, DNF list and DNF search commands work just as they always have. But they also include not just the non modular packages, but the packages coming from default or enabled module streams. And just to see a list of what modules are available along with their streams, profiles, default streams, default profiles, use DNF module list. There's more information about that on the modularity documentation page or in the DNF man pages if you can find them. And as this slide mentions, there's a couple of things I'm just going to bring up so you're aware. It's kind of beyond the scope of this. But the Fedora is a module build service. And the product definition center have APIs that you can query directly and get all sorts of internal information about modules and what they contain. API calls return these blobs of JSON. So you can hack together a script in your favorite language or can even do it from the command line and use the jq tool to extract some of the bits. Check that out later if you want to do that. The product definition center is literally pdc.FedoraProject.org. Cleverly named. Yes. So if you do decide to create a module, it comes down to identifying a set of RPMs that you want in the module, creating one module MD file for them, and submitting a single build for the entire module. And that builds all the included RPMs for all current Fedora releases. If you were going to do that in the non modular way, you have to do the RPMs and then submit each RPM build for each current Fedora release over and over and over. And this module MD file is a YAML file defines a module stream specifies what source RPMs need to be built and what disk it branches they come from as like a basic summary, description, licensing information, and the information about how the build system is to deal with that and build it, such as the Fedora releases that it should build for the order in which to build the packages, extra RPM macros, and it also lists the installation profile packaging lists. And if you want to see what one of these files looks like, I grabbed the one from the Fedora modularity documentation and stripped out the comments, trying to get it on one page. That's not the right art chart. This is the I chart. If you really want to read this, go back to the Fedora modularity web page, you'll find this full example full of comments and picked apart bit by bit, telling you what is legal for each thing. And then copy and paste it as template for building your own module. And when you've got your file ready, it needs to be pushed into disk it into the modules namespace. Make sure that the file name you use matches the module name with a dot yaml on the end. The repo name also needs to match the module name and put it in a branch named after your module stream name, so that the build system can find that. And remember that documentation website, it explains all this, as well as how to request new disk it repos and branches. If you're converting some existing Fedora packages to modules, there's a Fedmod RPM to module command that might be worth checking out as an alternative to manually copying, pasting, editing the module MD file. At the very least, it'll give you a syntactically correct file that you can use as your starting point and start tweaking it. And if you run that it does some dependency checking and other things, so it's going to crank a while before finally dropping out the file. And whatever means you use to get your module MD file, the Fedmod lint command will run a syntax check and tell you if you've forgotten any pieces. And there has a few other options that if you just run Fedmod, it'll tell you about them and I'll give you a link to the documentation at the end of the talk. And there are several types of module builds available. I'll go through them quickly, how they work, starting with the standard way that eventually will allow you to release your module build and send it for the Fedora repos for the community to enjoy. Standard module builds are simply submitted using Fed package with a module build. And you do that from wherever you put the local clone of the module, just get repo just like you would do with packages. That submits the module to the Fedora module build server and gives you back a list of numeric build IDs. And if you write those build IDs down or copy and paste them and then run Fed package module build watch. It'll show you the progress of the build. And then when it's done, Fed package module build info will give you all the gory details. And there's also a nice web UI where you don't need to remember all those module build IDs. You can click on the individual builds, it'll show you the components, you can click through on that and find out the status of each one as it's going. And it also keeps track of recently completed ones just so it's a whole lot easier to use. Okay, now you've finally successfully built your module and you're going to want to use it and check it out. In raw hide becomes kind of a matter of patience. You need to wait for the next compose to take place, wait for the mirrors to sync. And then you can just use DNF module commands and get right at them. Or older Fedora releases. You need to submit the update via Bodie just as you would do with non modular packages. But what you need to enter for the candidate build field in Bodie is a little non intuitive. Go back to the Fedora modularity documentation website and that tells you how to figure out what to enter there. And then as with raw hide, be patient and make sure your updates testing modular repo is enabled and then the DNF module commands can be used again. And if you don't like all that waiting before you can test out the module there is the odcs, the on demand compose service. It'll take your module build and create a repo for you just in a matter of a minute or so. If you run the command in that form, it'll spit out little lists, a link to the repo that you can use directly or a dot repo file is also a link is included in the output. And you can put that in at cm dot repos dot d. Correct. Once you've got the build odcs will compose it for you. Yes. Yes. But you already have to have had it built and forget to mention install the odcs client package. It does not release it for you. It just builds one that you can you can use while you're waiting for Bodie to do its work or test before you submit. Yes, exactly. That's why it's the testing repo. Okay. Now, if you just want to skip all that and avoid waiting on the Fedora infrastructure, you can build your modules locally. And there are two types of local builds. There are online builds and offline builds. The online builds build on your local system using mock, but they're still querying the getting external package and module information from the MBS and Koji. And if you do an offline version, you have to offline make make a repo that has all your base packages that your module might need and have that available offline. But that can be useful, such as in workshops or when internet connections are unreliable or behind firewalls. And then after you do a local build, you somehow want to reuse that. And you do the local builds under your your home directory under module builds slash builds. It will drop the build module actually as a repo there all ready to go. Easy just like to set a shell variable to point to where that's at. You're still going to need to look in that module builds module build builds directory and find out what the actual name was because the version gets filled in by the build system. And it's kind of tough to guess what that's going to be ahead of time unless you have down to the second guessing timing. And then the DNF using the repo from path options. If you use those before, you just point it at that local directory enable it. And but you have to do that every time you use the DNF command, or create a repo file and that's a yum dot repos dot d pointed to that and then you can just use it without having to fill in the extra options every time. And then there are also scratch module builds that we're just asking about. They're done in the fedora infrastructure. There's somewhat analogous to scratch package builds and that they don't quite count as an official build and you can never release them. They do also have the benefits that you can use a module MD YAML file that you haven't pushed into disk it yet. So you can kind of check it out. The front row, at least not yet unless somebody's working on that one. So you're going to get to the problem here that I was going to point out momentarily. There was a question also here moments ago. Eventually you'll be able to specify RPMs that haven't been pushed to Fedora's disk yet by specifying custom SRPMs on the scratch build command line. But there's currently a fix waiting in the Koji pipeline that it's been committed. It just hasn't been pushed out to the infrastructure yet that you get that working and monitoring the builds is done the same way because they are done by the MBS. Yes, there's synonyms. You can do the same thing when you're doing package builds. You can say, you know, the package scratch build or build dash dash scratch. It's the same thing. Yes, tagging isn't it doesn't go through the final processes of uploading. But it skips all the final steps. And it's kind of specially namespaced that it isn't called module dash. It's SCR mod dash for the for the tags that it does do. It doesn't know how to. There's no odcs. So unfortunately, it's using the artifacts from a scratch module build is kind of tedious. You can go through the process, create a directory, get the build info, download all the packages that you just built with the Koji download. Then you have to grab the as you build a module, the module MD file gets augmented with build information that you can download that manually from the PDC, strip out just the module MD file, drop that into your directory, create a repo from it. You'd also need to manually remove any RPMs that were filtered out by your module MD file. And then add that module MD file back to the repo. It's is quite a pain. But then you can use that directory just like the local module builds from the previous slide. So the biggest benefit is just being able to run a build before you push things to disk it and see if they're going to work. Gives you a few iterations to fix things before having to update like the release ID every time over and over. Okay. And that is all I have for you today. Hope you found something new and useful. Here's the links I promise the fidora modularity homepage there at the top. That has I said links to the issues and documents. There's the direct link to the documentation. If you forget to bookmark it if you're going to be working with modules and if you forget to just type fidora modularity in your favorite search engine, and it's going to be one of the first few hits. Fed mod that I mentioned, there's where its source and documentation is. And then on IRC, if you are stuck and need help, get the fidora dash modularity channel on free node, and you'll find a lot of people that are smarter than me that can probably answer it for you. So, but now any more questions? Yes. The scratch builds are actually using the MBS infrastructure. It will build across all the architectures. That's kind of their real purpose. It's using the same server. But they're just the it is using the regular build process, except for the final phases of actually composing it ready to get pushed out with the caveat that the custom packages just isn't working yet. I'm sorry. Well, Fed mod does the build odcs takes that build and does a compose for you. It gives you a repo that you can then use. Otherwise, if you just build the module, it's somewhere in Koji. There's just pointers to it. There's tags. There's no actual repo that you can just pick the module back out again. So odcs turns that into a repo. And then you can just point DNF at it. And it's kind of looks like it's already in the repos. Just an extra one. Yes. Scratch module builds are expecting to have an actual file. It says scratch module build dash dash file and then the ammo file, which you could check out from the branch that is the PR. You mentioned you're doing like yeah, PR in the module, you check that out. And then on command line specifically pointed to that path in the local checkout. Yeah, but the inside is the ammo file and pointing to the repo of the RPM. Yes. In that case, you might need to check out those packages. Build a source RPM from that and then handed handed the whole thing. Sounds like an enhancement request. I think we need to wrap things up here. So thank you for attending. Thank you.