 Hello everyone welcome to the packaging for beginners workshop here at Fedora Nest. My name is Carl George I am the Apple team lead for the For the Apple sub team of this community platform engineering group at Red Hat There's my contact info there on the title slide. We've also got Troy Dawson who is the Apple steering committee chair Well, you know go ahead and introduce yourself Troy and same for Nils. I won't steal your thunder. Oh, that's right I'm Troy Dawson the Apple steering committee chair I will also work with Carl and on some things at Red Hat And I'll turn over to Nils Yeah, I'm Nils Philipsen. I'm an engineer working in CPE usually doing development. I've been Caring taking care of packages for quite a long time of my career, but that's a while ago, but What held for for that a couple years ago still holds now. So there we are Also, you won't be stealing my thunder call Because I'll get it like at my doorstep as it looks like So if if I suddenly jump then that's that That's clear as a windows All right, well, let me go ahead and go to the next slide Quick plug for the we have a survey going on me and Troy mentioned that we work a lot with Apple Which is the extra packages for Enterprise Linux repository This is not part of the workshop at all. Just like I said shameless plug We got a QR code and a short URL there and the actual full URL if you want to take the survey that was also dropped in the In the introduction remarks, but I know a lot of people missed the QR code. It went by fairly quickly It Troy Nils if when you open copy and paste that link into the chat from the slide like that You got it. There we go. Perfect But that is not part of the workshop. I just wanted to mention it again That's going to be open for the month of August for anyone that Wants to just open it up and look at it later or bookmark it So our topics for today We're going to have a few general topics and then we're also going to then we're going to dive into RPM packaging specific stuff For the general topics, we're going to talk about what is source code how programs are made building software from source patching software and installing arbitrary artifacts if any of this stuff you already know that's great But we're going to make sure that everyone has the same baseline You know, some people may have already you know say messed with a make file before and they're familiar with that Or they know about patching software But we're going to go through these basics the fundamental building blocks just because they're crucial to understanding how packaging works Then when we get into the actual packaging side, we'll talk about, you know, what is an RPM? What is a spec file? build roots RPM macros The actual building of the RPMs and quality checking them after we've built them So for this guy, we're going to be following a guide today Roughly, it's a part of it's in the red hat developer Get GitHub group. We have a link down there at the bottom If someone could also put that link into the chat that would be great. We don't need to have it open today directly this link This is just the GitHub source for the guide will be we have a link on an upcoming slide That's an actual PDF that will follow along for this talk We're not going to cover every bit and piece of it the guide covers a lot more things than what we have time to get into today It has an appendix full of advanced topics that are really good If you notice something in there, that's not right. It is a it's a living document, you know Pull requests are not just welcome but encouraged So we'd love to see in fact all reference a pull request that I have open right now that I was hoping we'll get merged by Today so I didn't have to even bring it up, but it did not so We'll talk about that in the prerequisites a little bit And here is that said PDF that will follow. We've got a QR code and a and a tiny URL or you can or someone can drop the link in the chat That's the actual PDF that will be referencing with the page numbers as we go through the workshop today Once I see that in the chat I'll go to the next slide. Perfect That is our link So a prerequisites today I got it onto the schedule but not at the very beginning when the schedule was published I was hoping everyone had a chance to see it To do the labs today you'll need access to an rpm system ideally fedora centOS rel or a rel derivative to do the labs This can be either the system you're you're connecting from that you're working off of Or if you have a remote system like a cloud server that you have ssh access into None of the labs we're doing need anything graphical. They're all terminal based so Yes, carless. The PDF is made from ASCII doc Um I did not set that up. I that uh, the people that originally wrote this guide did and it is it is pretty nice So if anyone doesn't have access to uh, one of these types of systems Theoretically, I guess it could work from something like opensus, but I've never tried it and I wouldn't There may not be some of the tools that don't work the same way. So Ideally you have access to one of these types of systems If you don't you're still welcome to follow along and ask questions But uh, I don't you wouldn't be able to really use the you do the labs directly So Here's our first lab Related to the prerequisites. We're going to get a few packages installed to start off with On page three of that PDF that we linked earlier. There's a list. There's a dnf or yum command to run one special note If you're actually doing the lab on a el 8 system enterprise Linux 8 Which would be rel 8 or Alma 8 or sent to a stream 8 things like that There's a separate step that you need to do to set the default unversion python That is in that pull request on github if nobody has an el 8 system We don't need to worry about that or talk about it. This should be a really quick lab less than five minutes. Ideally I will just in order to avoid a bunch of plus one messages I'm going to say I'm done in the chat and just give that a thumbs up Like it or whatever Whenever you've completed the first lab and once that looks close to the number of people viewing then we'll move on or ask If there's any questions of anyone's stuff for some reason This lab's really short. So we're after we're done with it. We'll keep going but Some of the the longer labs in the future will we'll do breaks in conjunction with the labs So that way if someone needs a few more minutes to get finished They can or if people want to take you know a bathroom break or refill their beverage of choice. They can do that as well Also, if anyone's having any trouble with any of the setup so far either get into the pdf or anything like that Feel free to drop a question in the chat Troy and nills are here primarily to field questions while i'm talking and presenting So that will uh That is a resource available to everyone. Do not do not worry that you're interrupting me I'll still probably glance over at the questions, but I'll do my best to ignore them while i'm discussing the topic So the presentation flows smoothly All right, so we're going to get into our general topics and background So my first thing uh, what is source code? That this is going to be talking about things on page seven of the uh of the pdf I won't keep calling out the page numbers for the rest of this but just know that Uh, you won't really need that for the presentation right now But if you wanted to go back later and compare the presentation slides to the pdf Packaging guide you can use those page numbers for reference So source code is a human friendly representation of instructions for the computer The bashell is a good example. It's an interactive shell It's what most people use when they open up a terminal emulator It happens to be scriptable like most shells And its scripting language is in fact a programming language And so it's the instructions you type into a bashell could be considered source code That's one simple example James the page number where we should be done through uh, don't feel like you need to read through the pdf guide all the steps Corresponding to this we're just going to do a high level overview with these with these slides and The pdf guide is going to be a reference a reference for future if you wanted to dive deeper into any of the particular topics How programs are made? There's a process called compilation Which is where we take source code and we translate it into a representation the computer understands Either a native computer language or otherwise There are different types of execution for programs We have what's called natively compiled and interpreted which is divided into two other categories byte compiled and raw interpreted We'll talk about those categories a little more on the next slide So you'll see some logos for popular programming languages. They're on the right side And those correspond to each of these categories that I just mentioned So for native native builds natively compiled software, you know, we have things like c c plus plus go and rust There's plenty more I had limited space on the slide for logos So those the the top four that I thought were really popular or at least very common in our circles The code the source code is translated And or compiled directly into machine code And then that is executed directly on the system the resulting file that you get interpreted or byte compiled interpreted Will be things like python or java or ruby A lot of people might mistake those for raw interpreted languages like bash because they just see say a python script and they run it and they just see that it runs That but in behind the scenes the interpreter for that the python binary It is translating that source code into an intermediate representation known as bytecode And it still needs an interpreter to execute You don't just run say a python script or a ruby file by itself Then we still and then we also have raw interpreted as a type That would be things like how we mentioned bash how you can just how it's scriptable That's that source code is directly interpreted and executed by its runtime Right as it's parsed You can even get create interesting side effects and behaviors like if you edited a bash script Midway while it was running And timed it right you could do all kinds of weird things to the computer insect faults and things like that Where the underlying code changed and that's how you know that it is being executed line one line at a time essentially And that also needs an interpreter to execute you can't just run a bash script just the file by itself without bash the command quick breather here Does anyone need have any questions related to the types of builds Or the types of software and source code what we've gone over so far I'll pause like this periodically through the through the presentation just to give people a chance to ask questions But if if you have a question don't feel like you need to wait for one of those pauses You can just go ahead and drop it in the chat We'll handle it as we go Thank you Nils for getting those the pdf of the slides up in case anyone needs them That's why i'm here for Support team I love it. All right, so now we're going to talk about building software So compiling the software it's a process. Is this a duplicate slide? Do we already cover this? No, it's just very similar. Sorry The the software compilation process is usually called building. We have built things called build systems or build tools Um that would be things like make or mason that help to automate that process natively compiled source code That must be built in order to execute it as it doesn't have an interpreter to execute otherwise Uh, whenever you do a native Compiled build the binary that you get to run it's it's going to be hardware architecture specific That means like if you if you were to build build a program on Your x86 64 laptop and then you copied that file over to say An ar 64 raspberry pi you wouldn't be able to run that binary because the architecture doesn't match for interpreted source code It still needs to be built Whenever it's byte compiled not whenever it's raw interpreted A lot of a lot of these by compiled languages will automatically Do the byte compiling for you like I mentioned how with a python script Some people don't realize that it is byte compiled because they just have an interpreter and a script and they run it And it's abstracted away from them But it is doing the byte compiling behind the scenes and we'll look at that a little bit closer in the python lab We have later on Some of the other byte compiled languages like java you actually have to build that byte code by hand directly It's not done automatically for you So patching software A software patch is a lot like If you think of a cloth patch that's used to repair clothing like a shirt or blanket A software patch kind of works like that It's something that we'll apply on top of other source code either to repair a defect or add a new function For something that was missing previously This comes into play a lot for rpm packages because we'll find ourselves in a situation where We want to fix something or add some kind of functionality For the version that we're packaging Without necessarily updating the package to the next version from the upstream project That may be a case of like Backporting a security fix or a feature It may be a case where the upstream has Already made that change but not it's not in a released version yet Or they may have made the change and released it in an incompatible version Say we we want to add a feature or fix to version one of the software that we have packaged Upstream has come out with version two that has that fixer feature And from the upstream project you don't have a way to get that fixer feature Without doing the incompatible upgrade to version two Patching is a way we could cherry pick out that one change and put apply it to version one of the software that we're putting in package We in that patch mechanism. We still keep the original source code pristine We'll have a tar ball Containing the source code for the main primary version from the upstream that we're packaging We keep that around and keep that separate from patches for several reasons. It helps with auto-debulability Reproducibility and debugging especially debugging is a real big one If you run into some kind of problem being able to turn individual patches on and off and redo the builds Come in really handy All right, I guess not we'll go ahead and move on If anyone has any questions later just feel free to drop them in the chat We can get an answer there or go back a slide if we need to installing artifacts so on a Linux system, we'll want to install artifacts once they've been built That involves putting the file in the correct place According to and correct being defined by what we call the file system hierarchy standard the fhs That defines directory structure and it also defines the context for arbitrary files based on what location they're in You can just look at a file and know that okay. Well if this is in Slash etsy that's a configuration file or if it's in user bin that is a command that I want to run and things like that It's not just logical context of what you're looking at But also things like sc linux context come into play if you're familiar with that from other things in work and whatnot We have a tool available to us called the install command It's part it comes from gnu core utils And it we use that pretty heavily with rpm packaging It copies files into their destination But it has a few benefits over the copy command In that it can also in one command you can set Ownership and permissions and things like that right away So it's it's usually the preferred way to get files into location we need for doing the package build So that concludes our general topics for our overview If we have any questions about those go ahead and drop them in the chat The next slide next thing we're going to get into is the actual rpm packaging guide itself I'll pause for a minute just to give anyone a chance to ask questions if they have them And what one aside to the install command is very very often that is wrapped in the make file or Uh Similar construct way then can type make install It will run the install commands for you and install the the various files of the program in the right place Absolutely That is a great thing to point out And i'm not seeing any questions come through so we'll go ahead and move along with rpm packaging guide So what is an rpm package? It is a file that contains other files and metadata about them More specifically drilling down There's a lead 96 bytes of magic that we don't actually use anymore. It's just kept there for backwards compatibility There's a typically a digital signature. That's optional Then there's also the rpm header that contains all the metadata about the package Which we'll get into that metadata on future slides as well And then it also contains a cpio archive Which is an older format of an archive. It's not really i've never used it seen it used for anything else but rpms But it contains the actual payload which are the files that we're going to be installing on the target system when you install that rpm So one of the during the prerequisite lab We installed a thing called rpm dev tools that includes a command called rpm dev dash setup tree If you run that on your system, uh, don't get ahead That's actually the next lab, which is another one of the easy labs when you run that command on your system It creates these directories It creates an rpm build directory and within there build rpm sources specs and source rpm And that is exactly what we're going to do in this next less than five minute lab On your on your target system. Just run that command. It should be available if it's not available Then make sure you have the rpm dev tools package installed and that'll give you that command The uh, the page numbers in this in the slides they reference the page numbers that are printed in the Uh printed on the pdf page is not the actual Number from the application that you're viewing it with That's a good thing to point out Imagine imagine it's So Robbie asked an interesting question here Is it not recommended to build rpms as a separate user anymore? I've always had a special non-privileged user for building We're going through the this uh with the base. We're going to be doing our labs with just the basic rpm build command I wouldn't even recommend using that You know quote unquote in production later on after this We'll talk about mock later on towards the end of the end of the workshop Which mock has its own uh, its own special set up for doing the separation of concerns there I don't think there's a problem with doing rpm builds as your normal user You definitely don't want to do them as the root user Doing rpm builds as the root is your if you were to have it's possible to have errors in your Spec file that can can affect and damage your actual system that you're running things on Doing it as a non-root user even the primary user you run as limits that potential impact Running it inside popman is also another option Yeah, the main thing it doesn't have to be a separate user. It's just as long as you're not root Unless your primary user is rude if you just log in directly as I've seen people do that We've got seven thumbs up on the i'm done I think we had eight thumbs up on the on the first pre-requisite lab So I think that's the most everyone does anyone need more time to run this one command and get their uh directory structure set up So I think we're About good and we'll move on with the uh With this So we're going to talk about spec files What is a spec file it is a recipe or set of instructions to tell rpm build How to actually build the rpm by the way in case we didn't mention this yet rpm build is an act is an actual command on your system That you can run that we're going to we're going to use later in labs That does the actual work of building the rpm The spec file is going to be composed of various sections and headings It's going to have metadata that this is where you specify the metadata that we mentioned gets embedded into the rpm later It's also going to have build instructions and a file manifest This is where in the metadata. This is also where we define The name the version and the release of the package This is a really common uh set of data together that we refer to as nvr That gets used as an identifier For a lot of things it gets used as the identifier for the build when you're doing it in a build system It's also used for comparing packages With the package manager on your system to determine what packages that are available are actual upgrades for what you have installed Here's a simple example that I got off of the fedora 36 system from the bash package You can kind of look at this and split it on the hyphens Split it on the hyphens from the right Into three groups so that way if you had you could it's still allowed to have hyphens in the package name But going from the right if you split that off At a hyphen you get 2.fc 36. That is the release of the package Then the 5 116 that is the version of the package and then everything else bash in this case is the name of the package The version is going to correspond to The upstream version from the software project bash in this case The release that is sort of like a different version number It's sort of like the version of the package as opposed to the version of the software that's being packaged Another way you might look at that is that is the this would be the second attempt at packaging version 5 116 from the upstream That's needed a lot of times if you need it if you need to make package changes or rebuild a package But not change the version So the the very first section of the spec file is what we call the preamble This is some of the stuff that I already mentioned we've got the name the version and release We also have a summary, which will be just a short summary of what the package contains We'll have the license field that lets us set the Set the to define the license of what of the software that's being packaged We're not defining that we're identifying it. What's defined upstream? We also set the the url for the like the software project or the software vendors website And we'll also list out our source and patch files The source is going to be The the source that most packages are going to have is going to be a path or url to an archive of the software source code We can also specify additional sources Things like if we have to carry our own configuration file or system to unit file Anything like that can be specific that's separate from the upstream project for whatever reason we can carry that as a source in our spec file Then we also have patches Any any any changes that we need to make To the software when while we're building it we can include those in this preamble section It's really common to see the source and patch directives numbered like source zero source one Patch zero patch one patch two But it's actually not required anymore So you could just do source repeatedly or patch repeatedly They're numbered internally and you can still reference them by numbers Because you don't have to actually specify the number yourself So we have a few more items that are common in the preamble A lot of these are optional You can play around with spec files and figure out how how many of these you can actually leave out entirely It gets to a point where it won't actually build but Most of these you'll have defined typically Build arch that's only going to that's really only used now nowadays when you have an architecture independent package What we call no arch That would be something like if you were packaging a bash script And you know that that is just a text file that gets run by the bash interpreter That's already on the system separate from your package That means the package you create isn't going to be limited to a specific architecture because it's just that bash script So you can create a no arch package that works everywhere We'll also be able to specify build requires here Those are going to be packages that must be installed on the system building the package That's different from requires which are packages that must be installed on the system where you're installing the package at We'll often refer to these as build time versus runtime requirements Another another two more directives around architectures are exclude arch and exclusive arch Those those are a lot less common You most of the time won't have to deal with those The architecture for your package is going to match the build system that you build it on Typically what we'll have In say fedora you'll build a package you won't specify anything about architecture And as long as there's not no arch it will do multiple builds for every supported architecture And you'll get different packages that correspond to each architecture With exclude arch and exclusive arch you can do special things where you either Allow list or deny list basically the architectures if you needed to have that level of control Um, some examples might be If there's upstream software that flat out doesn't work or isn't supported on a specific specific architecture You can exclude it Sometimes you may want to just skip the test suite on a particular architecture Which is separate from these directive these these directives define where you can actually build it But we'll get into the conditionals later. I'm getting ahead of myself Does anyone have any questions around preamble items that we've gone over here Jonathan's asking us to reiterate the source and patch part Uh the patch number part so And a lot of the templates and guides and most of the documentation you see online You'll see things listed out like source zero as the url Archive of Where it's a source one for a system, you know, and it's two You can just leave the numbers Source repeatedly and They're numbered eternally in the order that they're listed in the spec file And so you can still reference them by number, but you can just leave the numbers off. They're optional on this part Uh, sounds cutting off again. This internet out here is really bad. I apologize for that. Yeah, it's Going bad again Yeah, sorry about the the bad internet connection I have I Actually agreed to do some dog sitting out in the country before this before the conference was scheduled and so I'm just trying to make do is uh as best I can apologies If I end up dropping off entirely, uh, troi nils, we'll just have to pick up the slack and take over for me But hopefully we can avoid that I just wanted to point out since we were asked about source and patch Um, I think we come to this later on but those go into the the sources Directory there's a spec directory. There's a build and whatever Both the source and the patch go into the sources directory But I think we correct We we will explore that in more detail in uh in the labs as well But that is a great thing to point out I guess we could very briefly Look at that uh that directory tree So the when you do the rpm dev setup tree command you get your build directory That's where sources are unpacked and things are intermediate files are compiled and things like that The rpm's directory is where the final rpm's You know instead of going alphabetical or let me go in the order of the build kind of Your specs directory is going to have your spec file Or spec files your sources directory is going to have all those sources and patch files like troi mentioned The when you when you do your build, uh, we're going to do it in two stages You can do it in one step, but uh for the lab purposes We're going to do them separate when you build you build your source rpm first That's going to be a special kind of rpm. You don't install those on your system That is a it's an rpm that that contains the spec file all the sources and all the patches and can be reused to build a What we call a binary rpm the traditional usage of the word rpm for different systems After your build with rpm build those are going to go go into the rpm's directory The build directory is that intermediate step. It's where the software sources source tar balls are Extracted and where all the build steps are take take place as the working directory Hey, we've got a celebrity here adam's here. He uh, this is actually his original source material that I have shamelessly stolen So thanks a bunch adam Let's get back to where we were All right, so that was our preamble section Um, that was a great point. Troy about the sources and patches being in the same sources directory So if there's no other I don't know if there was any other uh questions I missed in the chat Um, let me scan that real quick adam was just saying that slice this was was free anyway, so you can't steal what's free Excellent point. I like that phrase Omar said he asked one and I missed it Yeah, the the one like that's uh, oh, yes epa trance material Yes, we'll touch on epa briefly in the uh towards the end. Um Hopefully you never have to use Don't use epa No, unless you have to use it unless you have to you will know when you have to We'll we'll get there. We'll get there All right, so I mentioned how they we the spec file is kind of divided into the preamble part There's also the body of the spec file. That's where we're going to have a description Uh, it's a little bit more verbose version of the summary. Uh, well, it can be multi paragraph multi line uh, and Sometimes some packages don't really have much more to say about them and people will just shortcut that and Take the uh, use the summary as the description as well Just add a period to the end to make a sentence or a sentence fragment and that's fine too But it's it's a place where you can go into a greater detail on the um On the software itself of what you're packaging We also have the prep section That is going to be where we prepare the source code for being built that involves unpacking Any of the the archive or the archives if there's more than one that's where we're going to apply our patches at Um, sometimes we need to delete files or make small modifications to files Without without using a patch, but just you know say you could use like said on a file in the prep to make a small change to it Those are really common things Then we have our build section That's where the commands for actually building the software into machine code for compiled languages or bytecode For the byte compiled interpreted languages. That's where that takes place We also have our install section and that's going to be where We take the files that we've built And we place them into the appropriate file system locations But it's relative to our build root directory Uh, the build root is kind of where we stage all of the files Uh for the final rpm We wouldn't do that on our actual system because remember we say we're not going to be building as root Uh, and also you don't want to put them on your system You want to put them into a separate directory that gets Compressed together into that cpio archive and then gets put into the final rpm Another section we'll have is the check section This one is optional. Uh, some of the other ones are optional as well depending on what you're doing Check is one that very often becomes optional A lot of upstream software doesn't have test suites And that's what this section is for it's where you want to run Run commands to test the software such as the upstream test suite Some software doesn't have a test suite. Sometimes it doesn't work correctly or it has known failures I would recommend when you can escaping individual test rather than not running the test at all But that's that gets into best practices later on for now We're just going to cover that that is what that is where the section where you're going to run the test If you're doing that We also have a file section That is where we're going to list out the files kind of like a manifest of what's going to be installed on the target system So that way you can just look at a spec file and you'll know what files would be Would be contained if everything builds correctly and the spec files written correctly You'll know what's going to be installed in that final rpm That's a good way to keep track of changes in the upstream software if they say Say they started to include a new file out of their build And it conflicts with what you have in the file manifest That's a way to notify you the build would start failing Because you would have an additional file in your build route that you don't have listed in files And it's a good way to just make sure that the package has the files that you're expecting to be there The next section is the changelog and that's usually the very last section of the spec file That's where we're going to record changes that have happened to the package between different versions and releases One thing I see new packages get confused with here often times is The difference between this and the upstream changelog You don't actually have to You don't have to duplicate the entire upstream changelog in the rpm changelog and most packages don't It is a useful place to talk about packaging changes Upstream changes oftentimes just get summarized as you know latest upstream release or latest upstream version But it is also it is sometimes You'll see people call out specific upstream changes That are worth that are notable in the changelog The real common example is cve fixes those will often get put into a an rpm spec file changelog Just so that way people know which version of the package they need to need to upgrade to in order to resolve that relevant cve rpm macros These are going these are variables that are used for text substitution um There are more advanced macros that use a programming language called lua But at the end of the day there's their end goal is still to substitute text in into somewhere in the spec file So it may be some it may be programmatically generated text or it may just be a direct text You know this is this this key is this value They can be conditional which means that they'll only be expanded if the macro Only expand the macro if some other condition is true They can you can also explore these outside of an rpm build actual rpm build context On your on your test system. We don't have a lab for this, but if you wanted to play with this on your own You can run the rpm dash dash eval command That will and then that takes arguments and you can ask it what a specific macro is defined as To evaluate that macro You can also combine that with the rpm dash dash define flag And that lets you define a macro either for its own sake or to influence other makers that are being evaluated When you're doing it ad hoc on the command line That defining it doesn't do anything other than something that you're evaluating in the same command Then there's a really useful command called rpm dash dash show rc And that'll print out all the makers you have defined on your system It's quite a lot. You'll probably want to pipe that into a grep or a less command in order to look for exactly Either just page through it or to look for a very specific specific macro That comes in handy a lot of times if you know there's a macro that you're looking for But you don't remember exactly the syntax for it where you thought you're remembered and you're getting errors that it's not defined You can run that to to see exactly how it's defined You might have forgotten an underscore or something or spelt something wrong and that comes in very handy So here's some of the common macros that we'll use They're very common for file system locations For example underscore bender Evaluates to user bin There's underscore libexector that'll evaluate to user libexec There's a whole lot of file system location prefixes like this that we'll use Prefixes another one that evaluates to slash user The uh, and I don't think I cover this yet, but macros that's kind of the syntax right there It'll be a percent and typically a curly bracket around the actual name of the macro There are there scenarios where you wouldn't you wouldn't need to use the curly brackets They're there kind of they work kind of like curly brackets in bash if you're familiar with that where You could do dollar foo in bash and that works as a variable name to expand the variable But if you needed to Separate that from other other strings you could use the curly brackets to say this is where the the macro name ends A lot of people will just use those curly brackets regardless just out of uh, just for consistency and that's that's fine and understood as well Macros are often a lot of times used for distribution properties as well On a centOS stream system, for example, you could expand centOS and you'll get the value of 9 on centOS stream 9 You can use el 9 on the same system and you'll get a value of 1 If you were on a not on an el 9 system that would not evaluate at all Then The dist macro Before I mentioned the nvr of a bash package how it had fc 36 As part of that release string um That's what's called a disc tag and that's typically used to to provide an indication of what release of a distribution that uh The release of that package was built for That's a very common way to help avoid installing Say a rel 8 package on rel 9 or vice versa or any other combination There are There's several there's some situations where you could have a dist independent package But they're rare and you should kind of avoid them anyways You really want to use packages that were built and targeted specifically for the release that you're using And yes, I'm seeing some stuff fly by in the chat about other distributions like you know percent fedora Percent rel those are those are real common macros defined too. I just I just grabbed three off of a Off of a centOS 9 system that I was looking at Yeah, when these are these are ones that you can also explore with that rpm show rc command One word about those that that are may not be defined like what you mentioned before the continual macro expansion That's when this comes in very handy because um If you use a macro that isn't defined It will just expand to the macro name which is a bit like unexpected Yes, and if that's an in in a context like build where it's trying to execute You'll get a strange. Uh, you'll get a very strange error message about Foreground control. I believe which there's something in bash with the percent sign where it tries to do I think it's is it at jobs or something similar Um So if if your build log has it trying to execute, you know, say an el9 command You know that something's funky and it's not being defined, right the way to handle that there's A syntax you can use where you put a question mark before the macro name and then it will That will expand it if it's defined and make it an empty string if it's not defined That avoids dumping the macro syntax into Into the build context wherever you're at Just like nil's drop down there in the in the chat There's other other things you can do there too with say a colon after the the macro name Which will let you do something else if it's defined And you can even use there's an exclamation point You can put next to the question mark to set to say Do insert this string if it if it's not defined Lots of different controls and things you can do there. So working with spec files A big part of packaging software into rpms is editing the spec file itself. That's where most of our work takes place Most packages we don't create spec files completely from scratch We're going to use built-in templates from our text editor Or there's a command that came with the rpm dev tools package called rpm dev new spec That'll create kind of a skeleton of a spec file with your preamble and body sections It won't be valid from the get go. There'll be empty sections that need to be filled out There may be some lines that you don't need For example, it'll have a build requires line that's empty If you don't actually have any build requires for your package Then you can just remove that line entirely But that gives you the basic template of what you need Uh to create everything There's nothing wrong with writing the spec file file from scratch if that's what you want to do There's just there's faster ways to do it So We are going to get into the next lab. Uh, this is going to be about a five minute lab And it's going to be working with spec files On page 25 of the guide not of the pd slide pdf But of the actual packaging guide pdf And the page number that's printed in the document not the one from your pdf reader There is uh, there's instructions there for tar balls and patch files Go ahead and download those on your working system And place place them into the rp and build sources directory We're going to be this is set up for Three example labs we're doing Or three example programs we're going to build as labs later on So you're going to have sources for A bellow a pillow and a cello package. Those are all hello world examples written in Different programming languages, which you can probably guess by the first letter that they're that they start with In this lab, we're going to use that rpm dev new spec command With all the instructions the actual commands are listed there in the guide And so in addition to getting our sources and patch files We're also going to run rpm dev new spec new spec to create the template and start filling in values For those packet for those packages that we build later We're going to spend about five minutes on that and uh, I've been talking for a little while now So I'm going to all we're also going to do a break Uh, let's say a 10 minute break after Um after the lab's done And then we'll move on from there Just so I don't lose track of time I'll set a timer For 15 minutes. So a five minute lab and a 10 minute break And then we'll get back started Some of us will still I'm just going to go to the bathroom and come back I don't know if Troy and those are going to be here the whole time But uh, someone will be here to answer questions if you want to drop anything in the chat And have fun See jonathan made a really good point. We were talking about this the other day and I meant to mention it in this lab So I'm glad he brought it up in the comments Um The rpm dev new spec command it has a dash t option that lets you specify a template A lot of those are kind of old and not really well maintained And don't necessarily comply with all modern package guidelines. So They're there you can use them But don't just assume that everything in that template is going to be correct for what you're doing or modern Look at that. That's one One thing You'll have a couple they are you'll have regularly if you maintain a package in in federal or apple for a while that you Got to be updating your spec files to the respective current standards Like if new macros are introduced or all macros are deprecated and removed eventually one really good thing with rpm packaging is that it's been around for a long it's been around for a long time and A lot of things will keep working for a long time in the future until they're eventually deprecated Just because they keep working doesn't necessarily mean that they're still the best practice The best practices do kind of evolve and change over time rpm gets new features package guidelines change And so some of those things you just kind of have to You know participate and be active in the in the community to be aware of those You know bookmark and reread package guidelines when you're trying to do something if you don't remember A particular syntax of a spec file particular section of a spec file go check the guidelines for it and see if See if anything's changed Really not a great way to be aware of these maybe pay attention to Release notes for rpm upstream if you're really into it So a couple of things other things here in the comments there was a A Yes, there's a lot of older guides online And yes, it would be nice to have this give this pdf to our past selves. I agree Link I ask a question about what the dash q option is in setup that just stands for quiet It's kind of the default and there's a there's a newer macro I believe the rpm dev new spec it gives you the there's an example of one of those things best practices that changes over time It'll do a percent setup And then a dash q for quiet leaving off the q doesn't do anything. It's fine. It just prints a few more things in the log But a more modern example is to use percent auto setup and that'll have the benefit of automatically applying any patch files you have specified in the package And it does a setup dash q quiet by default without you having to specify the flag manually For this lab. It's fine. You don't have to do the auto setup You can just leave it as setup dash q or if you want to you could do auto setup But you'll have to uh Adapt to the instructions around patch. There's one one one of the labs has a patch file and you'll have to adapt Uh to that and make sure you know what you're doing I would I would recommend doing just sticking with the setup dash q for now I mentioned the packaging guidelines. I'll drop a link for those here in the chat I'm curious if auto setup is mentioned in there and it is I'll explain it in greater detail So mark asked why not auto setup? Auto setup is great if you know you're going to apply all your patches If you need to have any kind of granularity around which patches are applied Say you only want to apply a particular patch on a particular architecture Or you're you sharing a spec file between fedora and rel And you only want to apply the patch on rel because of an older library, but not on fedora That's a that is a use case where you'd want to use setup and then put conditionals around the patch application lines For this uh for these labs in particular, uh, we're going to stick with setup instead of auto setup just because that's what the pdf is written for and Um, and so we're just going to stick with that for consistency If you really feel strongly that the that the guide should focus on auto setup first, which I think there's a good case that it should Uh, that is a that is an example of something that could be contributed to the upstream ascii doc sources to get that updated Nils, I don't know if you were listening at the time, but Troy said he was going to Go take a break also and then try and come back and join from chrome and see if it works better for him No, I didn't uh, didn't get that He'll join us again shortly. I'm sure I'm back and both of you sound much better now Yeah, I thought it was the all everybody else's problem Had a strange issue in the fedora museum during one of the socials yesterday where I guess my slow connection was doing some kind of weird side effects in jitsi where To me it looked like my camera was on and everyone else's camera was off But when I mentioned it everyone people told me no everyone else's camera is on your camera is off Which was quite weird i'm not even sure how someone would go about troubleshooting that It has been uh, my timer has one more minute of it Of the break and we'll get going on I did some quick math for the rest of the uh, the rest of the slides in the labs And it looks like I've talked a little bit too long on uh the introduction stuff So to uh, I did this the last time I gave this presentation to two hours is really cutting it kind of close You got to really be on top of it to get get through all the material in two hours with breaks The alternative will be two hours straight through with no breaks, but that's no fun So honestly, I think what we're going to do is uh, we'll skip one of the three hello world examples Um, they're all useful, but For the purposes of this we can probably cut one of them out If you still want to do it later on on your own You'll have all the material you'll have these this presentation slides as well as the packaging guide You can follow through the packaging guide front to back on your own with and it'll have more material and all of the uh, all of the labs I don't want to call it homework But if uh, if you want to call it homework, that's fine I mean, we're all at home right now as a virtual conference. I guess So it's all homework, but We'll probably just do the bash and see example programs bellow and cello We'll skip the pillow one for now I think that one takes just a little bit longer because you're having to manually deal with the bytecode Which is a very interesting thing to do if uh, if these type of labs Tickle your fancy if they if they catch your interest. I would recommend doing it again on your own I'm not sure if we have anything scheduled directly after us or another break on the schedule And I think we're the last thing in of the day Oh There there's a there's another like, uh, when our two hours are over. There's a break of half an hour or something and there's another Short block of Social hour Well social half hour really well, I don't want to deprive any one of the social stuff So rather than go Half an hour of wiggle room Okay. Oh, okay. So we're already that late. Okay. Um, yeah, mr. Time. We'll go we'll go ahead and cut out the uh I'll tell you when we get to the next uh, when we get to the pillow level skip over that one and just do bellow and cello So So our time our lab and uh break time is over. I should just done a shorter break Looks like we've got about six thumbs up on the uh on diego's. I'm done comment So I'll assume that pretty much everyone's done with this lab and we'll move on And here's our uh, here's our first program This is a hello world program written in bash We're going to write a spec file for it. We already created We're going to finish filling out the spec file for it We started the spec file with rpm dev new spec in the previous lab and then uh, we'll spend Um, let's uh, let's check in at 10 minutes and see if we can short uh, shortcut this one a little bit But it'll be pages 26 through 31 in the in the pdf guide You don't need to go past page 31. That's where the uh, the bellow part ends and the next example begins But all the commands are listed there in that pdf first to just go through and Uh We'll work on that for the next Let's say 10 minutes and then check and see if we need five more after that And just like before I'll even I'm done comment. Just like that whenever you've you've uh, completed this part of the lab And of course if anything in the guide doesn't make sense or you have any questions at all Feel free to drop them in the chat Carl, I just want to be clarifying because some of the letters sound familiar We are doing the bellow with a b for bash And what was the other one we're doing? Uh, cello, which is a c for c okay So, yeah, the uh, this is the bellow one The next one is Yeah, the pillow is the one we're going to skip Whoops no go backwards. Yeah, so we just have these three after a will we'll do the first and the third one here But for now everyone just hand uh go ahead and go through pages 26 to 31 Follow those instructions and look at pdf guide and then that'll be our bellow spec file Carlos went ahead and went ahead at page 43. So the uh Yes, I guess if you uh, if you want to squeeze in the bellow spec file And you can do it between uh between the this one and the other lab then uh, there's nothing stopping you For some people this stuff will make perfect sense and they'll just Bose right through it for other people to mind a little more time or have questions and that's totally fine too Everything's new new to you until it isn't anymore I guess I will I'm going to go ahead and describe the uh the next lab Just for anyone that wants to get started with it. If anyone still needs to work with the bellow spec file a little bit longer Um Go ahead and do that. Oh, we had a quick question here Regarding the line install dash him and then the rest of those arguments Um Is the extra slash needed? Okay, I see what you're saying robby and you are correct that um Build the build root uh macro That doesn't end in a slash But the bender macro which evaluates to user bin does start with a slash So the slash between build root and bender Is not needed So you could actually skip that one the other ones you do need the slash so like between bender and name Name doesn't have a slash in it and bender doesn't end in a slash. So you do need that one um Really, it's just Just the just before the bender. You don't need that slash. It doesn't hurt anything to have it in there so But that actually would be a good little typo fixed for the uh for the peckney guide We can uh I'll grab that github link again Just like in just like in the shell multiple uh Multiple consecutive slashes uh get switched down to a single one so it doesn't actually hurt anything But I do agree that would be a good little thing to clean up just from uh From an ocd perspective All right, so this was the uh, that was the bellow lab. Let's talk about the uh cello lab Remember, we're gonna we're gonna skip over the We're gonna skip over the pillow the python example for now. Just be just do the time Uh, and we're just gonna do the the cello spec file So in this example, we'll write the spec file for the cello program. It's an example. Hello world program written in c And we'll we'll spend uh, let's do the same thing Work on this straight for about 10 minutes and then see where everyone's at if people want a few more minutes And if anyone needs to finish up The bellow spec file or has any questions or anything they're stuck on there Feel free to shout out questions in the chat and we'll help you get caught up and the page numbers here is pages 39 through 44 So try to try to limit yourself to just those pages as you're going through We'll after this we'll get into the lab of actually building the rpms The real fun stuff where everything hits the fan And remember please if anyone has questions at all, uh, there's no no dumb question except the one you don't ask Um, I know I've done this workshop one time live and It was really nice because people could raise their hand have one of the trainers come over and just Ask them one-on-one directly not in front of you know in front of the whole class per se Uh, we don't really have that luxury with with hopping but So like I said, don't don't be afraid of asking questions ask away If anything's not clear in the uh In in the guide We'd definitely like to have feedback on it um I kind of jokingly kept pushing people to To submit, you know pull requests to the to the ASCII doc sources for the packaging guide pdf But I am trying to make note of those things as we go And if that drops off someone's plate if they're not able to do that Those are changes that I'd like to eventually make sure to get in there because I I agree with what I've heard so far yeah Speaking of that, I guess somebody did ask a question and I I groaned at the epoch Um Let's let's just do a quick summary of what epoch is sure people are doing That's not frustrating with the question frustration with the question. That's frustration with epochs in general Yeah Um an epoch so we have our version That's the version of your software. We have the release which is usually we consider the release of the rpm So as you change spec files or add patches you do the releases an epoch is sort of Uh a trump card And I'm not meaning donald trump. I wish his name wasn't that but um, I don't know what what else to call it And that might be an american term, but basically super version Super oh that's much better. Thank you Um, I know what you mean by trump card with that like I've played spades. I know what you mean, but Yeah, it's it's It's above everything and you don't use it unless things happen What the most common use for an epoch is Uh when a package version goes from like a year To an to a number like let's say it's packaged was 2022.4 and now they're saying hey we want to go to Six You know drop the year Uh, well six is much lower than to 2022 So you would switch the version to six and then add by default epoch has a version of zero So you would add an epoch one And then that six would technically because the epoch is one So version six epoch of one is higher than a version of 2022 So that's when you would use the epoch Um, the problem is every once in a while you find somebody using an epoch incorrectly and For the most part if you can get away with not using the epoch Don't use it because you can never take them out To rephrase it another way it is a tool a very big hammer if you will For forcing the sorting order for versions Yep And like like Troy mentioned you you generally want to avoid that you once you put it in there You can never upgrade it or you can never remove it without ruining your upgrade path so There have been there are some packages in fedora that have ridiculously high epoch numbers things that should have never have been reached Because of various mistakes and misuse of epoch so In general you want to avoid it. Hopefully you never have to interact with it but if you get in a situation where The upstream the upstream versions change in a way that RPM doesn't understand it as a a proper upgrade if you need to force it. That's what epoch is for Oh, is that one of the high ones and you'll epoch bind has an epoch of 32 apparently Which is It should never happen. Yeah, thankfully epoch is an integer number so we can go a little higher than that Yes I think pearl has a really high epoch as well Or I may be thinking of a pearl packet a pearl component that was like that All right, we've been going for a little over 10 minutes for this for this lab Um, I think I dropped an undone comment earlier. Let's see how many thumbs up we have on that Or did I not put one in there? I'll just go ahead and do another one there at the at the bottom And if you're already done with this cello lab just give that a like Once you get a good number of a good number on there, we'll go ahead and go forward And actually start building some stuff We're up to five. I think that's the majority of people that have been uh Thumbs up in these does anyone specifically need a few more minutes to wrap up this or the or the bellow spec file lab Remember, we're skipping the pillow one the python example If nobody mentions anything in the chat, I'll assume that we've we're down to five people actively following along with the lab And that's okay. Uh, we'll just focus on getting those people those five people a good experience and uh move on to the next lab so building rpms Everything we've done up to now. We're just preparing ourselves to run the actual rpm build command Uh, we've covered how to build software from source code how arbitrary artifacts are built from source code Um, how they're how those artifacts built are installed Uh, and we've talked about preparing our pmb build environment And how to instruct rpm what to do via the spec file We are going to next we're going to use rp Uh, we're going to use rpm build to build source rpms as well as binary rpms I touched on that briefly before but the source rpms That is a it's a similar, uh file structure, but instead of It has all of your sources and patch files as well as the spec file And it's a way where you can ship around a unit that has everything you need to build installable rpms Alrighty, so we're uh We talked about we're going to use rpm build to build our source rpms as well as our binary rpms We're also going to explore some aspects of rpm build that might be a little surprising So here's our next lab And just like before we said this this actually should be allocated for 15 minutes But let's go for 10 minutes and see how far people get In this lab. We're going to build binary source binary rpms and source rpms for Bellow and cello we're going to skip. Hello in this case And make sure you only run the rpm build commands as a non-root user Because errors in your spec file can have a negative effect on the system you're performing the build on We've got our page numbers there for the guide pages 44 through 47 Go ahead and run through those uh those steps in there And uh, feel free to drop your questions in the chat as they come up So we are just about out of time. So let's go ahead and move forward and finish up what we can We'll go a little over But that's okay Quality checking rpms Uh rpm lint is a tool is a linter tool For spec file source rpms and rpms It can report common packaging errors And fedora 35 has rpm lint version 2 Uh, if you're on a fedora 35 and up system, which most people I saw in the poll are using fedora So you better you should be on 35 or higher because 34 is end of life So that has rpm lint version 2 the output doesn't match the examples in the guide That's another example of something that would be a really welcome contribution To that guide an explanation of how Maybe updating the examples to match for fedora So that way people don't have unexpected uh output in those examples All right, let's go and go to the next slide troy And here's our last lab This is uh Just checking those spec files and source rpms and rpms that we built Checking them for quality with rpm build That's another thing that I'm looking to I'm hoping to improve on that. I haven't done yet Because I'd like to uh, I believe in the packaging guide. It uses the term sanity check which Uh, some people don't like that phrasing. There's um It's not the nicest phrasing if you consider some of the connotations around it So quality checking is just a more accurate term anyways Rather than sanity So that's something that I'm personally going to work on. Uh, at some point in the future getting a pull request together to Make that terminology more accurate to what we're doing So Carl your slide says rpm build but the Labs are rpm lint Oh, that's correct. That is a typo in the slides Very sorry about that. Oh, and Diego just pointed that out too I just wanted to double check to make sure I wasn't looking at the wrong lab No, you're correct. That is a typo And if you're looking at the live if you're looking at the live slides, I just fixed it But if you're looking at the pdf export, we'll need to export it one more time for that fix The only way I could get it to fit on the screen really good without taking my whole screen is I'm this is on a pdf Okay, that's fine. We'll uh, we'll fix that for the uh, we'll re export one more time before we submit it for the conference collection of slide decks Is fixed And in fact, I think we're we're technically out of time for the presentation the two hours So Let's go ahead and make the the quality checking the rpm lint stuff We'll leave that as an exercise to the reader homework if you will if you optional homework, you don't have to turn anything in Just like the pillow exercise if you wanted to take a look at that And learn a little bit more about python bytecode and how how you could handle that manually Um, it's a good example for learning learning about bytecode. It's not very useful in real world because We have makers that handle lots of the python bytecode specifics So you don't have to really think about it at that level It's more of a proof more of a learning example than anything But let's go ahead and go on to the advanced topics real quick. I'll just talk on those and wrap everything up So in the packaging guide pages 52 through 74 Uh Well, the entire packaging guide is going to be a resource for your packaging adventures after this workshop's over Um, those advanced topics are have additional things that you that you could learn about They go over things like mock Disgit defining your own macros epochs, which we talked a little bit about earlier Scriplets and conditionals In particular the one thing I do want to call out explicitly That you should definitely read up about that if this were a three-hour workshop I would have a lab a lab specifically on it Is mock which go ahead and go to the next slide Yes, I agree with neil that uh once you learn mock you'll never go back to plain rp and build um Troy told me the other day he actually run Charoots uses mock to open up the charoot and then runs rp and build manually in that But I don't know anyone else that runs rp and build manually directly or indirectly Um, I say indirectly everyone's doing it by running mock, but anyways mock itself um So there are a few drawbacks of using rp and build directly like we've been doing in these labs Your build requirements have to be installed on the system where you're running rp and build and The a build requirement that is already installed. It's really easy to forget to list that in your spec file Uh, you may you'll forget to list it, but your build succeeds And you don't document that build requirement because it's already installed and it's just kind of a mismatch there Additionally you can only build rpms that target the same operating system and release It's not 100 true. You could create packages that technically work But they are more working by accident rather than intention and they'll be in most situations You don't want to do that because the resulting packages won't install correctly Or they're installed and have problems So mock is a tool that actually solves all of these things for us It it uh, it'll set up your build requirements Into a charoute not on your main system Though and then you it'll build it'll actually run rp and build Inside of that charoute That lets you build rpms for Different operating systems and releases them the one that you're running on your system Additionally those charoots they're automatically created for you and removed so you don't even have to think about them You don't have to worry about say, you know a compiler that pulls in a lot of dependencies that you don't really want installed on your main system They'll get installed in that charoute and automatically cleaned up and they won't be polluting your list of system packages Modern modern versions of mock also are actually using containers not just uh plain charoots Um and that that has additional isolation benefits But uh for most of your packaging work, I highly recommend using mock rather than uh Then rpms build directly for all of these reasons listed here Mock is also used by fedoras koji build system and in the copper build system as well It's pretty well pretty well much standard across Uh at least across the red hat ecosystem. I'm sure neil will comment if uh if open soos is using it in their ecosystem as well Go ahead and go to the next slide trey I think that's it Yep, that's all folks. That is uh, that is our workshop for today Sorry, I didn't manage the time quite appropriately with uh to get to that that pello lab or to spend more time on the rpms lint But uh two hours is what we had a time limit for we also had a little bit of interruptions with my own My own uh connection issues and I again, I apologize for that But thanks for attending the workshop. I hope you got something out of it If you have any questions in the future, uh, all the presenters here, we're happy to happy to talk in public channels We had our contact info on the beginning slide, which will be in the uh The slide export that we do later We're in the common channels for fedora. There's uh fedora devil on irc There's apple on irc. There's also corresponding matrix channels and The fedora discussion uh discourse has also different uh, you start different topics in there if you have other packaging questions and of course The oldest the old old fallback of uh regular mailing lists if you want to ask questions there We welcome in welcome those questions and hope to help uh help you get get along in your packaging adventures in the future I guess now it's time to head over to the uh social in the fedora museum. So Y'all have a good one and thanks for coming to the workshop workshop Mike Bye everybody