 Hello everyone. Good afternoon Welcome to my talk Actually quite surprised to see so many people. Thank you for coming The title I wanted to make it a bit catchy Maybe overdramatized but still Let's begin. So some of you already know me But my name is Dominik Mirajewski. I go by the handle Rathen and I Have comic tag the direct comic access to around 100 packages in Fedora Being a proven package Let's me touch a bit more I'm also a proven. I'm also a sponsor Hold or used to hold some other Titles in Fedora over the years because yeah, I've been contributing to Fedora for quite some time I think it's around 13 years now Long time here and Intent to stay So today talking about What What you should not do in the packages and fortunately Fedora guidelines Prevent us from doing that or at least give us clear guidance But still Outside Fedora, there is a whole world of RPM packages that don't follow those rules and I will show you some examples of Things people do in those packages Make you cringe to say the least But first I'll give a brief Overview our best packaging practices as a reminder To reference When going over each case that they have So most of you At least those who are package maintainers should already know the best practices of packaging and of course Disapply not only to Fedora, but to software packaging in general In the package metadata we need to have explicit list of files that the package owns or Wants to touch because as we know RPM can be core RPM database can be great for the list and we can We can ask our PM and database what which package a given file belongs to and so that's very important for discoverability and maintainability of packages and and keeping track of what we have all on the machine on the file system The other important thing which is In some which is in my opinion it has in some in my opinion become a bit more difficult These days is the no band link Policy which we used to have in Fedora, which is now relaxed But it has it has its pros and cons as many of you know but in general We should avoid bundling more than one software project in in our package and try to make use of the libraries and other projects Available in the distribution or if if we are bringing an entire new stack of software Each each each component should be packaged separately so that it can be reused by other new consumers and Following that is the mandate to use shared libraries as much as possible Because as we all know a new detective security vulnerability in a library Updating just that one with a fixed version fixes all its consumers all in theory it should fix all its consumers unless Somebody someone built Against a static version that one is to be rebuilt of course some Software ecosystems like Go or I think Rust as well Not very familiar with either They basically mandate rendering or bundling and are maybe moving towards Shared libraries but not not so fast Everything needs to be rebuilt with with each. Oh, and I think OCaml as well needs to be Everything needs to be rebuilt when when there is a new compiler or there's a new Library version So another thing is well at least in Fedora or in RPM ecosystem We don't really support interactive installations that there is no guarantee that there will be a life user or even a terminal present when an RPM is being installed so that's another thing to avoid in RPM packages and For mostly for security reasons but We cannot assume that the user account under the under a package under which a package is built is in any way privileged so We we do installations in a Sub-directory with where we have access to right, but We don't touch the root file system during build Nor should we This is distribution specific, of course, but in general our PM package should integrate with the rest of the distribution well It should follow the defaults like for example We should follow Cryptographic policy that we have in for Dora The distributions have their own Standards policies or quirks so it depends from this true to this true But as a general rule, you know, we should integrate well, for example, oh Firewall D versus IP tables We have firewall derived as a default Right now. So if your package supports both Well, I mean you should support firewall D as well if possible and and that should be the default but I think with Boolean dependencies we can For example, if we put things in a sub package one for firewall D one for IP tables you could Get those installed automatically depending on what the user has installed Just a thought and of course the builds should be irreproducible. So each time you build from the same exact Source or Commit in this kit you should get the same package So like I think it's also yeah, it is codified in the guidelines that We should actually try to patch out Dates or timestamps from from generated from files that are generated at build time to help reproducible reproducibility Right. So let's move on to the interesting cases which I actually encountered in the wild Mostly during my job So This was actually well, I did edit the excerpts so that the guilty parties are not named but Yeah, this would be most mostly in the scriptlets so this ties very well with the recent proposal to What was it called to to make RPM packages? archives instead of What was the what was the phrase privilege escalation Bombs or something like that Because the script is run as route during installation actual RPM installation So you have to be careful So what what was done wrong in that in that excerpt user interaction, right as I mentioned as The best practices dictate RPM packages are non-interactive by design unlike DPKG for example, so we cannot assume there is a user or terminal and We should provide a Safe working default configuration if there is a need for one or if it really requires manual intervention by an admin or I don't know some configuration management tool then there should be a At least some documentation provided on What parameters need to be specified because they cannot be guessed or Detected Right, so Jamila Of course, you won't you won't see that in Fedora, although I think there's one Maybe couple of packages which tried to be distribution agnostic What's that? We don't do this in Fedora Outside people try to write packages write spec files for multiple Distributions and they end up with with constructs like this and Well, in my opinion, it's not not a good practice Because it well, it's of course nice to have everything in one place But Depending on which with distribution versions you want to support the scripts can get unwieldy and Actually harder to maintain So I think it's better to just have separate Spec files for for each distribution that you want to support Because it also can lead to having Files that I mean dropped on the file system From inside the script that and they will not be owned by rpm unless you take special care to do that using ghost Interactive in the in the in the file section. So yeah, it gets complicated. So it's better One spec file for one distribution unless it's you know, I mean even even between Fedora and entrell There are some differences for many packages. You can have the same spec file and built everything from from the master branch, but Sometimes the dependencies are different and you have to enable disable things in configure or whatever the build the build system is so It gets gets complicated and to do it in one spec file It's this is not not not the worst. What I would suggest Like I said separate branches separate spec files for for each distribution and Doing automated builds Which there are a lot of talks about doing automated builds And these days so choose your poison Yeah, this one was a killer Actually, yeah, I encountered that mean The package did drop files in a specific directory that it was afterwards Removing in a post uninstalled scriptlet, but that Developer didn't take into account that another package might also drop files in the same directory so Actually, I had one Outage when we were insta uninstalling some legacy package from from all servers and a Few minutes after we did it A colleague called me. Hey, this my software stopped working. What were you doing? Installing some other package But all the files are gone and then I went inside Actually, I didn't have I didn't even have a spec file just the binary RPM. I saw this in the script Okay. Yeah, we know what to do. Just use no scripts in this case and never install this again Yeah, so have to be careful with your directories. So let let our PM do his job for list everything in spec in the file section and You know, if there are files generated at runtime You can also list them in files And using the ghost directive You you can specify the expected attributes also as well so that RPM dash capital V also works to the extended can to verify that the File mode and ownership so on so yeah There are better solutions than just running RM dash RF In a scriptlet Yes Yeah, it's it's a big hammer not Not to not run any scriptlets and yes, there is a way to rebuild a binary package. I mean like reverse engineer a binary package together spec file from it There's a tool called RPM rebuild and actually I Was surprised to to to find it. I wasn't aware of it a couple of years ago, but When I found how some of my colleagues were Doing releases of a new package versions using that too That made me cringe a lot But yeah, it's it's a tool and it's it's useful when when you don't have access to to the sources I Don't think it's installed by default, but it's it's there somewhere You can install it by path. I don't remember the package name So yeah There are ways around that. Well, if it's a you know vendor package that cannot be modified because otherwise It's not supported and not much you can do Yeah, this one was so interesting because I I Explained to a colleague how to How to make proper RPM packages and the first of his releases was fine And then he released an update and then I found this in in in his updated package and I What are you doing man? I think I told you how to do things properly I don't even remember the explanation, but yeah, that was terrible because Which which we are more product was it a player, okay, I actually haven't haven't touched that sorry use that Yeah Yeah, I IBM IBM software does that a lot. I I think I tried to to force what was some Tivoli monitoring agents Into into a proper RPM than I found it was actually It was an installer Tarble with an installer script which which also contained RPM binaries and it was Installing RPM during the RPM packages during this install process. So it was it was terrible but it's it's what you have to work with in a in a big company which which uses a lot of proprietary software, so I mean Yeah, I Nothing against IBM's personally just some pieces of it So, yeah Don't do this Just put extract everything in in in the Section where you supposed to which is install if even software is stubborn and for example was asking for route During install because they they were assuming Yet an admin would just run their install script and you can try using fake root or pseudo and it It works in most most cases. So I recommend trying at least and well Worst case, you can you can you can actually run the installer in the install in Post-installed script but but then put every file in the file section and Use ghost, but that that's the absolutely worst case Actually try that one Yeah, exactly there is no no terminal even So But people do things and Yeah, that that's what I first The names of the libraries are of course real the paths are not but we had a package that did something like that and Then I I think I tried to Uninstall curl and it succeeded and then something else stopped working Because the curl was already well lip curl in this case was already provided by my pet So, yeah, I mean this is bundling right and bundling of things that we have been in the main distribution so You can of course do bundling but do it properly and and filter out those library duplicated libraries from from the metadata, so they are not exposed to RPM database and Don't conflict or make your software pretend that it provides something that it's not for general consumption so, yeah That's that's why tools like RPM if RPM inspects can help him also even for non fedora packages Because they will file all the many of the fedora tests But still the output will be useful and since you can select which tests to run Or I guess you can also write your own tests That will be useful That's That's all the cases I had But just before the talk we had a beginning of an interesting discussion with Tom About about tech life being a real monster in in fedora and he did some marvelous Things to make it less horrible So, yeah, if you have any questions or have an interesting cases to discuss Only they require a bunch of 650 of them not So Because it can't be removed from trying to try to contact npm and use these npm's I Pre-download them as I'm packaging Yeah, that's what I would probably do as well when pressed for time because well you can't I mean it would take months to Or maybe years to package 600 dependencies Wow I think I mean to do it properly. I think automated packaging is the only way Like like I think I was I know someone did it with Python just took all of pi pi and Filled rpms out of that automatically was it mirror or someone else? Think also we have a lot of work done for a go ecosystem for this But yeah I mean go also tries to Download all its dependencies during builds unless you already have them That's yeah Then what modern modern language stacks are kind of like that Stories Interesting Oh It's not provided like P-Python over, neither in Pekkin, neither in GDAL, but the GDAL version was like a compile, so it was a different file, while the Pekkin was normal, no arc, so it was in different file, there was no collision, so you were able to install good version, but of course, they didn't have those correct class, so it was masked, but the GDAL version was totally first, so that's one interesting thing which not our QE tool catched. Right, so it was a case of name overlap, it wasn't the same software, just the same name. I have a similar case, just came to my mind when I was trying to package something new and it had a dependency on a Broadly module, there are actually two implementations which are, I think, completely or mostly API compatible, one is Broadly and one is Broadly Py, but they provide the same module name, and one is Infadora, one isn't, or maybe it is already, I haven't checked in the last few months, and the developers of the software I was trying to package, because I think the one Infadora is implemented is pure Python and the one that was a dependency is rewritten in C, so it's a C Python module, so they said they want to use that one, because it's faster, but still there is a name, well, module name clash, but not project name clash, it actually installs into different directory, and yeah, there are the traps you have to watch for and avoid. Yes, Tom? Someone sent me a package a while ago for a review, they figured out why it wasn't building in Koji, and the configured tool was querying GitHub repos to pull down dependencies automatically and failing if it didn't, it wasn't able to succeed, and so the person packaging this attempt at solving this problem by adding a pre-scriptlet that was trying to use nmcli to turn on the network Seriously? Wow. And they were a little disappointed that this wasn't working, and they were wondering if they'd gotten a network manager command line since Accent for Accent. I mean, I respected their attempt to solve that problem, but I set the vote on the stream. The other thing is that they made an earlier story, Chromium has a check for specific fonts with specific checksums, it doesn't need them. It just makes sure that they're in the directory, and it's not in their git tree, so part of a Chromium source file is a download of these font files, shoving them in the directory so it can pass the build check and build them. They're not attached. I guess that was easier than patching the build script. Yes. Yes, it was. Yeah, I know. All right. Any more questions, stories? So thank you all for coming.