 Let's get into the big topic of open source, something that we actually have in mind. This is so awesome. We are an open culture that needs to be used to process that developer or let's say... As the Kubernetes ecosystem really joined up. And good morning, good afternoon, good evening, and welcome to the level up hour, where we talk about all things Kubernetes, containers and OpenShift. Thank you for joining us today. We ask you to like, subscribe and share so that everybody knows that we're on the air. And we have a great episode planned today. I'm Randy Russell, Director of Certification at Red Hat, and I'm joined by my colleague, Scott McBrien, the legendary Stabby Mick. And Scott, I think we're going to today shift a bit away from the world of OpenShift on Kubernetes to the world of Red Hat Enterprise Linux in containers. So what have you got in store for us? Yeah, so the two are kind of intertwined in some ways. Because OpenShift is a container deployment management application platform. But you actually have to containerize your application to get it onto that platform. We've seen in the past some methods like CI CD pipelines and other stuff. But today I want to talk more about specifically using the Red Hat Universal base images and some thoughts and recommendations for how to build applications off of them. Well, so we've touched upon this topic previously, haven't we? Yeah, it was an episode 47, if memory serves correctly, was on containerizing applications. But in that one we were talking about taking an existing application that you had running on something like a RHEL or other Linux host and shoving it into a container. Today we're really talking about using the infrastructure that Red Hat already provides using universal base images and the software that's provided as part of it to build out what your organization's base image or your build should end up looking like before deploying, either on a RHEL host or through OpenShift and Kubernetes. Well, okay, so just so I'm really clear here, you actually have a blog coming up on this topic, right? Yeah, so one of the things... Just to give you a little bit of a teaser. Yeah. So a couple of weeks ago, the Red Hat developer advocacy team contacted me and said, hey, we got some reports that people are having difficulty installing different versions of software into the UBI's. Could you write an article that tells us how to do that? And so we're going to start with just really simple across the different UBI's because there's some different tooling between them. And then I think we should talk a little bit more detail about that next level of complexity, which is choosing application streams that you want to put into your software. And I think we can get into what is an application stream, how do we manage it, all that kind of stuff. Live demo style, what could possibly go wrong? So, dare I ask what an application stream is, or are you about to unveil before I very eyes what an application stream is? So application streams is a concept. It replaces an older concept called software collections that we shipped with Red Hat Enterprise Linux 7. So application streams was newer to Red Hat Enterprise Linux 8. And we're carrying that forward into the UBI. So application streams was newer to Red Hat Enterprise Linux 8. And we're carrying that forward into Red Hat Enterprise Linux 9 as well, which you should expect middle of this year. And the idea is that historically Red Hat Enterprise Linux has been seen by developers as old and crafty and doesn't have the right versions of stuff or the latest things. And with Red Hat Enterprise Linux 7, we tried to resolve that with software collections, and 8 and 9 were doing that with application streams. And so the concept is, we want to provide newer versions of stuff during the lifespan of a Red Hat Enterprise Linux release. So way, way back, back when Randy and I worked together as field people, we would package up, say, Node.js 10. And on day one of Red Hat Enterprise Linux dropping, it had Node.js 10 in it. That was what you got for the entire duration of that rail release, period, full stop. And so like eight years down the road, when you're still working on Node.js 10, it was really uncomfortable, right? Because there were a lot of stuff that you wanted that you couldn't have as a developer. So what we've done with Red Hat Enterprise Linux 7, 8 and 9 is we have this kind of intermediary method of providing newer software. And I'm not going to get into 7 because we like, man, we made some bad choices. And we fixed that. Surely not. I know. I know. Things like, in 7, you could have as many versions of something as you wanted installed at the same time. So like, you could have Node.js 10, 12, 14 and 16 all in the same box. All living happily together. No problem. Happily is probably not how I describe that. It was like having all of your extended family over for an uncomfortable holiday dinner, right? You had to know what everybody's thing was and navigate the complexities of their relationships. Dietary restrictions. I mean, yeah, it's, yeah. Exactly. Oh, oh, little, little Jimmy is a vegan now. Oh, excellent. I didn't know that until just now. Oh, that's the version of G-Lib C that he eats. Exactly. So in rail eight, with application streams, again, same concept, provide newer stuff over the lifespan of rail. Except we changed one of the big things we changed, which you can choose any version you want, but you can only have one. It was like the Highlander model. The Ford Model T, any color you want as long as it's black. Exactly. And so while we offer Node.js 10, 12, 14 and 16 on any system, you can have one of those installed. If you want a different one, you have to uninstall the one you have and then install the new one that you want. And that allows us to do things like make sure that the executables and libraries are always stored in the same places so that you don't have to use like weird extensions on your executable calls or library directories to make sure that you're using the right ones for the version you want to call. One fairness, if I think historically about Red Hat Enterprise Linux, one of the things that has been very valuable about it is that it has historically done a lot of support for being able to manage this huge number of different projects in all the different libraries and requirements to go with that. But perhaps the idea of having multiple versions of an application on there was just a bit more ambitious than is practical, right? Well, you could do it, but like I said, it was complex and you had to know a lot of things in order to navigate that complexity. And you had to know a lot of things at different levels of your organization, right? Your operators and administrators needed to know things about like how to install it and manage it. And your developers had to know a lot of things to make sure that you called the right thing that was installed or it wasn't there or how to request that it be put there. And so by decreasing the complexity, we made things easier on the entire stack. And then if there is a case, because occasionally I get asked like, well, but I want all the nodes on the box, the answer is containers, right? Have a Node.js 10 container, a 12 container, a 16 container, and there you go. So Scott, Conan has called you out already on in chat and said, UBI's, it's plural now? Thank you, Neil Galba. So, yes, there are actually four universal base images. How universal could it be if there's four of them? Well, they're all built off the same package set. And so that's where the universality comes from. But if you want, we could switch to the demo system that got set up and we can talk about that a little bit further. Okay, let's go to the live demo. What could possibly go wrong, Stephanie? You know what to do. What could possibly go wrong? So I just want to make sure that everyone can see my screen. Randy looks good to you. Yeah. Awesome. All right, so I wanted to just go ahead and download one extra UBI. I've already pre-positioned some of them already on the system. And so what I'm doing is I'm using the build a command to build from a container we're going to download from registry. And I'm pulling down UBI micro in this example. Because I didn't specify a version that goes ahead and pulls the latest version. You could also use podman to pull this image if you want to build off of it. I like build up because you'll see down here at the very bottom, it automatically creates a working container for me. So it creates a file system overlay and then allows me to interact with that container immediately. As opposed to with podman pull, you'd then have to melt the overlay and do some other commands to get your environment going. There we go. Sorry. Not enough coffee at this morning. So I've pulled down all four of the UBI images. And we're going to mainly work with three of them today. So UBI micro is the smallest image. It has no package manager associated with it. And that leads to some interesting things that we'll get into in a bit. UBI minimal is a fairly small image, but it includes a package manager called micro DNF. And we'll get into using micro DNF a little bit today. And then these two UBI and UBI init also include package manager, but they include the full version of DNF. The difference between UBI and UBI init is that UBI init also comes with a full implementation of system D. So back in episode 47, we were talking about containerizing your application. If you were containerizing application that already used system services and system D units to control daemons and other stuff, init may be a good choice for you because it includes that system D infrastructure. If instead you were wanting something with executables and didn't need system D, then maybe UBI would be the one that you choose because it doesn't include that extra package set. But those are the four that we've gotten, and they also vary in size. So UBI init, because it has the most stuff, is the largest. You can see that it's 253 megs compressed. I take that back. That's uncompressed. UBI is 235 megs, minimal is 107 megs, and micro uncompressed is 38 megs, 39 megs. When these are compressed, they're even smaller. So when you're doing things like copying their images to hosts or to different nodes in your open cluster, they're transferred compressed. So I think UBI micro, for example, is like 12 or 13 megs compressed. So it's not a lot of transfer time that you're looking at there. So hopefully that is a really long answer to Konan Kudo's question about there being four UBI images now. I think it's clear and comprehensive, sir. Okay. So because I had already downloaded the others besides micro before the start of our show, I also have working images created for them too. So they're already mounted and available, and I can start working with the content of them. So the first thing we're going to do is work with the UBI image. So there's a working container called UBI-workingContainer, and I am going to install a package in it. One second. We'll make sure I have the right syntax because everyone loves it when I mess up the syntax. We're actually kind of quietly cheering you on, Scott. Quietly cheering me on to make a mistake? I know. Well, yeah, exactly. All right. So for that one, we're going to build a run. We're going to execute a command inside of our container. The suspense is killing me here. And any requests for packages to install already? I'll leave that as an exercise for the demonstrator. Okay, let's do... No, since we were talking about it. There you go. We were talking Node. Node's pretty popular. All right. So what it's doing now is it's pulling the repository information for the DNF repositories where all the software is stored in the universal base image. And then it'll go through and just calculate dependencies, pull anything it needs, and then install it. And Neil wants to see Python 3. So how about we do that a little bit later when we talk about application streams? Okay. So we pulled in Node.js 10, and that's now installed in our container. Okay, if we were using minimal, minimal is smaller, and it includes not the full DNF package manager, but instead one called microDNF. So a very similar command. I'm not going to do Node.js. I'm going to do... So the command that we're looking at is same thing. We're going to run inside of our container. This time I'm choosing the minimal working container that was created when I downloaded it from the registry. And now instead of using regular DNF, I'm going to use microDNF because that's the package manager included in the... So let me ask a question there. Let's say that I'm trying to do this and I lack your awareness that really I need to use microDNF. And I try to use just the standard DNF. What would happen? Well, let's do it. The answer is not much. Basically it says there's no such command in there, so sorry, Node.js. And so it's, you know, it would behoove our audience to be aware because note that it doesn't say, oh, but please go and use microDNF. It just says, hey, the executable isn't there, which is in fact true. But really what it's, what the underlying message is, is that you need to use this other version of DNF, which is scale... MicroDNF. All right, so microDNF works a little bit differently. You can see the output's slightly different than regular DNF. But it's essentially doing the same thing. It's going into repository information. That's what this downloading metadata is. And then it'll calculate the dependencies and then install the software that we said we want. And so that's like one of the big, one of the big gotchas when working with the UBI's is that init and regular UBI, they both use full DNF. Minimal uses microDNF. And micro, yeah, UBI micro doesn't have any package management at all, which leads to some other interesting things. Okay, so here we go. It went ahead and installed all the, all the stuff. And now my Apache server software was installed in my minimal UBI, or UBI minimal. All right. All right. So I've not tried this at all. So we're, we're kind of shooting from the hip. That's my... If fails, we're just going to move on. That's my favorite. Nothing ever happened. No, no, no. Let's diagnose it here live. I don't have enough, enough background to do that. So we're just going to be like, if it fails and creates a dumpster fire, we're just going to move on nothing to see here for UBI micro. Okay. So I mentioned that UBI micro doesn't have a package manager. And that's important because we can't use the DNF commands to download and install software. Instead, we're operating UBI micro as essentially a file archive. So how we interact with it is also going to be a little bit different. So let me find my right commands to use. Or what I think is the right command. Okay. So we're going to DNF install, install root. Oh wait, hold on. I need to know where it's mounted. Come on. Yeah, give me some gripping stuff. Not there. Not there. All right. So see, this is where dumpster fire begins. It's not, it's not yet Flavy. It's merely a spark. All right. I know that Neil is going to be running around looking this up right now. So what I'm looking for is when I'm doing the install root, I need to know where to tell it to the root. Tell it to the root of my UBI micro working container. Oh, I found a blog. Here, let me, let me give this to you guys too. So in our back channel chat with our producer. Go ahead and paste the blog that I found. Coming on his way down. And I'll share that blog. Yeah. So the problem is, let's, let's see if micro mount is a variable in my working environment. We can do this. I'm just going to create it. And it says you need to actually set it first, which I think you are doing now. Yeah. Shellage. Hold on. Hold on. So in the blog that I posted, and it looks like Neil has found a similar blog. So what they do is they instead set some variables. I think by now Neil is wanting to basically SSH into your box. Take over the demo. That's fair. It's really painful to watch someone else struggle while you just do nothing but help them to look on. Welcome to Randy Russell's life. Yeah. I literally get paid to do that. So we're going to do something a little bit different here. You know, the more people watching you, the better your typing gets. Okay. Finally. All right. So remember I said when you do it a builder from it created this, this overlay mount. So all we did was we essentially had to capture that metadata in a couple of chauffeurs. All right. So now that I've got on it, let me clear it off. It's really distracting because there's so much. All right. So now that I've got it. All root. It looks super. Yeah, it does. And in fact, I bet you're asking the question. I just told me there was no package manager in UBI micro. How are you installing stuff with a bag? How can it be? How can it be? All right. So let me let this finish and then I'll pull up my command again. So in the previous examples, we used build as our base command. Right. So we did a build a run saying that we're going to run some stuff inside of our already melted container overlay. But notice that here there is no build a command. So what it's doing is it's using the system installed DNF. To install this package. And normally we would install it on the host system. But with dash dash install root, I said instead of putting the files and directories around on my host system like a normal package install. Instead, what I want you to do is unpackage that RPM. And it's dependencies. Inside of this subdirectory inside of this chain rooted environment. And I pointed it at my container files to overlay. So I'm using DNF from the host to install the files into this micro UBI image. So again, we're operating UBI micro essentially as a file archive. And we're just making a chain rooted environment and then placing the files around in that chain rooted environment. Makes perfect sense. Okay. So before we move on, quick recap. So we installed using regular DNF inside of the UBI init or UBI containers. We installed a package using micro DNF in the UBI minimal container. And then for micro container, we had to use the hosts package management to then unpackage and build out our file repository inside of a micro container. Everybody good? Yeah. Okay. Yeah, we actually did have a little bit of discussion in the chat. Shwetty was asking about, you know, pointing out that there's a small difference in the size with micro UBI versus other UBI's and he was saying, Well, what's the, what's the main reason people use it? And Neil had jumped in with I think a pretty good answer subject to your approval, Mr. McGrine. Let's see. To avoid having a patch manager in the final runtime, that is accurate. I would go a little bit broader and say that you have very little software provided as part of micro. And so that allows you as the container developer to choose exactly what packages build that container. So you're limiting your size is one big thing because you're not including anything you didn't need like a package manager. The other thing that you're doing is you're minimizing your, your management and risk footprint because you're not including all of the software packages you didn't need. That means that you are not going to have to worry about like maybe the pull kit CVE that just got released because you know the X windows is not in there. You don't have pull kit installed. So you don't have to worry about managing that and mitigating that CVE. So Bruno also had a comment. He said it would be nice to have a UBI compilation container image for CI CD systems to build software written in C4L. I will take that to the developers. So, so there's several different people that build UBI images at Red Hat. And so like maybe that's a good place to transition from choosing software to pre-used or pre-defined software. So if you look at the Red Hat container catalog, there is a Python container image. There's a Node.js container image of various layers of Node.js. There is a JDK container image. There is a PHP container image. And some of these are built by, actually most of them are built by our one-time engineers. So the people that maintain those software packages build those images too. And I was working with a customer recently who was like, why is the PHP image so enormous? It's almost a gig in size. That's unacceptable. And it's like, well, okay. Well, you, that's private accurate statement. But what they do is they take like all the PHP libraries that ship with Red Hat Enterprise Linux, and they shove them all inside that PHP container. So it's, it's a- Everything you would ever want that we provide basically. Right. And so like, I think there's, you know, developer people thinking developer ways and operations people thinking operations ways. And dev off people maybe think a little bit of each. So if you were a PHP developer, how annoying is it that you're writing your code and all of a sudden you're like, I'm missing a library. And I have to go out and install it on the container and then rebuild the container and then redeploy it to continue my daily work. Super annoying. So the PHP container that we build that has all the things in it doesn't run that risk. Right. As long as you're using libraries and stuff that come from Red Hat Enterprise Linux, it's going to be in there. And you can just like code until your heart's content because it includes all the PHP stuff that we ship with Row. Now there's going to be a lot of stuff in there that you don't use because you're not using some random library, but it's in there anyway. And that goes back to like the earlier idea of minimizing your software footprint to one reduce size and to reduce risk and compatibility and security stuff. So when you look at the container catalog and you're like, Oh yeah, PHP, that's the one for me. Awesome. Use it. But if you look at it and you go, that's more PHP than I need. Well, then you're back to what we're just doing, which is taking one of the base images of your choice choosing and then installing the PHP bits that you desire inside of them to build out your development environment. Yeah, so I can almost see where, where, you know, in the process of kind of building out a solution here, one might start with the big so that you know that it's all there. You establish what you really need. And then maybe you do a new pass and you say, okay, now I'm going, you know, now I know pieces that I need. Let me go and rebuild and build it smaller. Maybe. I mean, one could do that. And I'm sure that there are people that do do that. However, my experience has been, here you go operations people worked on my laptop. And there you go. You just deploy that thing because it worked on their laptop. Yeah, that's true. Yeah. So that kind of brings us full circle because we started this discussion talking about application streams. And so if you've decided that you need to build your own thing, like PHP, or actually a Neil wanted Python. If you want to build your own thing, Python. You need to know which one to stick in that right because we ship multiples with right at Enterprise Linux. So let me go back to my screen share. And I'm just going to clear the screen here. So when we refocus on it, you'll see that it's a blank box, but it's still the same system. Yeah, yeah, a little bit of bait and switch. I get it. I mean, we can look at the history if you wanted to clarify. All right, so we're just waiting for the demo system to be brought into focus here. So you Stephanie, you could bring up some mellow elevator music while we wait. Oh, there it is. All right. So I mentioned earlier that Red Hat Enterprise Linux ships with multiple versions of different things. So I'm raising the NF, not young, adopting the new ways. Okay, in Red Hat Enterprise Linux eight. The way that we manage this was primarily through a packaging technology called modules. So the command I use was DNF module list to show me all the modules. I will warn you that in Red Hat Enterprise Linux nine, if you download and install the beta, you will see that there are many less modules because we're moving a little bit away from modularity to reduce complexity. Kind of like what we did with application streams in the first place. All right, so here we go. Conan Kudo said he wanted the pythons. And you can see that we ship several different pythons. Python 2.7, 3.6, 3.8, 3.9. Python is a little bit weird in that you can still have multiple versions installed at the same time. That's why the version number is included in the executable name like Python 39 or Python 27. So let's like suspend this belief for a second and instead look at our friend node. For a second, we're going to suspend it for like an hour. I think that's what we do. So you can see that node doesn't have that version extension in its name. So when I installed Node.js earlier on my regular UBI, I got version 10. I noticed that there's a little D next to it. That's the default version. If you just say install Node.js, that's the version you get if you don't specify a version. And then down here in this version is Perl. Python is a little bit special because we can specify the version in the package name we want to install and it'll know which one to get. All right, so we will go Python, but we'll do it a little bit later. Actually, you know what? Let's start with that because that'll make kind of Kudo happy. And when he's happy, then everyone's happy. Well, he'd better come back. He had to leave us for a minute, but maybe by the time he gets back, you will have this shiny install successfully completed. What do we want? Python 39. And I need my container of itch. And, you know, again, Python is sort of the exception to the rule that you had stated earlier in that potentially there could be different versions, right? Yes, they can be installed at the same time. But the reason that that works is because we included this version information in all the library paths and all the executable names. So if you're sitting on a box that has Python 27 and Python 39 installed in your shebang at the top of your Python script, you actually make a call to user bin Python 39 or user bin Python 27 to make sure that it gets the right one. And there is a way to make it using the alternative subsystem. So user bin Python points to one or the other. But that's outside the scope of our discussion today. All right. So notice that when I say install Python 39, there it is installing Python 39. And now Kun and Kudo should be happy that we talked about Python because he loves him some Python. And just to illustrate sort of the size question there is I was noticing that, you know, the style installed size of this payload is something like, you know, 45 meg. Well, if you think about micro, micro UBI is less than that for the entire container, right? Yes. But that's okay because we start off with really small and then you're going to increase the size knowing what you're putting into it. Right. So if it ends up being two or 300 meg because that's the stuff that you needed in your container payload. Okay. Well, you made the choice. Yeah. And the stuff that we gave you as the bare minimum was what, 30 meg. So of that 200 or whatever, 30 meg is container overhead. The rest is actually your stuff. And you actually want need for your particular application. Right. All right. So let's let's instead shift gears and talk about and let me do one example of using the modules. On the install. So let me look at the list again, because I already installed node on this, this UBI image. And let's say instead I want Ruby. Right. And so if we just did a DNF install Ruby, we get version two to five. But I don't want version two to five. I want version three. So for that, you same install command except you a module install and you tell it what version you want. And that's how we identify that we want the package group associated with Ruby version three. Pretty straightforward syntax. And so you can see in the plan transaction here, right? It figured out that it wanted the right version of Ruby or the correct version for me. Okay. But when we get into the smaller container images that don't have full DNF or any DNF in some cases, things get harder. So in the case of our minimal image that uses the micro DNF executable, let's say I want to do the exact same thing. This command pretty much worked on our regular UBI image. But now I'm making a call to micro DNF. And yeah, boom. So the problem is micro DNF, if we look at the commands, does understand the concept of module, but it only understands the concept of module in these three ways, enable, disable, and reset. But I said module install because I wanted to install a module software and it doesn't support module install. So that's why I got that error message, right? Unknown sub command install. Okay. So let's look at the list again. So remember that time I said that one of them is set as the default. So what we need to do is we need to specify inside our minimal image which one of these we really want. So before we do the install. Okay, wait, for the slow child in the room here, one is set up as the default, but you're telling me that we also have to specify. Well, so when it says the default, so if you do a micro DNF install Ruby, you're going to get Ruby 2.5. That's the one that's the default. But I don't want Ruby 2.5, I want Ruby 3. Yeah. So if you want something under the default, you have to specify. Correct. But I need to interpret what you said is something different than that because having to specify the default seems kind of contrary of the idea default to continue. Very good. So what we need to do is identify that when we install Ruby inside this minimal container, we wanted to install version three. So what we're going to do is we're going to use micro DNF and we're going to enable version three Ruby. So Scott, could you do a control that? Never mind. I was going to ask you to do a control L so I could see it because in the, you know, your handsome face of mind were covering up the command. I want to see it in its entirety. There we go. Okay, go back to the other view. That was great. We're good. Okay. So we just said enable Ruby version three. So now when I do my micro DNF install Ruby, I specifically set with micro DNF that when I said Ruby, I meant version three. When I do this install, it should pull version three. And sure enough, in my package calculations and stuff, it's talking about installing Ruby version three. Okay, so you have a little bit of a little bit of extra steps to work with when you're using EBI minimal and the micro DNF implementation that comes with it. Right. All right. So we're ready for my potential dumpster fire of micro. I am ready. Bucket of water in hand. Wolf. So let's, oops. All right. I was trying to be slick and recall my history from earlier because that would make my life less typing. But we're going to do is we're going to do a DNF install. All right. Let me just double check that I have the syntax right module install. Oh, that'll probably work before I have the package and the install route option switched. Let's see what happens. All right. So again, with the UBI micro, we're using the hosts implementation DNF. So the host implementation DNF knows about modules and can pick between them to know which packages to install. And then with the install route option, I'm saying, but don't install it on the host. Install it inside this change rooted directory environment. I'm feeling I lost my. You had a 50 50 chance. Yeah. No, I started either. Let's, you know, let's do this. Okay. So it's still there. Nope. Just checking to make sure I'm using the right syntax for install route because I'm not looking at my history. Well, while Scott consults that, I'll just remind everybody to post their questions in the chat. Should they have them? Okay. So it's the equals that's the problem. Dave, when you use the right syntax, maybe things work great. Maybe not. And for what it's worth, Scott, I'm with you on, you know, if you have a dash dash, you know, you sort of assume there's an equal when it's a dash dash. It's kind of the convention, but you know, sometimes it doesn't work that way. And truly, I mean, I'm looking at your value for, for micro mountain looks pretty, pretty absolute to me. Starts with a slash. I think it's about like ordering where like maybe module expects very specific argument ordering. BNF as a whole is usually pretty forgiving about that. But maybe it's not this day. I still know. Given path module. Yeah, I got nothing. Yeah. So mod, but it's interpreting module as the path. Oh, you know what? It may be needing something like what I did with my machine. All right. That's the end of the demo. Wait, wait, wait. No, no. Alexis suggested to drop the parentheses. Oh, around. That's true. That that may also work. But like, it's going to take, it's going to take five or more minutes for that box to re initialize. And then I got to download all the UBI's to it again, like maybe another day. And that's a task for another day. Well, this has actually been really, really fascinating though, to sort of see the operations of getting applications into a container. You know, some of the, some of the interesting differences between different containers and how you have to approach getting the stuff in there. You know, so how do you summarize all this for our viewers and readers and listeners? A good question. And I do have one last topic to bring up after the summarization. I would say right up your topic and then you summarize. Okay, fine. Don't listen to me. Go man. So the other thing that we've talked about historically, episode 52 was managing your source code. Managing source, ensuring that the provenance of your stuff is GPL compliant, right? Right. So when, when we talked about that episode, we talked about source containers that go along with UBI images. And so I showed everyone like pulling source containers for UBI minimal or UBI in it. But in this episode, we're talking about installing extra software beyond that. And so when you're tracking your GPL source code, you can get the source container from us that'll give you your base image. But then when you install Ruby or you install Node or you install Apache, that's additional stuff that you need to grab from Red Hat 2 to then make your full and complete image source for the image that you're building out. So that's just something to keep in mind as you're deciding what to stick in there. And that's another example of why one might use minimal or micro instead of the others because micro has just those packages that you picked. Although since we give the source code for all the images pretty easily with a scopio, yeah, you can really use any of them and then you just have to track your changes. Right. And again, you know, a reminder it's posted in the chat episode 52 delves much deeper into this and why you might, you might care about this because again, you start bringing things in into these containers. You still want to make sure that you are, you know, GPL compliant with any GPL compliant applications you're bringing in and the ability to bring the source in is actually very invaluable for that. So Scott, can we now get to the summary? Sure. I would say know which image you want to start with, because that's going to determine some of the tools that you use. You're doing things like building your container file to decide how to do automated pools and builds. You need to know whether you're using DNF or micro DNF or DNF with an install route. All right, so make that choice early because that's going to affect some of the other infrastructure you put into place and how you work with that container. The other lesson learned is what we give you the Legos, but we don't give you the instruction set. It's a free build. So if you don't like the PHP container we built, that's fine. You can build your own and you can choose what goes in there and how much stuff goes in there and what libraries go in there and everything else that you want. Or you can take the one that we've already built and be like, everything's in there. I'm good to go. And I'm just going to take the extra software payload. So do you have any sort of tips in terms of how to kind of make that decision early on? Because I can sort of see where a trial and error might be something that is sometimes going to be the case, but obviously, you know, it's better if you know initially. So what are some things that somebody can think through or consider that would allow them to make a good choice early on in the process? So the first thing I would say is size and threat vector. So if size and threat vector are two things that are critically important to your organization and how it manages software and containers, then that very clearly directs you to either micro or minimal. And at that point it comes down to how much customization you may actually require, right? Right. And between micro and minimal, like, you know, there's a hundred or so megabytes of extra stuff in minimal, but you get micro DNF which makes like the building and maintenance and some of the other things a lot easier. So are you willing to sacrifice some size and overhead to get that easier build and maintenance path? And I guess that sort of depends on how often you're doing build and maintenance, right? Potentially, yeah. Or that's at least one consideration. Right. If size and threat vector are not the overwhelmingly important criteria that you use for choosing something, then that opens the floor to regular UBI and UBI net. I'm sorry, yeah, UBI net. So with those, you get the full implementation of DNF. So again, it like makes building and maintenance easier. Maybe that's an important decision criteria for your organization. Maybe you have something like an already existing application you are containerizing. And UBI and UBI net provide that higher level of Linux resource with system D or more software packages or, you know, more fully featured Bash shell or whatever it is. So, you know, again, convenience and ease of use and maintenance and build versus minimalization and Spartan-ness. Spartan-ness. Well, okay, that's actually very enlightening. So, you know, I think we are coming to the end of the show. We don't have any outstanding questions. And so I would just ask everybody to, again, like, subscribe and share so that everybody knows that we're here. We have moved to a bi-weekly schedule. And so we will see you again two weeks from today. Again, thank you for joining Level Up Hour and we will see you in two weeks. Thanks folks. And thanks, Scott.