 Hello, everyone welcome. Thank you for coming and thank you to the organizers for Letting me speak to me the slot my name is I can laugh I'm an engineer at red hat on the image builder team and Today I want to talk to you about how we not going to talk so much about how we build images But how we think about defining and configuring the images and letting users sort of like Invent the universe of image definitions. Let's say So one small thing I'd like to say I Don't know if this is a good idea, but I'm gonna I'm gonna be using it if you like some terms That we use internally and with these things It's always a little hard to understand trying to explain everything of course But it's always a little hard to know what other people are familiar with and what not So if I use a term that's unfamiliar more than a couple of times raise your hand and We'll see if we can Keep the ball rolling so that I don't lose my audience So Yeah, let's get the show on the road A quick overview of what I'm going to talk about I give you a little bit initially but So We build images and it turns out that building images is pretty easy But making sure they boot and that they're useful. It's tiny bit harder So the trick is to sort of restrict Users and the code itself so You know so that it's hard to build Sort of invalid configurations or unusable things But we also want to give users the power to sort of like Explore the space of what what kind of images you can build And these two things are kind of in open this opposition to each other right so you can imagine a solution where? You know you can only be like a project where you can only build five kinds of images or even one That would be really easy to test you can't do anything else except build that one image It always builds it always boots. It's fine. It's just not a very useful project On the other hand you can imagine a situation where you can build almost anything But most configurations wouldn't work So we kind of need to find the sweet spot there or sort of guide users and And ourselves to only build things that make sense But also be powerful enough so So that it's useful and and so What I want to talk about is our solution or like our way of thinking Where we sort of define abstractions about how we define images as configurations of components and these components inform both the way we implement things in code the way we Define our own sort of libraries But also how we how we want to present these configurations to the user so that they know what we're doing So part one let's have a look at image builder image builder builds images and Let's have a look at that the first, you know give a bit of an overview about what it looks like to use it and I'll talk a bit about like I said how we build images But most of the talk is about how we define them and how we think about what an image type is or an image configuration is This is image builder is what it looks like when you want to build an image This is the image builder running on console Red Hat com And this screenshot shows sort of the first step of the image building wizard and This is where the user sort of selects a target platform Which defines the kind of image that we're going to build in this case you can see Amazon web services selected So we're going to build an AMI type image and and make sure it works with Amazon web services By the way image builder also works on premises. We have It's not just the a red hat product, but it's also utilizes and run On your laptop and build images from the command line or from pop-up composer or Yeah So this is like a follow-up step sort of like halfway through the image creation process Where users can sort of select additional packages to add to their image In this case we use the user selected engine X. So presumably maybe they want to run a web server and And and what I'd like to do is sort of explain how these options and others like them like selecting images or selecting the target platform How they affect the process of creating the image and how we might be thinking about these configurations in the future So at the core of image builder like sort of like the bottom layer of layer of our Stack is OS build and OS build is a command line utility that takes in a manifest and returns one or more more file system trees and It does most of the work well it does the actual work of building the image and a manifest is a Giant well not giant a rather big Blob of Jason that describes a series of pipelines and steps So you can see sort of here like there's an rpm stage There's an the kernel command line stage post name stage So these are all steps that sort of modify the file system tree In very specific ways to create file system tree that resembles a blue food image I was bill has Important know those people have no knowledge of distributions or workloads or anything like that it quite simply and very stupidly Execute stages as described in this manifest and just returns whatever the result is at the end of the process This is sort of a simplified way of Simplified look of what the Jason object looks like I just pulled out the names of the pipelines of the stages And you can sort of see that each pipeline is blocked broken down into a series of stages and Most of these stages have pretty self-explanatory names if it looks like the name of a shell script That's probably what it's calling So for example the rpm stage install pack rpm packages into a tree the cloud in its stage configures cloud in it System D and enable services You can configure the boot loader install the boot loader Configure engine X and The nice thing about stages is that they define their inputs rather strict strictly and they don't sort of expose the full Well, we don't always expose like the full sort of like Breath of what you can do with a with with each thing but They kind of in a type safe way modify the the tree to create a bootable image And So on its own like I said, I was filled doesn't really have any concept of what a distribution is or what it's actually doing In the grand scheme of things So it isn't very useful on its own and no one's expected to write manifests by hand or even understand the The individual stages that we define so we provide a library that Right I agree that holds the sort of like the domain knowledge of what a bootable image or a specific distribution looks like and how to and the library knows how to create a manifest that will accomplish that and It's within this library which we call oil spill composer is Is where we define the the base images that we present to the user like the AWS image that we saw in the first Second slide so Right, so again following up Sort of repeating again what I what I'm trying to say here is that always will makes no guarantees about what it's going to produce for any given manifest In the sense that a manifest doesn't guarantee anything useful or bootable always will compose around the other hand needs to produce images what needs to produce it needs to Produce something that can be built and needs to match what the user respected the user expected, right and Ideally it should be useful and usable and the easy way to guarantee this Like I said in the beginning is to restrict user choice the smaller the configuration space the fewer things that can go wrong and This is sort of what we have right now. So when we talk about image types Like the AWS image type we refer to an image or an archive that contains an operating system tree and it's sort of a predefined configuration That matches a distribution a platform and an environment kind of like this at the three tuple configuration and Additionally the user can add their own little user customizations like we saw in the example where the user can add engine x to the image build and These user customizations here are basically only the real They're the only real control the user has to affect What won't come out at the end to actually like apply their own preferences to what they want from the image? So for example, you can build a fedora 38 image that can run on x86 in AWS and then add an engine x To the to the image or you can build something like a rel 9 image that runs on arm in Azure and Add a 20 gigabyte o pt partition And and to repeat myself a bit in code as well as the user interface to an extent It's these three first components that we use to sort of define the image types And and we do that in a rather static manner So for example, if we if we never explicitly added an image type that is called Fedora 38 on arm on AWS then that's that doesn't exist essentially in our code So even though it's a perfectly valid configuration we don't support it it doesn't exist as far as we're concerned and And and And so Each choice of these Each each choice of these components sort of the distribution the platform which is like the hardware architecture and the environment It's going to run in have very specific Effects on the image building process the the they They're associated with different configurations of the stage that we saw earlier To produce an image so for example the distribution so it defines the base packages that you're going to need And and repositories that you're going to use to download them and then the platform again Specifies the specific subset of those repositories and some additional packages for like the bootloader or the firmware And then the environment which in our example was sort of the AWS you add additional packages and configurations to make the image Run there run in that environment And so we would like to sort of move away from this sort of static configuration situation That I described earlier where like everything sort of defined as static combinations of these three components And towards defining the effects that these that the choices of these components have on The image building process and if we can define if we can have like a well-defined set of what these things do Then are sort of like our we can expand our configuration matrix to everything that's valid And so this is the part of the talk where I describe where things that you don't yet entirely exist So this is sort of like where we move it where I imagine that we're moving with the project And and some things that might be more conceptual for example and might change in the near future But what we want to do is to find the choices we saw earlier as abstractions in our code and then conceptually sort of Well in our code, but also conceptually and then help us sort of reason About what an image definition is and how we present these choices to to the user So we can go back to this sort of like breakdown of the of the components where we have the distribution The platform and the environment which before we sort of used in this sort of static way to define what an image type is And then a workload Which sort of defines what the image is intended for and and so the distribution as before you can imagine like I think we all know What a distribution is you so a Platform which is sort of the watch it which is the hardware-active architecture But maybe a bit more generally could be a specific device as well the environment like we saw before it could be a cloud environment like Amazon Azure or Or Google Cloud or it could be a bare metal environment and the workload which Again, like it's sort of the intent of what kind of work the operating system will be server will be running And you can you know you can sort of think of the workload about the kinds of things that you would normally do at provisioning time Installing packages setting up configurations and things like that And so we can sort of sort of start selecting this we can go through sort of an example of how Of how select selecting each one of these components making a choice in each of these components has an effect on what kind of Stages we will be running and what how it'll affect to the the image building process itself And so for example you pick a distribution like for door 38 and that sort of restricts the repository. Well, it selects the repositories All the fedora 38 repositories, so if you're going to build a fedora 38 image, you need the fedora 38 repositories You if all you know is that you're building a fedora 38 image the You start up with the base package set which is the core package group and then If we're going to build this image, we need the fedora 38 build environment So that's all we have now. We only know that we're building for door 38. So this is what we start with And then you add on top of that the option that you write you'd like an x86 image So then you restrict the repositories to the x86 repositories Don't need to be downloading anything else. And then on top of that you need to add for example the bootloader packages for x86 And at the same time you need the stages in the build process to configure and install the bootloader So and then at the next step in the environment if you select you want it to run on AWS It'd be good to have the cloud server package group and the cloud init, which is how images get sort of like provisioned or configured at first boot through the AWS cloud console and since we're installing cloud in it then in the stages We also enable the cloud in its service using the system d stage and then finally You might select a workload like a web server which would add an extra package engine x in this case and then since we're adding engine x We also want to enable it and configure it So that's an extra option on the system d service and the system d stage in the manifest and then Stage to configure engine x And so if the effect of each of these components on the image creation is well-defined We can sort of freely combine them and we don't have to stay in this sort of like world of statically defined configurations and Instead of in place instead of explicitly defining valid combinations We can sort of restrict the configurations only to the things that we know are invalid And then sort of like just let the space of valid of known Configurations exist so we can sort of expose the entire configuration matrix encode to the user and and know that any combination It would be would be possible And So we're thinking now about what choices we can what choices should we give users and how? Can these choices? map to the components that we're talking about here the components that we define And so like the current state like I said is you have this sort of like static list of these These triple configurations here the distribution the platform in the environment And then you give user a few knobs to sort of like add some packages or tweak some configurations And Like it's sort of this this this conflict These static configurations here are sort of exposed to the user, but not really like it There's a if you're if you're on a system with a Of a given distribution or if you're if you're building for a specific distribution Then you're only limited to a handful of image types that we've already defined And sort of the identity of the image is a sense is essentially this static the static configuration here and then the user can sort of just tweak a few things and Our new set up on the other hand sort of puts equal weight on each of these components So our question is why don't we just expose all the components to the users all the components and sort of let them freely Snap them together and and create what they need. So What this would enable us is to sort of like both ourselves and the users define to select Each configure each of these sort of components and What what we do in in turn is abstract away Sort of the meaning of these components in terms of the image build process But not the meaning of what's going to happen to the final to the final image and So My question here the reason I'm talking to you all about this today is to basically ask you Does it sound like a good idea? At least this is how we're thinking about things and like does this is there something here that kind of Is there something that we missed is there is there a case that kind of wouldn't be covered by these configurations by these components Does it sound like a good idea to be able to just? Have an image like the one we saw but instead of on AWS you can just swap out the environment and have The same image for the same Workload just able to run on a different cloud service and be optimized for that cloud service And that's pretty much it. Thank you Questions and comments are of course welcome and this is the website for our project and the github repository for our organization Service All right So the question was what's the motivation in general and I guess I guess the extension to that was how does it differ from? If I'm getting like a base image and provisioning and modifying it Well, there's a couple a couple of answers to that first. Well, someone needs to be a build the base image and there's So there's a lot of alternative projects that do exactly what you said Which is like get the base image. It's just like in our example For example, it would be just get the base fedora image and then add the bits to it to make it So there's a couple of things that this project does solves that I don't know that we think is a good idea at least first of all every time you build an image you get fresh content, right if you If you're if you're building an image today to deploy today You're gonna get the up-to-date packages instead of maybe like needing to get in an image that was built a week ago and or a month ago And need to update it. You can there's there's also the concept of like you're building an image for purpose and it's Well, there's a lot of there's a lot of things that are Either impossible well better to do or must be done at build time instead of provisioning time There's there for example, I think Openest that is what we have like hardened images you can build a hardened image with certain configurations and Take an existing image booting it and then flipping those configurations isn't considered like compliant with certain security standards, for example I'm sure there's There's a few other reasons in use cases everyone on the front row can probably answer your question in a different way Hmm Partitioning for example is a good. Yeah, it's another way like you if you get an image like we can partition during build time Create any partitions that almost any partitions the user wants and you just get an image and deploy it to AWS You can you can add certain you can add partitions after the fact, but It's it's certainly not well I Guess like if you if you get a rel image and deployed to your refrigerator image and deployed to AWS You probably can't create a user partition After boot after booting it right so there's there's a lot of reasons you want to configure something before you even build it Yeah, so the follow-up question was I Guess it was too well the second part of the follow-up question was how long does it take to build and If it's provided as a service Yeah, so yes, it is provided as a front service. That's on console red hat com now for all red hat customers and the Image build time is well. It depends on the image. It can be as low as I guess five minutes Up to I guess 10 or 15 if you're building and It's like the cloud images usually take like between five to seven minutes And we can also like deploy it for you directly to the cloud environment We can we can also build isos and those usually take a little longer Repeating that there's also the on-premise version that you can run if you have an RPM based operating system You can DNF installed OS build Composer and have it running and build your images now I don't know who's first you Look So the question was how do we How does this model work for situations where you might not know the workload you just want to build an image? well, so it Well, first of all the sort of like the workload situation is kind of optional. It doesn't need to be a predefined workload The way it kind of works now if we map this sort of like new way of thinking about things to the old way we would of course crop have a Custom workload or even a null workload where you you don't get any extra packages. You don't get any extra configurations You just get the base system Obviously you can't you can't skip the selection of distribution and platform that wouldn't make sense environment You could have like a base environment that sort of it doesn't belong to any cloud or anything It's kind of just something you know, it's just something that that can Maybe maybe maybe you can have to think of it as like like the default is like a bare metal installation for example But that that would totally be an option, right? It's like any There's they're saying defaults. I think for for the environment and the workload that we can sort of think about And Custom software the follow-up question was how would custom software be layered in Well, that depends. I mean right now we already have the capability of Letting the user define their own content sources if you have an internal in your company an internal RPM repository Or a public one that's third-party. You can just define it and pull content from there and it'll be installed We can bet containers at build time as well I Mean any other ideas were open to suggestions about how to get Yeah Yeah, you can also inject custom files at build time for the on-premise version You can just write a file and it'll get injected at build time into the Etsy directory Is a question or am I out of time? Okay Do we integrate with Ansible short answer no long answer No Yeah, there is an answer. Oh, sorry answer will integrate with us Right Right Right Yeah, which is sort of what OS build is doing, I mean not sort of that's what OS build is doing How much first of all how much How much time do you have? Because this this seems like a discussion for over there And I think there were a couple more questions if we have time to get to and there was one there before Sorry, can can I get some? No, sorry start again. I didn't hear anything. Oh, yeah, of course Well, the question knows if we consider other platforms and yes, we can build Okay, we don't have virtual box. Yeah, we have right. It's the same as lip. Okay, so we have you Yeah Right and that's sort of the thing right. It's not just about making the file the file format correct it's also about knowing what does this virtualization platform need and We're always we're always expanding we're always adding more stuff and that's sort of what part of the reason We're thinking about this. We're rethinking about how we put these things together is that For example, we said, let's add Let's have let's do that. Let's do vagrant, right? We would figure out vagrant for well nine and then it'll be like and then we would we would be thinking Okay, this is this is livered for Well nine x86 and then we would have to go through that again and redefine it for every configuration And now what we want to do is say, okay What is what is vagrant need it needs these things? We define them in sort of like a well-defined way and how they interact with the other components And then you can drop it into any configuration and it should work Okay Okay, so the question is if you if you want to do something that isn't covered by OS filled you need to write your own OS build stage Yeah, um, I mean that's kind of the way to do it like if there's if there's some kind of configuration or I don't know. There's something that needs to be run at build time that the OS build library doesn't cover with a stage The answer is write a stage but the The best answer is write a stage and send it upstream please because we need them Right. Is that no, it's not Well, there's someone behind me and I think you are the What if you want it? So the current state Yeah, the question was yeah, the question was We have the yeah, we have the manifest we have the the UI and stuff, but what if I want to Version control my sort of set my configuration my image type of what whatever you're doing well, the good news is the the way you build an image on premises is with what we call a blueprint which is a toml file and I'm out of time, but it's a text file. You can version it We also couldn't we also support versioning the way it works now, which might change soon But the you write a toml file which configures the image and you push it into composer and composer itself also has a versioning system That might be going away But it's text files You can yeah, you can version them and even in cockpit composer, which is sort of like the web UI that you run on premises You can click through and configure the image and then you can export that configuration as a toml file and save it We're an amount of time. Thank you all very very much