 Atomic host. Who's actually played with atomic host? Okay. So, okay, so one note is, and I'll repeat it from time to time if people come in late, but you can get these slides at this address. So if I go too fast, you can go back and look at the slides yourself. And yeah, well, I'll make it really easy for you to catch up if you need to go back to the previous steps and stuff like that. But at any time, don't hesitate to stop me and ask questions, especially if we're a small group, like just, just it's very, we're very chill. So, and gentle. So one, well, I'll make you do for the ones that are actually gonna do it on their laptop following along with me is before I even start talking anymore, maybe go go to the photo for our website and download the the image right now because it might take some time to get to download. I do have it hosted on my laptop so we can try actually and let me see. So I'm not sure if you'll be able to ping my laptop over the Wi-Fi but I guess it's worth trying. So I do have a hundred sixty five sixty nine sixty nine. So just because we don't want to hammer the Fedora server and it's way closer to to get it from my laptop than all the way to the US or wherever it's stored. So you can try to pull the image from my server. It's the exact same image that you would get from the Fedora website. So can someone verify if it works or not? Yeah, a hundred sixty five sixty nine sixty nine. Yes, you're not. Okay, so I guess it doesn't work over Wi-Fi. Unfortunately, let me double check that I yeah it should have been. Okay, well I guess you're gonna have to get it from the Fedora website. As an alternative, if anybody wants to use their EWS account, you could just spin up in this and send it to the US. Yeah, yeah sure. One of the final parts of the workshop is you'll be making your own image. I'm not sure if you'd have to be able to upload your own image to EWS if you want to do it from there. Or if you can start virtual machines on your own computer, then you can do that too. But okay, so did everyone actually start downloading the image? Okay, so you're also going to have to install these packages. So these packages are needed to run the actual RPM machine compose and to actually test it in a virtual machine. So I'm going to be using Libbird here because it's just amazing. It's so easy to set up. And these packages are needed for the final part of the workshop if you want to be able to create your own images. So this package, image factory, Plugman's Tinman is actually going to pull in all of the image factories. So which is actually maintained by Ian who's sitting right there. Great time to say that. So if we're doing this on its own, are we installing these in the bottom load? So right now this is on your dev machine, on your main machine. I'm assuming you're not using atomic hosts. If you are using atomic hosts, brave man, then you can do this in a container, Libbird and QE new, I guess, you can run Libbird in a container, but you're going to have to, it's going to have to be able to create virtual machines on your host. So I'm not 100% sure how that will work. An image factory too, it's kind of tricky because it talks to Libbird. So I'm just assuming everyone here is actually not using atomic hosts as their primary work station, right? I'll take that as a yes. So I can just give you a point, maybe I'm particularly slow, but it's giving me about 30 minutes to finish downloading the atomic host image. Okay, so then I can transfer, I have some keys here. So what we'll do, thanks for coming. So let's go to USB 3.16. Oh, okay. Let's copy to the key. Let's see if I can copy to two USBs at the same time. So you have which one is faster? The flush. On Windows, it's always funny when it's like five seconds remaining and then one hour remaining and then two seconds remaining, three hours remaining. This is why they have to worry about it. Okay, nice. So I'm going to start passing it along. Let's see if the other one's done. Yeah, that one's still going. So we got two right now. So it might be okay because it transfers pretty fast and reading is going to be even faster. Thanks for the marketing. We've got USB 3.16. Really? I think it might just be an illusion. Let's see. Yeah, maybe. Welcome. Which one is ready? Oh, the USB disk. Okay. Okay, start distributing this one too. Okay. So it might give up on... Oh, it did finish. Yay. So if anyone wants a slower USB key, anyone needs it? So I think we're okay now with three things. Distributing geometric progression. Okay. So for those of you who came in later, you can just get these slides from the address at the bottom there and you can see what you missed and follow along. So I'm just going to quickly go over what Atomic Host is just to make sure we're all on the same page here. So in a generic sense, Atomic Host is actually a pattern which follows the project atomic pattern. So that pattern pretty much means you have your RPM or S3 stack to direct how you receive upgrades and how you reboot into new upgrades. And on top you use containers. So the idea is that it's a very lightweight operating system and you just have to bare minimum needed to run containers on the machine. So the main features of Atomic Host is that it provides atomic upgrades and rollbacks and that's where the atomic part of the name comes from. And what that means is that, for example, in a regular system, if you're using YUM or DNF and you're updating your packages, at any point in time, if your packet, if for example a PostScript in one of the RPMs or something goes wrong, you might be completely breaking your whole system because it's left in a transitional state. Some packages have been updated, some packages haven't been updated and it's going to be a mess to figure out what you have to do to resume the transaction. And even with PackageKit now which supports offline updates, so it's much safer because you're in this very like this very minimal environment where you make your updates so it minimizes the chances that something goes wrong during your package upgrades, right? But even then, if you just cut the power at that point, you're going to have problems when you boot back up, right? Some packages will be updated, some packages won't be. So the idea with Atomic Host is that you get all your updates in one big chunk and then on your next boot, you boot up into all the new packages at once. So it's either you have all of them or you don't have... Does anyone else need the image, the Fedora image? Cool, thanks. So that's the key part, right? So imagine if you're managing a server of a thousand servers, you don't want to have that possibility of having intermediate states on some servers and some servers completing sexually. You just want it to be in a known state all the time for all the servers. So that's the first part. And of course rollbacks means if something goes wrong after you upgraded your packages, you can easily just go back to the previous boot where you used the old packages and the old kernel and everything is in a known working condition. So the other big feature of Atomic Host is that it's immutable. Immutable means pretty much you can't write anything directly on the host. So for example, you can't... Well, let me clarify that. Anything in the slash user directory is untouchable. So that might be really weird at the beginning, especially if you're coming from a regular system where you can just hack whatever you want. But the idea again is imagine if you're... If you have a server farm, if you're using embedded devices, you want to have a known system, right? You want to know exactly what's on your memory. And that's why it's great to have immutability and only mutate things that need to be mutated. So anything in VAR, for example, is you can change anything in Etsy for configuration changes. You can change, of course. But in user normally you have code. There should never be a reason for you to change code. You should always be updated from somewhere else. And that's it. You don't want to be updated on your machine. And then... What just happened here? Okay, I just don't know how to use my computer. So, right. And the big idea here, or the main thing, especially for this workshop, is that at the base level, it's just a bunch of RPMs, right? So in this workshop, we're actually going to learn how to transform RPMs into an atomic host. So this is sort of the general content flow of how RPMs become an atomic host. So you start with your bag of RPMs, which most likely are going to be in, you know, regular young repos. And you feed that to RPM-Oestry Compose. And the Compose will be the actual process that will convert the RPMs into an Oestry commit. So essentially, conceptually, it's pretty much doing a yum install dash dash install root into this temporary directory. And then we take this temporary directory and we put it into OS-Tree. And then on the client side, you pull down that Oestry commit and you set it up so that on the next boot, you will boot into this root of S. So what we're going to do here is in the very first, actually let me check. Right, so I'll just go to the agenda now. So we'll do this, I guess this workshop will be broken down into three parts. So the first part will be how to compose your own trees. The second part will be how to manage trees. And the third part will be how to create cloud images from your tree. So if you go back here, so the first part will just be about converting RPMs into an OS-Tree commit inside your S. Reboot and then pulling it down to on your client. The second part will be more about, let's say you're composing hundreds of times per day. You need a way to manage all that data, all those commits. You don't want your clients to just have access to everything that's coming out of the wire. You want to be able to, first of all, do some QA process, do some testing, do some promotion and then only let the clients see what you want them to see. So that's what the second part is about. And then the third part is, so let me back up a little. When we're going to start this, we'll be using Fedora RPMs. But conceptually, you want to be able to do this with just starting with just RPMs. Because right now what we're going to do is I'm just going to make you guys start up a VM that's running Fedora 25 Atomic Hosts. But really, you should be able to create your own Fedora or your own image, right? Let's say you have a new distribution called DevCon for distribution. You should be able to create your own image from RPMs that you can then bootstrap into and then start feeding upgrades from there. So we'll be cheating in the beginning because we'll just be using Fedora stuff. But in the final part of the workshop, I'll show you how to create your own image so that the only input to your process is RPMs, which hopefully you know how to handle and how to create your own. So I guess now we'll start the actual process. So these are actual links to the actual steps. So if you get lost or if I go too fast, you can go back into the text file and see the steps that you missed. So this is the part where I sit down. So now I'm going to, I guess I should switch to mirroring. Let's see how to do that. Okay, awesome. So I'm using T-Mux here, but if you don't know T-Mux, by the way, it'll change your life. You should really start using T-Mux. But if you're not using T-Mux, you can just use multiple terminals because at some points we'll need to use multiple terminals at the same time. Yes, I can make it bigger. Is it better now? Can everyone read this? Okay. So this text file here is the same text file that you would get from if you click on part one in the slides. It's the exact same text file. So I'm just going to have it here along with me to tell you so you can see what I'm doing and how it relates to the text file that you have access to. So we hopefully already did this. If you didn't, if you came in late, start downloading the QCAD, or we have some USB sticks here with the images. So the first thing we're going to do, we're going to start with, is we're going to set up an HTTP server. And the reason for that is that a lot of the operations will require you to pull from a server. So you're going to want to have a server running locally, and then you can just paste in the address wherever you need to do that. So I'm going to do this right now. So actually, I guess the first thing you should do is create your own, create your own, I guess, working directory for where you're going to do all these operations. So I have one here. It's just in my DevCon file. So let me remove everything here. You don't need to be rude for most of it. Some things you still need to be rude. I'll talk a little more about that. So let me just make sure also that I've reset my server. Okay, so I have nothing here. That's great. Well, this is the stuff that I wanted you guys to pull from me, but we couldn't. So we're going to start with making... This is the actual directory we'll be serving over HTTP. And then... So you can use whatever HTTP server you want here. I'm using, in OS 3, there is a bundled HTTP utility called trivial HTTPD, which is really nice because if you have OS 3, then you already have it. Or you can use Python. Python has the simple HTTP server thing. I like to use OS 3 because I'm just used to it. So we're going to expose over... Of course, you can change the fields that make sense for you to change. You can change. You're free to do that. And then I'm going to use logfile- which just means output the logfiles on the standard output. And we're going to serve this directory. Okay. So now we got our stuff going. So I'm going to break out this pane into its own terminal. Again, tmux is awesome. Okay. And then... Okay. And then this is... Now we're back. So just so I didn't confuse here, this is the pane that I had where I set... I started the server. And then this is the pane where... The window where I have the text file and where I'm going to do all my work. So in the new terminal now, we're going to clone the Fedora atomic... GetRepostStory. So let me go again to my Defcon folder. And then clone that. So I get some background here. The Fedora atomic repo is where all the metadata information is stored for how the RPM OSP commits are going to be created. So this is actually the stuff that Fedora release engineering uses to create the new OSP commits. All right. So, okay. So this is definitely not a good sign. Let's see here. Make sure this exists. Yeah, the network is just... Yeah, we're... Okay. Try it. It looks like you've got... Yeah, maybe not. No, try it seriously. It's not... Maybe Google... The website is not going to let HTTPS traffic through. Google will automatically redirect it to HTTPS traffic. And yeah, who doesn't? And if you haven't been in a while, no way of using it. It's true. I think the whole system is just... Yeah. You can use a third of the cable. I can't, but I need everyone to be able to pull the repo. So, wow. I didn't realize how bad it was. The text file... Oh, sorry. Let me just make that more clear. So, if you go to this URL, you will be able to get these slides and these links are links to the text file. Right? So right now we're doing step one, or part one. Which URL? My... www.jlbond.com slash dev.com slash slides.pds. So... Yeah, this is... So I guess the... So no one can actually... Maybe I'll discredit over USB then. I have the RPMs... I have a mirror locally, but... Yeah, if you can't pull... Let's see if over... If I have the wire, if you guys can ping my computer. Let's try that. Oh, wow. I think they said don't use hotspot, right? So I got an IPv6 address, which is... Oh, okay. Can someone actually try to ping this address? Let me... That's... Let me just... You can't? Okay. Well, then this is an issue. I guess I'll do the steps and then you guys can follow along later. Yeah. I mean, I knew it was going to be a problem with everyone hammering the Federal Service, but I didn't think it would... Even just getting the Git repo would be an issue. That's the wrong name. How so? Yeah, it's not... Oh, my bad. No, this is the... This is the wireless, but the wireless you definitely can't ping. So I plugged it into the wire, but I guess you can't ping the wire also. So... Yeah, that sucks. I mean, I can transfer the Fedora Atomic repo, but like Adam said, when you guys are going to have to be pulling down a bunch of RPMs, it's just going to be a showstopper. So... Yeah, okay. So I'll just do the steps. Not a big deal. You guys can follow along. You can see the steps that I'm following, and you can do it later when you have a proper internet connection. Everything is on my website, yeah. So I'm literally just following what's here, although I'll be adding a lot of... explaining a lot of stuff along the way, but the actual steps are all there, so... So let me make sure I have another copy of the Fedora Atomic repo. Oh, you were able to get the... Yeah, using my phone. Yeah, yeah, yeah. But then there's also all the RPMs that go along with it. So that's... Let me... Otherwise, but this will take more time. Okay, so let me just do a quick check here. Yeah, I guess we can do that. How many people are actually doing it on their laptop right now? Okay, so maybe we can transfer the RPMs, and it's not that big a deal. Can I use your keys again? Sure. Where's the other one? Oh. Oh, okay. So I'm going to have to delete some things here. Well, we don't need this anymore. Stick is big enough. But you can take off the Dev Studio stuff. I can take this off? Yeah. No, no, the new OPS installed and ready for step two. These are virtual machines on virtual... I can remove both of those? Yes. Okay. Open shift virtual machines for... Awesome. Thank you. There is still not enough space actually. Wait, that doesn't make sense. We just freed up... Oh, right. I have to empty the trash. Right, right, right, right, right. I'm not used to trash... to the trash can when using the Kubernetes line. So now we're in business. Awesome. Where's the other key? Just to say, I just put up the... I cleaned that one and I put up the Fidora Automation Directory too. Awesome. Awesome. Good stuff. Let's see if we can transfer it on two sticks at once. So actually you will need also the... For those who have internet, Let me... Oh, you were able to get cloned eventually? Okay, you're still going to need the stick for the actual packages though. I think we have enough space, so it's okay. Oh yeah, I'll show. Okay. So this one has atomic, this one doesn't have atomic. Okay. I think that's all you need. Let me just double check that that's everything you'll need. Oh right, you'll need... Well, this you can probably pull from my... It's just a small script, so you should be able to pull that. All this is okay. Okay, now let's do part two. All that is also okay. Yeah, it should be fine actually. Okay, awesome. So let's look at the status. Did everything copy? Oh, almost. Or no, it did, right? Oh, very nice. Do you want to bring it to the back? So you'll need the three items. You'll need the Fedora Kickstart. Fedora... Sorry? I just cleaned the rest of the stick. Okay, Fedora Kickstart, Fedora Atomic, and the folder called packages, because that's where all the RPMs are. Fedora Atomic. Okay, then you don't need. And then the packages repo. Packages directory. It's all the... You should just use the stick. Or you'll be doing later, you mean? No, I mean I'm on the cloud instance right now. Okay, then you can just let it download as usual. This is only... I kept it as a mirror in case the network was down, but... Yeah. The packages... It's just a bunch of packages from the Fedora repos. Yeah. I also need to... So we did this. So right now we're here. Okay, so I'll give you guys some time to distribute the stuff. Okay. That's really weird. Yeah, I don't know why. But okay, so everyone's good now? Everyone has all the files? Almost. Almost? Okay. Let me go get the slides again. So if you go to this address, you'll get these slides. And then on that slide, the first part here is the link itself. So if you click on that link, you'll get the text file. For the slides? Oh, that's... Oh, no, no, don't go to my website. Type in that whole address. I don't have a website. I'm just using S3 to store some files. Okay. So I'll just go ahead. And then if anyone has any problems, just let me know. But okay, so now we have the Fedora Atomic repo. And if you look in the repo, it's not very big. There's not that many files. So these are all... This is all the metadata that you need to... Well, not all of them. You just go over the ones that you do need to do the rpms3 compose. So the major file here is this one, which we call the manifest. But really it's just a JSON file. And this is what tells rpms3 exactly what you want to put inside of your atomic host attribution. So this is the major, major file. And then you have repo files to tell rpms3 from where you want to fetch the repos. And then a few other minor files here that we don't need to worry about right now. But let's open the JSON file. So I'll just go over the structure very quickly. So the ref is the actual name of the branch that you're going to be committing to. So OS3 supports branches, very much like Git. It does branches. So this is the branch that we're telling rpms3 to put the commit into. Right. And then of course it'll substitute in the architecture for which you're composing here. You got a question? The one that I just wrote down says Rohan instead. Oh yeah. You have to check out the F25 branch. Yes. Sorry. Thanks for bringing that up. And then here at repos it'll tell you, it'll tell rpms3 from where you want to fetch the rpms. So a very interesting part here is this part packages. So this is literally where you're telling rpms3 what packages you want to have in your distribution. Right. So here we have Kubernetes. We have cockpit. We have Docker. So this is all the stuff that constitutes a basic distribution that you need to run containers. So what we're going to do here is what do I have in my... So for front we'll pretend as if we're creating our own distribution. Right. So obviously we're basing everything on Fedora data, but let's just pretend like we're going to have our own DEVCON distribution. So I'll change the name here to... And of course you can change the name to whatever you want, but I'll just use DEVCON here. DEVCON slash 25 would I use standard. Okay. Standard. All right. So that means the branch that I'll use will be named DEVCON 24 standard. And this is very important because this is the actual branch that your clients will see. Right. Or your gate to the world, or your face to the world pretty much. And then... So for now that's all we're going to change. Okay. So now we can actually start our very first compose. So for the first step is to create your OS free repo. So to do that, you do OS free dash dash repo equals and then wherever you want to store the repo. We'll be storing it under where we have the HTTP instance running, which should be under SRV DEVCON. So I'll put it in SRV DEVCON repo and then in it dash dash mode equals archive. So OS free supports... Oh yeah, this is kind of low. Can you guys... Can everyone still read the last line? That's okay. So OS free supports multiple types of repositories. So there's the archive type. There's the bear type and the bear user. So the difference is that on your... So actually, am I connected to my... I am not. So the way OS free works is very quickly is the reason why it's so efficient is that it uses hard links to create new checkouts between routes. For example, if you're doing an upgrade, most likely user bin LS is not going to change between one upgrade to another because LS is pretty slow moving as a project. So it would be pretty bad to store two LS just because you have to have LS in the new route also. So OS free uses hard links to make sure that you only have one copy of LS in the OS free object store and then you just keep hard linking for however many deployments you want to have. So the bear repo, the reason I said that is that if you do dash-mode equals bear, it means that it'll be the actual files from which you can create hard links. So this is what when you have an actual atomic host running, you're going to want the OS free repo to be a bear repo because you need to create hard links from that repo, right? But here on the server side, we're the content providers right now, right? On the server side, you just need to be able to push down those files. So you don't care about creating hard links from the files. So the archive mode is pretty much telling OS free, compress all the files, make it really easy for people to pull to their computer. So we created the repo and now finally we can actually run RPM OS free. So the first flag you have to pass is the actual repo you want to commit into. So that's this repo we just created and then compose tree, of course. And then RPM OS free supports cached directories, but actually, oh, right. So you guys are not using the Fedora repo, you're using the repo from the stick. So as another step, so you have the packages folder, right? So what you'll need to do is you can do DNF install create repo. So you have to create a repo from that packages directory that you got from the USB stick, right? Because we're not going to be using the repos from Fedora because the server is too slow, yes? But you guys all have... Whatever you're talking about. Oh, okay. I think they have network, but it's just that it's very slow. But hopefully you can at least install that package, right? Can you guys confirm that you can install create repo underscore c? Working, not working. Downloading metadata still. Is downloading? Is it not connecting to the Fedora server? Okay, so it's sort of spotty, yeah. Okay, awesome, awesome. Okay, so now... So this is a bit not how I foresaw it going because I assumed everyone would have the stuff. So you have this folder, right? The packages folder. So just place it to... So move the packages folder into the slash srv slash defconf thing that you created. All right, let's do sudo on that. So actually you don't... Yeah, so I'm just going to do... So now it's in your defconf folder, right? And now you're going to need to create the actual... Yum metadata. So you can use... It's very simple. You just do create repo underscore c and then dot enter. And it's going to scan all those rpms and create a yum repo from it. So now that you have that, we can go into... So if we go to... Into your working directory where you're doing all your... So where you checked out Fedora Atomic, for example. Now we have to tell our VMware Street that that's where you want to create those... That's where you want to fetch the rpms, right? So for that, you go in the manifest. You edit the JSON file. And in the JSON file, just remove... Yeah, just leave Fedora 25 only. Remove Fedora 25 updates. Everyone following me so far? Am I going too fast? In case you have good internet connection, just leave it. Yeah, so you transferred it from your USB stick, right? Oh, you didn't get that other... Here, this... I know I did. So other than Ben, is anyone... Are you okay? Oh, you created a repo. Okay, awesome. Okay, so now we're telling our VMware Street we only want to look into Fedora 25. We want to change the definition of what Fedora 25 is to make it point to our new repo instead of the Fedora server. So let's remove all of Fedora 25 updates because we don't want it to fetch from there. Actually, you don't have to remove it because we removed it from the JSON, but I'll remove it anyway. We're not going to do a GPG check. It's enabled. You can be pasting in your own server because, right, we're hosting the RPMs ourselves. So you can do local holes. And for me, it's on port 35,000, so I put 35,000 here. But whatever you've used is what it'll be. And then... Why don't you do file... Yeah, you can do file. That should work too. Yeah, good point. Don't need that. Let's just keep it simple. So something like this. Packages. No, packages is where the packages are. But metadata is at the same level as the packages directory. So you moved it out because I have a package. Okay. So here, you put whatever path for where you ran create repo dot, right? I'm sorry, this is very complicated. So now... Let's see if it works. So we didn't run the compose yet. Okay, so let's do... Defcon repo. Compose tree. And now... Well, actually, since it's local, you don't have to use a cache directory. But if you were pulling from Fedora, from the Fedora service, for example, you could use dash-cache-dir to cache the RPMs so that the next time you do compose, you're not pulling down all those RPMs again. And then you tell it... You give it the actual JSON file that RPMs are going to be using. So actually, let me go to Fedora Atomic. All right, so this is Fedora Atomic. Get that JSON file and start creating an OS tree repo. OS tree commit. So the cool thing is, because it's all local, it completely skipped over the downloading step. And it's going straight into installing. So right now, what it's doing is it's literally doing a yum install into this tab directory. So first you have to run this command, right? To create the actual repo in which you're going to do the compose. And this repo has to be where you're serving HTTP. It has to be within that directory. Because later on, the client will be pulling from that repo. After OS tree repo? After the morning for the archive. Sorry. After this one or this one? After the archive. Oh, then you can do this. Based on your... Apparently you have to create that cache directory. Yeah, but since you have it locally, you don't... Oh, you're doing it on AWS, right? Yeah, okay. So create... Yeah, sorry. That's the step that I'll add it here. It seems like I have to fix record to not spill so many error messages about missing product modules. Right, so the interesting thing here is a bunch of errors that shows up when you run the compose. But the reason is that packages, when people create packages, they expect it to run on the machine where the package will be used, right? But we're sort of misusing that right now because we're installing the packages inside of Chirrut, essentially. But the packages... The package is confused because it's... For example, it tries to do system CTL restart service, but there's no service running because we're just... All we want is for it to output its files and that's it. Does that make sense? Yeah, yeah, so let me... I can fix my software to not spill. So the directory where you have your Fedora Atomic... Do you have the Fedora Atomic repo? Okay, so in the Fedora Atomic repo, you edit the JSON file to only say this, right? And then in the actual repo file in this file, you change it to point to essentially the local repository that you got from the USB stick, the packages that you got. So where you run create repo underscore C. So you put whatever path to where you ran that. Because you're basically telling RPMistry fetch the RPMs from this young repo. Does it know what this cache is created? Well, since it's local, it doesn't... There's no point in caching, right? But if you were pulling over the wire, then you would use dash-cache-dir RPMistry or wherever you want to save your cache, right? Okay, so right now it finished installing all the packages and all it's doing is committing it into the OS for repo. So because it's an archive, we did dash-cache-mode archive, it's compressing all those files using gzip2. It's compressing all those files so that it's faster for clients to pull the files. Then you should be able to run this. Or I guess, well, you don't have to do this if you have the cache local. Then you should just be able to run this. So does everyone use libvert here for creating VMs and stuff? Okay. When the dev.com repo directory should be supposed to be created? This one, you mean? Yeah. Yeah, we created it here, right? Okay, did you create this? Well, okay, this is where I have my HTTP server running, right? But if you have another directory then... Yeah, just change that. Okay, okay. But it's giving an error when you try to... So that init should create the repo, right? Yeah, exactly, yeah. Did it give an error when you tried to init? No. Maybe sudo... No, no, no, you shouldn't have to sudo, no. You have to sudo to run the actual Rpms3 command. Yeah, okay. All right. Unfortunately, right now Rpms3 requires you to run as root. But that's something we're trying to change. Eventually, you should be able to just run it as non-root and everything is done unprivileged, which is going to be awesome. You cannot change root. Sorry? You cannot change root. You can't change root? You cannot change root when you're not root, right? Yeah, but... And run and drag it, for example, inside that thing. You can if you're using username space, right? You can be non-root on your machine, but you can be root inside of a container if you're using username spaces. So that's hopefully how we're going to work around it. So it's almost done committing. And the reason why it takes so long here is because it's the compression, right? If we weren't compressing, it would be much faster to commit to the repo. So is it composing for everyone right now? It's already done? Done. Really? So your machine is faster than mine, then? I started earlier. Ah, okay. Okay, awesome. So now, let's actually create a VM that's going to pull from this repo. So I made a little script to make it easier for you guys to create VMs. But I mean, you can check out the script, see what it does. It's very benign. Nothing... You do have to run it as root though. But other than that, it's really benign. But no, I mean, in all seriousness, it's very straightforward. It just does... It creates... Because the way it works for atomic holes is it uses cloud in it to actually provision the virtual machine with, for example, username and password. So I make that task easy for you. I just create the cloud in it metadata for you. And then I create a new KUKAO image from the Fedora image. And then we actually do the virtual machine installation. But if you want, you can use the GUI or whatever to create the machine. But you still need to have that cloud in it metadata ISO to attach to the virtual machine. So it might be easier if you just use the script. Use the script? Ansible. Oh, yeah. Or you can use Ansible, yeah. Find the script that you created. It's here. It's on my website. Clock. Yeah. So I'm just going to call it devconfnode. And then here you would put the path to where you have the Fedora 25 IS KUKAO image toward the very first time we passed around the USB 6. Don't tell me. Oh, I do have it. Okay. No, it's okay. Okay. Let's create it. And then very helpful script. It tells you how to find the IP address for your new VM. So it's still booting up. So it doesn't have an address yet. Okay. Now it has an address. So let's log in. All right. So here it says you may log in with cloud user slash. So username cloud user password atomic. Yes. Atomic. So now we're in a VM. Right. So the VM is still vanilla Fedora because we haven't done anything yet. We haven't actually pulled any of the of the data that we just created with RPM history. So if you do RPM history status in here. So it's just using the regular Fedora repo. So now we need to actually tell RPM history from where to fetch the new history commit that we created. Right. Do you have a question? No, I'm still, you know, I'm still building. Okay. Okay. Okay. So first we have to add the remote. So we do a remote ad. And then we're not using GPG ver. So that's another thing. Everything here, you can add GPG signatures to make sure that it's, it's, it's valid. I'll just call the remote. So this is the same as in git, right? Git remote ad, whatever. And then here we'll put the address of the of your host. So this is your, this is going to be the address of your virtual machine bridge. So if you go like IP ADDR. So it's this address, right? So it's the gateway for your, for your VMs. It's the, it's the bridge that Libvert creates for your machines to access the internet. And then I'm exposing it on 35,000 and repo. Right. So I added the remote. And now finally I'm going to tell RPM history to rebase on that new commit that we just did. So what rebase means is that I'm telling RPM history, I don't want to follow the stream from Fedora anymore. I want to start following this new stream, the stream that we just created. Right. So now it's receiving the data. So obviously it's really fast because it's just pulling from itself. I mean, it's pulling from, from the, from the hypervisor, which is my laptop for when inside the VM. Right. Right. And don't worry. We'll show you how to erase the VM later if you don't want to have a VM hanging around after the presentation. It's very easy. Okay. So now if we do RPM history status. Right. So it's telling us, right? This is where we're, so the star means this is where we're currently sitting. This is the actual tree that we have. But if we reboot, this is the actual tree that we'll be using. Right. And it's a DEF CON 25 standard. So let's reboot. So it's always annoying when you reboot a VM and you have to wait for us to start. So I actually wrote a little utility to make that easier. But I actually already started working. So it's didn't showcase really, but now if we do status again. So, okay, we're in the deployment. So yay, we created a working or a street commit that you can boot into that's awesome already. So now you can already picture. Maybe we won't do it because we spent a lot of time on getting stuff set up. But if you go back to, so let me, let me exit the VM. Right. Any fit or atomic repo. So if you look at the JSON file, so everything, anything you change here when you create the next compose, it'll be reflected in the, in on the client side. Right. So if I do, if I do add a new package, for example, let's say I really like Vim. So I can do Vim enhanced here. And then if I rerun the compose and I, and then I, in the client side, I do our chemistry upgrade. It'll receive that new or street commit in which the, in which Vim is installed. Or for example. So another, so let me just show you this field because it's an interesting field. There is a field called post process script, which pretty, pretty much tells our chemistry. Once you're done running all the install scripts for these packages, once you're done installing everything in the charoute that we created, run this script at the end. So here we're telling it run this trip, the tree compose dash post dot sh. And it's in the same, it's in the same repo. So it's right here. So here you could add other stuff here. For example, let's say you want any time someone runs Emacs, you want them to be redirected to Vim. So you can do Ellen dash s Vim user bin. Sorry. Yes, user bin Emacs. So Emacs won't be installed on your tree. But if someone tries to do Emacs, they'll be redirected to them. So, so I'm not, that was part of the original plan. But I think we'll just skip ahead. It's okay to have just one commit. It's fine. So now we can go to part two, which is a little easier on the configuration side. There's nothing you guys have to set up anymore. You can more or less, oh, my bad. You can more or less follow along. So the big, the big problem we have right now is imagine that compose process, right? Imagine if you're running that 100 times per day, let's say every time someone pushes a commit on Git, you have some hook and let's say Jenkins or something that's that just runs a new compose. That's great because it allows you to, to test out new features and new packages as soon as they come out. But at the same time, you don't want your clients to have access to that kind of data. So you need a separation. You need to be able to have the production repo and then your developer repo where you do all your testing and stuff. And then once you're satisfied with a certain commit, you're like, okay, this is certified goodness. All the packages are what they're supposed to be at. They're all the right version. Then you can take that commit and transfer it to the production repo and so that clients can see it. And at that time, you can also add, for example, release notes. You can give it a nice subversion name. So this is the actual stuff that your clients will see, right? So it's important for it to look polished professional or if you're doing, even if you're doing for yourself, you know, you want it to be something that you can track later on. So, right, so the issue right now with, with how we're doing it's right now, we're literally just committing into the production repo as the equivalent. It's as if you're doing a Git push into production every time, right? Or Git commit into production. It's definitely not what you want. You want to be able to have some kind of QE process. And our industry also supports this thing called static deltas, which means that it has a very efficient way of going from one, from one commit to another commit. So when you do our chemistry upgrade, instead of pulling down all those separate files, it can, it can pull down only the diff between what you currently have and what you and the new files that you have at a binary level, right? So let's say this, the huge, for example, the init ramfs, most of the time it's a huge file, right? But with, with static deltas, it'll only, it'll look at the binary and it'll only download the part of the binary that has changed. Good. Oh. So, okay. So we only have one commit here. So I made this originally assuming we had two commits, but it should work just the same. So right now we have, so if we go to the repo that I'm exporting over HTTP, we have this repo, which is the OS3 repo. And if you go in repo and you do OS3 refs, this is the commit that we just created, right? That our client pulled down. So let's pretend like this is going to be our dev repo. This is a repo where you're going to be committing over and over every time a new package comes out you commit because you want to be able to test it out as soon as possible and start the QB process on it, right? So to reflect that, I'm just going to rename it to repo-dead. Okay. And then I'm going to create a new repo called repo-prod. So repo-prod is the actual production repo that the clients will be pulling down from. So as I, as I say here, right, we'll be using separate repos, the dev repo and the prod repo. And within the dev repo we'll be using branching to sort of mirror how content gets promoted, right? As your QE process goes through the data. So, so the, you can have, for example, a fresh branch. I call it fresh because it's just off the, out the oven. And then if that, if the commit passes some kind of test, so maybe an automated smoke test or something like that, then you can commit that, you can promote that commit into the stage one branch, for example. And for example, now you might have actual humus involved in testing and checking that the commit is the same. And if at that passage you put in stage two, et cetera, et cetera, until you have like the final branch where this is like the stage branch that mirrors what your prod is going to be, is going to look like, right? And then at the very end, you actually transfer it into the prod repo in the final, in the final branch. So I'm just going to quickly show you how that works. So maybe if I sit down here. So right now we have a dev repo and a prod repo, right? So the prod repo is empty because we just created it. The dev repo is the same repo that we had from part one, right? So if we're going to dev repo and we do OS3 refs, this is the ref that we created right now, right? And you can also do OS3 show to actually see what the commit is. And it'll tell you, so version 25, we didn't put any subject in it, or IPMOS3 didn't put any subject in it. Now let's say you test this. So let's say you have like on your test story before you want clients to be able to pull this in your production, you have a test VM, you pull down the stuff and you start running code, you start running a test on it and it passes. So you want to promote it to the next branch, the, let's say the stage-1 branch. So to do that, we do OS3. So normally you have to do the dash repo and pass the full path to the repo that you're manipulating. But because we're already in that directory, OS3 will automatically use the repo that you're in. So we don't have to actually pass dash repo commit. And this is the, this is where you tell it what branch you want to commit on. So let's use stage one as like our next, our smoke tested branch, if you will. And then here we tell, we're telling OS3 in that commit, I want you to put this data. So to do that, we do dash dash tree equals ref equals the actual commit checksum that you, of the, of the commit that you want to promote. So here is this. So with Tmux, you can easily copy. But if you don't have Tmux, you can just use the mouse to copy the checksum. Really, you could do devconf slash 25 slash standard, but really you want to be explicit here, right? Like you've tested this exact commit that has this exact shot. So you want to paste past paste the command using the same shot, right? You want to be sure that's the exact same commit that you, that you, that you tested. So I paste this here. And now it takes some time because it's a huge commit, right? It has a whole OS in it. And of course, it says permission denied because so you might have to do that too, because we ran RPM ministry as root. So all the files are owned as root, but we're not root right now. And the reason why you have to do, run the RPM ministry compose as root is that we have to tell our, so RPM ministry underneath just uses RPM and RPM, the only way for RPM to install if a package into a charoot instead of directly on the host is for it to do charoot. But for you to be able to do charoot, you have to be root. So that's the, that's the only reason we have to do to run RPM ministry as root. But eventually we'll, we'll work around that. So right now let's try to commit again. Okay. So now we have a new checksum, right? So this checksum is a checksum. It has the exact same data. It's the exact same oyster, except that it's on the stage one branch, right? So at that point, you could, you know, do the same thing again, you can do some more expensive tests. So the idea here is, I don't think I made that explicit, but as you move through your promotion, right? Is you could have more and more expensive tests that you run. For example, you don't want to be able, you don't want to, you don't want to run some expensive test suite. If something really basic went wrong, right? So you may, you might wait for that until the very last part of your pipeline. And yeah, pretty much you move along the commits and just keep running more and more expensive tests on it. Start involving actual humans in the test. So of course you can extrapolate here and keep going and do a stage two, for example, and pass in the new checksum that you got from stage one, et cetera. But we're just going to stop at stage one here. And now, so now let's say you're ready to actually push this into production, right? So you have your commit, you're happy with it, you want clients to be able to access it. So now what we're going to do is we're going to transfer between two OS3 repo. So to do that, we can go back up here. So again, we have our dev repo and we have our prod repo. So you do OS3 dash dash repo, and then here you pass in the path of the prod repo, right? And you say pull local. So pull local means we're not fetching anything over the wire. It's all on the same computer. So I want you to pull data from another OS repo that exists on this very same computer. And then here, and then after you write pull local, you paste in the actual path to the other repo that you want to pull from. So here's the dev repo. And then here you paste in the actual checksum, right? So the checksum we want to paste here is this one. It's what you got from after committing to stage one. And then here, right, so you pull that commit into the prod repo. But it's just a commit, right? It doesn't actually have any metadata associated with it. It's not associated with any branch yet. So if I do refs, there's no refs in the repo yet. So now we're going to create the actual ref. So again, or street dash dash people commit. So now this is the actual branch that your users will see, right? So you want it to be something standard, something official or whatever you want your clients to see on their server. And then here tree, we paste in the same commit that we just did the pull local on. And then here you can actually add the metadata that you want. For example, you can set it to be version, so I think we had version 25. So let's make it version 25.1, for example. And then you can add a subject. For example, this is, so it's kind of like a commit message, right? You have the subject line and then you have the body of the commit message. So we're going to say release 25.1. And the body, you can say like in this release stuff was updated. It's great. And now we have our final commit checksum that clients will actually be pulling, right? So now if I go back into my VM, right? So you can do SSH. So it should still be running atomic pseudo. So if you remember, this is still running the same commit that we had, the very first commit that we created. So now we're going to pull the new commit. But I renamed the repo from repo to repo prod. So I have to tell OSD to use the new file name. So to do that, I can just edit the OSD configuration file. So here's repo. It became repo prod pretty much. And then that's it. We can just do an upgrade. Upgrade. And the server returns 404, right? Okay. Let's see what happened here. So we're on repo prod. And let me just make sure here. Yeah. So repo prod is okay. So what did you try to do? So this is the other cool thing with OSD, HTTPD. It'll tell you, it gives you like a nice... Ah, right. Okay. Conf 25 standard. Okay. I made a typo in the ref. I wrote the de-conf instead of defconf. So let me recommit that again. Defconf. Okay. And I fight SSH back into the machine. Atomic pseudo. So I should be able to upgrade. Really, it's the same commit before and after, right? So it doesn't actually have anything new to pull. It's just pulling metadata at this point. 24 modifiers. Yeah. Yeah, it's just basic things that change in between commits. There's nothing that big. But if we now do status. So you can see we have our new commit here, 25.1. And if you do OSD show. Defconf 25 standard. Right. It gives you the version number. It gives you the subject line. And it gives you the actual body of the commit message. Just like in git, you would do git show the commit. And it'll give you the commit message. And we can reboot into the node. Boom, we're in our new commit. So I know I went pretty fast here, but because I really want you guys to do the last part, which is I think the more cool part where you can actually create your own image that you can then, let's say you can upload it to AWS and start up your own VM there. But the idea pretty much here is that you can, it's just an example of how your release engineering process could work. You can have branches to promote content. And then at the very end, you do pull local into the production repo. And you can add your release nodes, your final version string and stuff like that. What was that? SSH? Oh, that's the little utility I wrote. I hate waiting for SSH to come back and not knowing what it's doing. But later, if you want, I can give you a link to the utility. So now we can get to part three. So don't worry if you didn't do all of part two. You can still do part three. Just fine. So part three will actually be creating the QCAP2 image. Just like the QCAP2 image we downloaded from Fedora, we'll be able to create our own and boot that up into a VM, right? So the reason why you may want to do that is that you might not want to be using, for example, Fedora or CentOS QCAP2. You want to be creating your own QCAP2 derived from your own RPMs, right? So the image we'll be creating, it's not an installer image. It's just a cloud image. So you boot it up and it's already running. Now, if you're using bare metal, for example, you're going to need to use an installer. So you can also create an installer image. For that, you can use the RPMs tool box tool, which will create the installer for you. But to make it easier, we'll just be creating a cloud image using ImageFactory. So who's heard of ImageFactory? So ImageFactory is a really cool tool. It pretty much allows you to run the installer for any operating system and then save that image and optionally even push it out to, for example, Rev or OpenStack or whatever you have, right? So we'll be using it to run Anaconda and install the OrestreeCommit that RPMs are composed. And then we'll be able to create the QCAL2 from that ImageFactory image. So the first thing you have to do to run... So who's used Anaconda? Right, so the way Anaconda works is that it boots it to its own little OS, right? It has its own boot ISO. And the way it works is it has a... Well, it depends how you run Anaconda. I mean, there's so many ways you can run Anaconda, but you need a boot ISO that has the whole file system for Anaconda in it, right? So that ISO itself is composed of RPMs. So we have to first create that ISO so that we can boot Anaconda so that Anaconda can then install our RPMs OrestreeCommit, right? So the tool to... And that's the same tool that Fedora, CentOS, REL uses. It's called Lorax. And Lorax pretty much allows you to create all the stuff you need to... For example, if you wanted to do a pixie boot, if you just want to do a regular boot, it'll create that Anaconda boot ISO. So we're going to run Lorax here to actually create the boot ISO. So I'm doing it in the container because this annoying bug, or it's not a bug, I guess, it's a known issue with Lorax that you have to disable SE Linux if you want to run it. And I don't want to do that on my host, but also because the system that we're installing is Fedora25, but my laptop is Fedora24. So it would probably work, but just for doing it the proper way, you would normally run Lorax on the same distribution that you're going to be installing because you want it to pull the same packages that are going to be used on your machine. So let's do Docker run. So I have to run it privileged because Lorax has to do mounts. And then here I'm going to pass it the actual directory that I'm serving over HTTP. And then it's just based on Fedora25. Okay, so now we're in here. So let's install Lorax. So it's kind of interesting. Lorax is in some ways similar to RPMistry. It just does like a yum install, dash dash install root into this root. And then it takes that result of all the RPMs that are installed into it and creates an ISO from it. And obviously in those packages that are installed, there's Anaconda in it because that's the installer. So thankfully the network is not too bad. So one, another funny issue with Lorax is it needs to use loopback devices. But in Docker right now, if you want to do loopback, you need to make sure that you have the proper amount of loopback nodes. So if you don't have, so if you do, one way to check is if you do L set up dash dash fine. And this is dev loop 2. And it's already installed. I already have dev loop 2, so it's fine. But if I didn't have dev loop 2, then I would have to do mk9 dash m 6 6 0 dev loop 2 b 7 2. So if you've never used this before, it kind of looks arcane, but it's pretty much create this new... Sorry? Yeah, it's not necessarily Docker's fault, but yeah, it's just the nature of the beast, I guess. So I'm running it as privilege. So I think privilege is just mounts the same depth. I think it's the same as my... Yeah, like if you go... Yeah, see, this is my local disk. Oh, yeah, I think it's... Yeah. Okay. Well, I mean... We can talk about it after, but... That's why I stopped cooking, so... Okay, sure, okay. So now we can do the actual Lorax run. So here you can give it a product name. I'm just going to call it devconfatonichost25. And of course, then version 25, release 25. I mean, you can put in whatever you want here. You can use a cache directory, but we don't have to do that because I'm going to run it once and it's going to work fine. But if you don't know if it'll work the first time, it's good to do a cache directory because it takes time to download the packages. And then here you're passing in the actual repo. So those are just regular YAM repos from where you want Lorax to fetch the RPMs that it's going to install into the boot ISO, right? So I'll just cheat here and use Tmux, right? And then I can do this. And then you have to sell it where you want to output the data. So I'm going to output it into the same directory that we're exposing over HTTP. And I'm just going to fetch the metadata. So I guess since this takes a while, I'll just show you, give you a bit more information about Lorax. So pretty much it'll run through a series of templates that tell it what to do. So actually if you go, do I have Lorax on here? I do, okay. So if we go to... So this is actually telling Lorax what it has to install in the boot ISO, right? It's pretty cool. Install Anaconda, install Tmux. If you ever use the Anaconda command line version, the text version, not the GUI version, you'll notice it actually uses Tmux to do it as its interface. And that's why it needs Tmux here, GDD, and RPMOS 3, right? Because RPMOS 3 is the one that knows how to take that commit from the remote and create the new deployment on your machine, right? So it needs RPMOS 3 in the Anaconda image. So that's going to take some time actually. So does anyone have any questions or... Is that a little too... Was I going too fast? I went fast a bit because we spent a lot of time at the beginning. And I want you guys to be comfortable with what I'm doing here so that when you look at the steps later, you'll know what it's talking about and what it's doing. So is anyone running... Are you running low-racks right now? Oh, yeah, that's painful. Surprisingly, it's working pretty well here. Oh, I'm on wire, that's why. Sorry. Maybe I shouldn't have said that. Maybe. Just go to the next room. They're in the Wi-Fi stuff. I don't know what to record. And we take all of them out for Wi-Fi. And we get more bandwidth. Thank you. I was doing really well. One of the instructions I had to start over. Okay, which instruction did you mess up on? Oh, I just didn't edit the... Oh, the JSON? Okay, okay. It probably shouldn't have all been in there. It's running inside of the US. Okay. Are you... Is it a regular... What kind of instance did you bring up in your MI? Change of liver and start. Okay. Because I know you can do liver in a container, but... Okay. The purpose of the liver thing is just to test the treaty. Yeah, yeah. It's just to bring up a VM to pull down your tree and make sure that you can boot into it. Okay. So, conceivably, if I went far enough to actually build up an image... So, conceivably, I could test it somewhere else. Oh, yeah. Yeah, yeah. You can upload the image to AWS and do it there or in OpenStack or whatever you want, right? Obviously, you're actually building an AWS image, and you're using that to start another business instance. That's interesting, yeah. Yeah, that should work. So, but the whole process is for building the base image. So, yeah, like right now we're building... So, if I go back to the slides. So, in that process, right? Like before, we've been just using the Fedora 25 image, the one that you got from the USB stick, right? But right now we're creating our own image that you can run here that already has your own tree on it instead of the Fedora tree, right? It's actually going pretty fast, which is nice. So, right now, all those things that it's doing here is just unpacking all those RPMs into the final location. So, in terms of actually distributing the tree, what's required for AWS Tree server So, the great thing with OS Tree is that it's a dumb protocol. So, what that means is it's literally just an HTTP server. I mean, OS Tree itself is not a server, sorry. But the files that you pull down can just be served over HTTP. So, any HTTP server will do, right? And that's why we have this... For example, when you do Git and you do pull on Git, it's a smart server because Git will actually compare what you have and it will say, okay, you need these files. Let me compress these files for you right now and serve them over to you. But OS Tree is dumb. It doesn't have any awareness. You just pull down files, but that's why we have static deltas. And it's called static because you're doing it in advance. You're assuming that if someone has this commit and he wants to pull this commit, then you give them this static delta, right? But if we had a smart OS Tree server, then it could do the comparison on the fly just like Git and say, oh, you need these files. Okay, let me create a delta for you and serve it to you, right? So, for OS Tree specifically, you can... So, it's an HTTP server, but the cool thing is now it has mirrorless support. So, you can actually, instead of pointing directly to the OS Tree repo, you can point it to a file that has the list of mirrors that OS Tree will fetch from those mirrors. Just like in a regular YAML repo, you can do mirror list equals and then the list, and then the URL to the mirrors. It's almost done. Wait, it's actually faster than usual. So, all the steps that we're doing here with Lorax and then ImageFactory, this is all stuff that... So, if I go back to... Oh, no, it's frozen. But now it does that sometimes. I just got to do this sometime. Summers and it froze my computer. My computer froze, so I have to take some drastic measures. Sometimes, if I do suspend, it'll suspend when I come back, but let's see here. Because that screen is still on. Yeah, yeah, it's... But it's frozen. Like, nothing is moving. Like, it still says 206, but it's 210 now. So, was anyone actually able to run Lorax? Or did it fail on you? I didn't try it because it takes so long. Oh, okay. Yeah, fair enough. Okay, so let me see if I have an instance of the Lorax data already cached. Well, that's too bad. Okay. Okay, well, it crashed, but let's see how far it got in the... Oh, yeah, it didn't create anything. Hold on a second. I didn't save anything. Okay. I'm gonna have to rerun it, but in case we don't have enough time, I'll show you the steps ahead of it, too, just so you can do it on your own. So, once Lorax is done, it'll have a... It'll output to this directory, which is served by HTTP, right? So, the only step after that is if you go to... So, there's the... Let me show you, actually. I might have it... Oh, yeah, I do have it already. Let's come here. So, from the USB stick, you should have also gotten the fedora-kickstart repo. So, if I do... So, let me also get it from my... So, in the fedora-kickstart repo, there is this file, fedora-atomic.ks, and this is the actual file that tells ImageFactory... Or that ImageFactory feeds to Anaconda so that Anaconda knows what you want to install, right? It's an automated install. It's not interactive, obviously. So, the kickstart tells Anaconda how you want it to be installed. So, there's a few key parts here. So, one part is... If you scroll... So, most of it is standard kickstart stuff. You know, it's nothing specific to S3, until you get to this part, right? So, this is new support that was added in Anaconda, specifically for OS3. So, it's telling Anaconda, I want you to get your data from an OS3 repo, and I want you to name the remote this, and to access the repo, you will get it at this address. So, here normally you would put in the address of your own local server, the server that you're exposing will report 35,000, and you would point it to the repo that has... So, here it should be repo.prod, actually, right? So, you point it to the repo that has your OS3 commit, and then you want it to pull this right. So, here you would put normally, like, defcon-25-standard. So, that's one thing you have to change. And the other... Oh, right, well, I already commented it, but normally this is like this, because normally... So, it's kind of tricky the way it works, but when you run OS3 setup, it'll create a new OS3 deployment that you'll be able to boot into, right, when you boot your cloud image for the first time. But the URL will point to the upstream URL, if you will, will point to this address, right? So, here you have a build system, for example, using Fedora uses Koji. So, they have, like, some custom Koji email URL here. But when they want people to do their upgrades, they don't want them to upgrade from Koji. They want them to upgrade from another official Fedora CDN, or mirrorless or whatever. So, here it's just... In the Postgres, it just does a little trick where it deletes the remote, and it recreates the same remote using the official Fedora URL. So, if you had... If that was your need, too, you would do the same thing here. But I don't need it, so... That's the first thing you'll have to change. The second thing you need to change is, if you go back into the Fedora Atomic repo, there's this file called Fedora Atomic... It's called raw hide, but it shouldn't be called raw hide anymore. It should be called Fedora Atomic 25, but... So, it's a .tdl file. And the .tdl file is called a template file, and that's what you give to ImageFactory. So, that's how ImageFactory knows what you want to install, right? So, I'm saying name Fedora, because I want you to install Fedora, so it knows that it's going to have to use Anaconda. It knows how it's going to pass the parameters on the kernel command line. And then I want you to get the install metadata from this URL. And this URL is where Lorax output its data, right? So, ImageFactory will know that it has to get the boot ISO and stuff from that URL. And once you do that, you run ImageFactory. And yeah, it's pretty straightforward. It'll just create an image. ImageFactory, by default, creates images in the raw format. So, if you want to have it in QCal format, you would just do this. So, first of all, let me just show you. This is the ImageFactory command, right? So, you pass it the kickstart file that I just showed you. And then you pass it the .tdl file that I also just showed you. And then once it's done creating the image, it'll have this weird path to it. And you'll want to convert this to QCal so that you can, for example, upload it to OpenStack or what have you. So, you can just use QMU to convert to QCal. And that's pretty much it. I mean, I know I went a little faster, but I don't have slides anymore. Actually, let me just pull up the slides really quick. So, if you need any more information, you can go to projectatomic.io. A lot of the stuff that I talked about are in there. So, if you come to FreeNote on the Atomic Channel, it will help you out. You can email me if you have specific questions about the stuff that I did here. All the stuff will stay hosted on my website for a long while. So, you can always just come fetch the stuff that we did here. And, yeah, that's it. Thank you very much. Thank you very much. Thank you very much. I'm going to try some technical difficulties, but you always have them, all right? Yeah. You've been happy to have fun? Yeah. Yeah.