 Okay, hey, all right, we got a good crowd going on good. That's great. Thank you everyone for sticking around this late in the day. Let's meet part of the Design Summit. I want to talk to you today about creating custom disk image builder or Dib for short elements. And we're gonna talk a little bit about Dib. But first, who the heck is this guy? You know me from OpenStack. I've been working in the community since 2012, previously at AT&T, now currently at Cisco, where I do a bunch of stuff. And this is me from Atlanta. So let's talk about disk image builder. What? First of all, like I said, disk image builder is an OpenStack project for creating images. Not pictures, we're talking about disk images for guest operating systems. The whole project is broken up into a series of elements that can be pieced together to do lots of interesting things. The real beauty of disk image builder is that it's scriptable, it's repeatable, it's automated, and it's really, really awesome. So why do we want to do customized disk images? Obviously, not everybody wants to use the generic vendor-supplied images. You might have to do things like specifying your own NTP servers or security protocols, or you may want to standardize or access across all various distributions, instead of logging in as Ubuntu or logging in as Fedora for different distributions. So I took a survey a few months ago of the OpenStack operators community, and the responses were awesome, a whopping 93% of the respondents in the ops community customized disk images. 7% use stock generic vendor-supplied images. So the drill down was what is being done inside customized disk images? We see here, internal infrastructure requirements by far and away is the most popular reason for customized disk images. Things like name servers, internal domain, internal NTP. I know this is all stuff that can be gleaned from DHCP, but it's not always there all the time. So that's internal requirements is number one reason. Package needs, performance tuning, and the smaller ones there is file system layouts and users and permissions. So that's kind of the reason why you would want to have a customized disk image. So let's back up real quick and talk about real quick what Disk Image Builder is. It's a way to just create Ubuntu, Debian, Fedora, CentOS, REL, and OpenSUSE images initially for bare metal use. And there's an element for creating virtual machine images for use in glance, and that does things like set up the partition in the file and modifies the bootloader. You can see there's an example you can throw in various architectures. And the way you call it, the Disk Image Create, you just name the elements. In my instance here, I would be creating a 64-bit Ubuntu, but 14.04 virtual machine disk image for using glance. So one of the questions I get asked a lot is there's a lot of customizations we want to do for our disk images. How do we go about creating disk images? What are all the tools? So in Paris, I briefly talked about the variety of tools that there are available. And I think most of them come down to four major groups and Dib, Packer, VM Builder, and Box Grinder. Now, the beauty of Dib being an open-stack project is it's got that really fast, really cycle, really fast feedback loop. Bugs get reported. We are, with the exception of the week of the summit, we're getting at least one commit in master every week as per GitHub. So we are a very active project. Packer is hosted on GitHub. It's pretty active, but Mitchell is busy with a lot of other stuff, so he's not merging all those pull requests all the time. VM Builder, I checked, it's still active, but it's kind of hard to compare Launchpad and GitHub activity, so I don't know. Packer's actually slowing down a bit. Since October of 2014, there hasn't been very much activity. And unfortunately, Box Grinder is dead, long-lived Box Grinder. One of the best reasons why Disk Image Builder is my choice of a creation tool is it doesn't rely on underlying virtualization technology. Packer is a VM that imitates a kickstart or a dead bootstrap, and VM Builder is the same thing. It spins up a VM, it's great if you're on bare metal, but if you're already virtualized, you have this nested virtualization problem and now you're taking 20, 30 minutes to build a single Disk Image. So Dib, it works completely in a charrute environment along with this Box Grinder. Now with Dib, I can make all of the major Linux distributions, and there's rumor of Windows being able to work. Packer does have an advantage that NEOS can be installed because it just imitates the user action, so if it has an installer, you can imitate it, somebody type in it on the keyboard. And Box Grinder obviously only worked on Fedora and RH-based. Obviously one of the cool things is anybody can work pretty much, if you know Bash and if you know Python, then you can come work for Dib. You don't have to learn Go or whatnot. And as I pointed out, the installation method is a modification paradigm where we take the existing vendor-supplied disk images as the source, we expand it out, we blow it up, and then we charute into the environment to do the element modifications inside instead of running through an installer. So how do we work with, now I know how to do Disk Image Builder, but how do I make my own element? Where would I put my secret sauce? So obviously the first thing you want to do is keep your configuration stored in some kind of config management system so you don't have a single point of failure if your laptop dies. You want to use a proditious use of environment variables because Disk Image Builder is highly tunable. You can just run the same command and just tweak a couple of environment variables and now you're making Fedora 19 instead of 21. So it's very cool. You create your own elements that they're stored and get and you just pipe them on to the list of commands that you're passing the Disk Image Create. So for example, I have an element called mukow. I think mukow is cool. You set an environment variable called elements path that includes where your thing is located. Just like you would export path, export elements path and now you can just tack on your custom elements to the end of the command list and you'll get your thing. Let's see what comprises an element. It's a really intuitive to look at once you start kind of looking into it. There's, you can have a bin if you have any scripts that your element references and there's a really slick order of operations that happens. It's really self-explanatory. It's really well documented. Some of the stuff happens before a charoute. The majority of the exercises happens inside of a charoute and then it exits charoute to do any finalizations and clean up items. So pre-install, install, those happen inside a charoute environment. The root.d, that happens before the charoute happens. That's where you download the image from the upstream. Sorry, can't hear myself. So, sounds good. It's a really fast process. It's prone to parallelization. I can do, in my environment, I can do all the major Linux distributions and current version and previous version. All the images can be built in about six minutes flat. So it's really cool when you parallelize it. So we're gonna go ahead and do a little bit of a demo. All right. So, okay. So here's a demo I recorded earlier. Here I go doing disk image create. I'm gonna do a 64 bit. I'm gonna do a Fedora and it's a QCount 2 image and I'm gonna make, I call out my first element, Fedora and I wanna do a VM for a virtual machine. Right away it goes out, checks the upstream, the vendor supplied image. It realizes, hey, it hasn't changed and it starts going through its list of dependencies. It starts installing them packages, modifying the boot loader. It's actually updating the boot loader. It's going through the base element which is something they all have in common. Then it's going and just going through the red hat common stuff. So all of the Fedora images, CentOS and Enterprise Linux ones are all gonna get the same kind of base level of packages installed. Like there goes SE Linux, there goes, you know, drop cut and this happens every single time a damage gets built. So it's always gonna get the same set of updates whenever, so you never accidentally forget to apply patches or whatnot. It goes through, it actually takes about two minutes. Let's go ahead and skip, oh, there it goes. There goes the cleanup process. I don't install the, I didn't install the language set so it's got no need to remove it. And, you know, let me skip a little bit. Here is at the end, it's going ahead and converting it to running the SE Linux, there it goes. Converting to QCOW 2 and that's basically the end of creating an image. So the demo that I have set up is, obviously you're here to see a custom demo of some kind of image customization. So what I've done right now is just taken the generic upstream Fedora guest image and run dib on it to kind of do the slicing and dicing of the original image. So updated some stuff, updated packages, upgraded the bootloader, reconfigured it, run some of the base element stuff and it's pretty much done. But what I wanted to show you is customization. So we've gone ahead and created a file called fedora.QCOW 2. Now let's go ahead and look inside there. So there it is, the file fedora.QCOW 2. I'm going to fire up guest fish just for a little inspection. Let's go fire it up. And this doesn't take very long either. One of the common, one of the real easy things to do is what I want to do is I like to change the message of the day. So people log on, they know what's going on. So the default image doesn't come with a MOTD set. So I have this demo element and check it out. All I do here is in the post install, in the post install section, I'm going to run a script that modifies MOTD. Real simple, I'm going to get today's date and I'm going to echo out this image was created with this image builder on this date. So I'm going to go ahead and set the environment variable elements path to whatever it was before and then just append my present working directory because that's how I work. Sorry, I typed slow. Okay, so now I have an element named demo in the path. So I run again disk image create, same command I ran before. Let's do the same architecture AMD64. Now you notice I made a typo. So I append demo. Oh crap, I don't know how to spell. So let me fix that real quick. Okay, so there it goes. Now one of the beautiful things about disk image builder is that it employs a heavy amount of caching. So the first time you run it, you're at the mercy of your pipe to download the initial seed image from Fedora or Sentos or Ubuntu and all the packages. But it builds a cache, so the second time, the third time you just iterate so much faster. The first time you run an image, it may take you 20 minutes. The second time you're going to do it in like two and a half minutes. Not only does disk image builder cache the root disk image files, it also maintains an apt cache and a yum cache. So all these packages are already on my VM. They're just being, you know, hey, same thing, no need to download, it saves on your bandwidth, it's really slick. So let's go ahead and kind of skip forward to the end a little bit. Okay, we're doing the SE Linux context there. Let's skip forward a little bit. Okay, we're doing the KUKA2 conversion. And I understand, I don't have to do this, but I didn't want to fill up my VM with more than I had to. So it's converting to KUKA2 and as soon as that's done, let's go ahead and go inside the image and see if that demo element gets applied. Now, if you remember, the demo element was just a simple shell script, gets today's date, and applies a message in XEMOTD. Okay, it's done. So let's go back into guest fish. Let's boot up the image. Not a KUKA2 image. If I convert it to raw, I can just mount it. That's MOTD and this image was created with disk image builder on yesterday. So that was like the hello world of disk image elements. It's really easy to do. You can do it in any language that you like. So if I can find my cursor again, I don't know if that shows up very well. Okay, so pretty much anything that you can write in shell or Python or Ruby can be implemented into disk image builder. So again, I'll show you the elements file here. It's just a simple shell script and gets executed because I called it at the end of the command and simple as that. You can even do more complicated things like some of the things that we have internally. One of my desires was to have, I want to have every different version of Linux use the same user. Because I think it's ridiculous that some disks you log in as Fedora, sometimes you log in as Ubuntu. So what I did is in the post install, I just go in and have this little Python script that modifies cloud.cfg and makes the default user be cloud user. So now anybody who uses our Cisco images for our product offering has a consistent kind of access entry. So that's one less thing for other automation that they don't have to worry about. It's gonna be cloud user for all your Linux systems and hopefully your Windows can do this as well. And that was Python, right? Yeah, yeah, that's Python. That's super simple, scriptable, automatable, repeatable disk image creation. Disk image builder, check it out, download it from, install it with PIP or check it out from GitHub. You can do it in Vagrant, you can do it on, I think you can do it on OSX even. As long as you have the necessary, if you have Brew installed, you can probably do it. So, disk image builder, it's awesome. Check it out, start creating images. And I'll take any questions if anybody has them. Okay, so the post install stage happens inside a Cherute environment. So you're gonna use relative paths for, so if you're gonna modify MOTD, for example, you would say slash Etsy slash MOTD because you're inside a Cherute environment. Does that kinda answer your question? So you would have that as part of a package install and that is actually, for example, if we were to look at a package installation, you just specify a YAML file with the things that you want that don't come in the generic vendor-supplied images. So you would just say like, and now you have go, but I don't want go. John, I do do testing. I do very, very generic unit tests, again, assertion tests. So I have, you can see right above the line, I set some repositories, I do some IPv6 stuff, I set the time zone, et cetera, et cetera. So when I run this thing, I just wrote a test script. Now, I'm converting this to run in GuestFS, but when I wrote it initially, it's running on Ubuntu, and Ubuntu has this really nice kernel module for NBD. So what I do is just mount up a QCAD 2 image in NBD and then Cherute into the image and then just run a series of assertion tests. Like, is UTC, did UTC actually get set? Yes, some of the packages, did Krani actually get installed? Yes, did we actually change the user? Did this, the binutils package actually get installed? So that's the assertion test that I have in there to kind of test the image. Now, when I convert this, kind of real generic primitive tests to do GuestFS, that'll actually test the bootability of the image as well, and that allowed me to be more, like the semantic guys were saying, had a higher probability of success when I launched an image. Well, in this example, the tests are being run on the build server. Ah, that's awesome. That's even easier in the back. Oh, well, dependencies. So one of the things that you don't see here is, this is under the hood, everything's gonna call, but if you look at your element, for example, for Fedora you just, you set all of the things that are required by it. So things like Red Hat Common, YAM, that way you don't have to kind of pony along all the things. So in my custom element, I wanna bring in, for example, I wanna modify the order in which data sources are fed to cloud in it, so I bring that in as a dependency. I also wanna bring in the package mapper and the installer. So once you have what your image is gonna be, then all the things that you wanna add onto that can be put into the element dependencies file. It's a great question. Let me show you real quick. So my build script, for example, real easy, real easy. There's this environment variable called the release. First I set it to precise. Actually, by default it's set to trusty for the Ubuntu element, so it's gonna make a 14.04. Then I set it to precise, and then it's gonna make a 12.04. So then I, for example, it's not always code words. Fedora 20 is the default version of the Fedora element, but if I set the release of 21, it's gonna make the next version of it. And not all of the elements use the release for the version. Actually, not all of them do. Like, so Fedora, if you set the release to 21, it'll make a Fedora 21 image. But the Ubuntu elements are pretty slick because you actually use the code name. So, Dib release equals precise. Dib release equals whatever trust, what's after trusty, quantal. Okay, so you're saying if I'm working in Red Hat 6, for example, it has a different version of, one of the things that you do is you can look for the environment variable being set in your script. So you say if Dib release equals rel6, for example, then you do this custom Python 2.6 stuff that you have to do. Otherwise, you exit cleanly and continue on. So this is all like, everything is pretty much driven by the absence or presence of an environment variable that is named after the element that you're using is one method that I do because I do two versions of each distribution. Yes, Dib architecture is also an environment variable that you can query. So, disk image builder, actually that's a good question, Segway, disk image builder supports I386, AMD64, ARM and PowerPC, I think that's all of them. And this is all queried in the shell scripts by the value of the environment variable. So for example, I call it like the bindutils package. In Ubuntu it's called DNSutils, in Red Hat it's called bindutils. So this is just a real simple YAML file that just kind of maps the family and it also applies the same for service mapping. So if you wanna do any like check config kind of scripts in your post install, you just use a generic name and map the service. This is, it's Apache 2 in Ubuntu and it's HDPD in Red Hat. Can you say that again? Oh yeah, well if you have like a local source, it's kind of, this part isn't clearly documented but a lot of the times you can just kind of point to an internal URL like you have here, I have this internal IP address and like that's the name of the file, I override the. So if I was just to run the Red Hat, make a Red Hat image thing, then it would ask me to have my RHN credentials so I could log into RHN and download the file. So I set that environment variable for where the local file lives and that really reduces your dependency on externals if you have things like Red Hat for example where you have to log in and download it. How big does your cache typically get? Excellent question. So I've done a couple of demos here. Actually let's just, so for having done a couple of Ubuntu 14.04 and Fedora 20, my cache is 1.1 gig and that's including the upstream image and the apt-end slash yum caches. So if you have a disposable environment, so like if you're doing this as part of like a CI system and you have a disposable environment, are there mechanisms that you can make that retrieval a little bit faster, is that just pointing to all internal URLs for these? Natively I don't think there is a native cache primer. Can that be a file or like a repository? Okay. Otherwise your Jenkins host has to never remove its home directory. Okay, so last month I noticed that we had Red Hat 6, Red Hat 7, you know Fedora that does 1920 and 21 but we only had CentOS 7. So I said, well, why can't we make CentOS 6? So we just go and make, I copied the CentOS 7 element and changed a couple of bits, did a little bit of tweaking for the dependencies from the differences between Python 2.7 to Python 2.6 and with the help of the community, now we can make CentOS 6 images as well. It's, you know, it was really easy. We just make a new directory in the elements path. So this one right there, that didn't exist last month and I wrote that. Basically I just copied this one and just did a couple of bits of changing and the community approved it and now we have a new image. Oh, I forgot to mention OpenSUSE is also supported. I did run into huge dependency problems specifically around Python version. So I ended up having to import a whole lot of the Red Hat common sections, break them out and put them locally into the CentOS 6 directory, really, just to make for the differences in Python 2.6. That's one of the things that, because I've done a couple of different elements both internally and publicly, I thought it'd be kind of cool and maybe next week I would write some kind of initialization thing where you can just kind of like, you do like a vagrant in it and it creates the vagrant file for you. I think it'd be pretty cool to do a dib in it kind of thing and it creates the directory structure for a new element. That doesn't exist yet, but I honestly think that would be pretty easy to do and pretty helpful to get other people who just started in making their own elements. Good question. Well, if nobody else has any questions and I really appreciate the crowd I have today, I'm actually surprised. I thought I'd have like four or five people at this hour. So thank you very much and check out the project online. Talk to this guy as well, he's one of the major contributors and oh, oh, Greg, Greg Haynes from HP also. The project is actually mainly driven by HP, Red Hat and Cisco.