 Hi everybody. My name is Isaac Levy. Everybody calls me Ike. This lecture is on the topic of free BSD jail, which last year got some interest here at DEF CON because it was used for capture of the flag part of why I'm here today. Quick Ike context so you guys know who I am. I've used jails extensively for a long time. I am not a jail author, no commit bit, no that fun. Throughout the lecture I'm going to say a lot of stuff. I'm not religious with operating systems and bashing other ways of doing things. Every tool's got a purpose. But some of my arguments are based on specific threat models. If they're not real clear or you feel like I'm being a jerk about something, talk to me about it afterwards. It's always interesting to argue in a constructive manner. Warranty. It's a lot of information. I'm going to be moving really fast through a lot of material. I will be around if anyone has questions. And I will be trying to stick to really classic stock UNIX processes and ideals. Stuff that is 30 years and older. So there's no real, what do I got to do? Oh hold it up here. Okay. So I'm not going to present any Ike specific magic. Oh my God. Thank you. This is what the BSD user group. Really. Thank you guys. So I'm not, and I'm assuming you know your way around UNIX. So I'm not going to spend a lot of time on simple things. Starting with a quick exercise, scale, patterns, and complexity. Has anybody in the room seen the Eames film, powers of ten? 1977. Really great. Great piece. It's up on YouTube, URLs in here, URLs in the notes. It's a great film where they zoom through the galaxy and they start with this guy sleeping, which you can kind of see over here. The whole film is just about a guy sleeping. And they zoom way out of the galaxy at these distance intervals of powers of ten. And then they zoom back in, into the guy's hand, and into, you know, down to quarks. Get really teeny. Do you need something? Good morning. Okay. And throughout it there's this really cool situation where you have all these patterns that keep reemerging visually. I find them really interesting. There's patterns, patterns. Okay. You get the idea. Well, this is the little Ike universe for the day. The Ike's Internet universe. And you know, start at the outside and we've got like, you know, satellites and all kinds of telecommunications equipment and gear. We zoom in. There's a map from the Opti project. Thanks. Dan Kamitski, who just spoke. And you know, zoom into the east coast and zoom into my home in New York City. And you slide past a nice bug meeting and into a building. And we're in the bowels of a data center. And in the basement you'll find the last bug meeting. And we all know what data centers look like in the room, right? You get down to the cables and the network infrastructure. And at that scale you find another guy sleeping, conferences. We all know what's inside a server. We all know what's inside a computer. Then we get down to the ethernet port and we'll jump over into, you know, what's listening on those ports. In this case we have Apache and SSH listening. And here's what we're here to talk about is the Unix system. And they're the gateway into the Unix system over the network. And then we've got our daemons of processes running and applications behind that. And down at the bottom we found these little things at really small scale climbing around the data center and we think the NSA made them. And all of that complexity, all of that mind boggling stuff, just to pull up a web page, right? Pretty wild. You know, this stuff can be more formally referenced in other ways, like OSI model. But I like my little idea of this. Yeah. And we all know about hardware. Not worth talking about. Hardware is getting smaller, faster, cheaper. Like those little things next to the matchhead are embedded web servers. Stuff gets small and cheap. So in all of this, let's focus in on the Unix. This is my diagram of Unix for the day. And it looks wild and complicated when you first stare at it. But then after a second it looks kind of familiar. See a bunch of familiar Unix stuff. And it can be broken down into something even simpler. You know, your system ends up being your user land kernel devices, at least in the way I'm talking about it today. And those things, though, break down into stuff that we're familiar with. User bin, user local, you know, home directories, that kind of stuff. Device nodes. So, you know, our world is complex. And it can do a lot with it. System administrators tend to think stuff at scale where a given system is just one unit. So let's look at it as a simple thing. You know, when you look at all this stuff, too, there's some fun, fun stuff. This is one other little part of this exercise. Trip out for a second for the math heads in the room. Mandelbrot fractal, Julia set. And as we scale through stuff, again, we're noticing all these patterns. It's the whole deal, right? This gets really trippy. Just kick back and enjoy it. You know, so look at this stuff. Start looking at our natural world and the world that we create, our world. And if you start applying similar principles, hey, man, get into virtual unixes, too. Why can't we, I don't know, do fun stuff at different... You get the idea. I'm going to go faster through all this stuff so we can get to the meat. Okay. Now, look, these ideas, like virtualizing stuff, virtualizing your operating system, the stuff's not old at all. Okay? Unix in the first place was designed to be a time-sharing operating system. All right? And this little snippet for you. To overcome this inefficiency so many people could use the machine, time-sharing was developed. The obvious next step was to allow each person to run several programs. Unix is one of the multi-programming operating systems that now lets each person execute several programs simultaneously. But the usefulness of multi-programming was restricted by the existing terminals. Then the first multi-programming terminal was built. Okay. So then we get Rob Pike talking about the... I've got this whole movie if anybody wants to watch it sometime. You get the idea. So, what kind of real-world context warrant virtualizing the entire operating system? We already run, you know, we have multiple processes we're threading. That's all pretty advanced, little stuff that's mature. But what about the whole OS? Why do we need to do that? Okay. External security threats, why we're all here, making development messes. But the real focus of why Jail was created in the first place is to deal with the problem of mutually untrusted users. These are a bunch of caps to the flag participants from DEF CON 11. Right? That's what you usually think of with mutually untrusted users is, you know, us. But, you know, really it was designed for these mutually untrusted users. A bunch of people using computers for various purposes. You know, why are these people all sitting waiting for subway train? Why are they mutually untrusted users? Well, the guy on the far left, he is not to be trusted in that woman in the pink jacket's kitchen. All right? He doesn't get to touch stuff in there. And she's not to be trusted with his motorcycle, conversely. You know, so you get down to situations where, like, users are going to step on each other. But this is why we're here. We're here to talk about hackers. Here's a bunch of them at Shmucon. You know, like, users do all kinds of crazy shit. I hosted these people's web, this fine lady's website, and they were a real pain in the ass. They wouldn't stop using Telnet. And, like, users do dumb stuff. We all know this. You know, like, this guy's barefoot on a ladder in a pool with an electric drill, right? And users have funny user interface requirements at moments and funny habits. And sometimes users are just outright abusive with the media. Right? Yeah, users get abusive. And some users just that's how they do it. Then you get into cultural clashes, right? People have different ideas about how, what's cool and what's not. Like in India, they love rats. In New York City, we do not. Okay? And, you know, you have mutually, programs are mutually interested users. Two bugs. Things blow up. And some of the worst mutually interested users are professional sysadmins. Muscle memory kills. We all know this. We make the worst mistakes and the most mistakes. Cause we're doing this all the time. You know? So what do we do when shared infrastructure? Like, if one person blows up something with shared infrastructure, everybody using this shared infrastructure is affected. Right? How do we mitigate this problem? How do we deal with this in some way that, you know, all users can be always doing what they need to be doing? Well, we lock down machines, right? It's not fun to work on machines that are so heavily locked down. We really cumbersome to do simple tasks. You can't run all the software you want. This isn't a way to manage everybody to keep them at the point of a gun. Right? I mean, UNIX used to be fun. I still think it is fun. Alright, one more last little thing is if you're maintaining a lot of old junk, Jails are great. So this rack with all those one gigabyte scuzzy hard drives is, you know, I've got three web servers, a local SSD and sCache file server for two people and two dev servers. It's a lot of juice. You know, all of that stuff could be shoved into a nice brand new high density 1U box and you run a bunch of Jails and we've got all of our isolated systems. And hey, get another one and it's redundant. Alright, so let's get into nitty gritty. What is a jail? Jail is a part of FreeBSD. How many people in here use FreeBSD? Yeah, alright. How many people use OpenBSD? Yeah, cool. A lot of BSD people around here, right? Great. What is Jail? It's a user space utility like if config. Alright, real simple. It produces a virtual system image and this is process tree based. Okay. What is a Jail system call? Well, it's a system call that Jail uses and it calls CHroot and attaches it to an IP. Pretty simple. And it's very few lines of source code. What Jail is not, it's not a classical machine emulator. Like, if we have time, I'll get into more of them at the end. It's not CHroot. Okay, this is commonly misused vocabulary in other UNIX cultures. OpenBSD people. Stop calling CHroot Jail. It's really annoying. Please. It's not CHroot though. It's something completely different. Great users for Jail. Hardware resource sharing. You can securely separate on trust the user's processes. They're great for learning, development, testing, hacking. You want to test your new root kits on some safe environments? Make a bunch of jails. Insane, high availability possibilities. Moving jails around from machine to machine. Cool for honeypots because you've got a God view of a system. And they're really cool for highly vulnerable network services. Stuff that you have to run that sucks. Poor users for Jail. You don't get a kernel in a Jail. So if you got to do anything that requires dealing with a kernel, you don't got it. You have limited network and device interface access. And, you know, poor users for Jail is like when CHroot will simply do the job. Like Bind on FreeBSD, you have some simple flags in a C startup that, you know, just start buying in a CHroot environment. It's not really worth building a full Jail for that because you got to build a whole OS. And note really, note worthy Postgres does not run securely in Jails because Postgres uses, this one's a real annoyance, uses system 5 IPC calls, shared memory, which totally borks the whole idea of, you know, keeping jails separate from each other. That's a Postgres thing. If anybody wants to talk about it with me later, I'd love to. It's been a real pain in my ass for years and other people's. How to Jail. Okay. Look, the definitive instructions are in the man pages. Go BSD for keeping man pages. You know, but it's real simple. Three steps. You compile your FreeBSD user land from the source code somewhere on the host machine with some minor tweaks. You create an IP alias on a network interface and you run the Jail call with the IP user land and you boot the Jail. Just like throwing flags at if config or something else. Practical comparison between a Jail system and a regular computer. Pretty much the same thing. Over here we've got, you know, a device node that's been hidden from the Jail. We have no kernel. And other than that everything's there. So let's make a Jail. This is the best way to talk about how to really secure and break Jails. Our host machine is a FreeBSD 6.1 machine. Preflight, we are going to get our source to build with. CVS up is a great mechanism. If you're not familiar with it, send the FreeBSD handbook. You can just get the source via FTP or whatever on disk, on CD. You make somewhere for the Jails to live. So you have to think about where you're going to put them on disk. Putting them on a partition that takes most of your system and is just there for Jails is a great idea because then, you know, waste all this space with a whole lot of, excuse me, then you lock down the amount of space your Jails take up and separate that from the host system. You want to make somewhere for Jail related start and management scripts to live. Comment on starting Jail from RC. There's stuff in the system to start Jails from RC. This is bad in most Jailing contexts. This can thrash painfully. If you have say 50 Jails on a box, 100 Jails running on a single piece of hardware, and a couple of them are having some kind of problem. You have to power cycle the machine. If you're having problems with those Jails when they're all starting up, they're going to bite you and it's really painful to untangle how and why they're barting you especially when they're bogging down your whole boot process and your system. Later. Let me see. So, pre-flight. Here's the man page from the Jail. Here's the first important lines. We're going to keep those up in the corner as we do this. So, we're going to suit a route. We are going to figure it out where we're going to put our Jails. Okay, our local Jails. Gravy. We're going to make a directory for our first Jail to live in. We're going to call it Chick and our host is Chicken. So, we just found user, local, Jails, and we're going to put this directory in there for Chick. Now, we're going to change to user source. Here's how you get your source, CVS up, or FTP or whatever. We've got our source. And we're going to run Make World with the destination directory being the directory we just created where our Jail is going to live in the host's user land. Alright? So, you run Make World and you cook and cook and let it go and you cook and you run Make, you make your distribution and you let it go. And so, what do we just do? We made our directory and we just populate it with the base user land utilities in our operating system. Now, we're going to mount Dev. We're going to mount our devices and real quick there's an old, little old trick when we mount Dev to, it's an old school thing from previous D4 where you soft link a kernel. It's not on the man page as a requisite, but nobody was sure what you would do. Like, we don't have something called kernel in the root directory of the server. What kind of problems could that cause? Well, none at the end of the day. So, you don't have to even put something in there that looks like a kernel, but not a tradition. Why not? You could actually compile and put your whole kernel there but in the case of making honey pots, for example, where you'd want something to look like it's for real. But, you don't need to. Let me see. So, yeah. What do we do? So, we mounted our device tree. One other comment. You can hide a number of devices from your jailed system so that you can't see, for example, all the disks you have in your system or all the network interfaces or all of whatever on your system by throwing flags at the mount commands for mounting DevFS. So, pre-flight. Common question. Why isn't this apart all automated? Why can't you just push the button and go make it? Well, because God knows how your system set up, how much disk space you have, where you're going to be putting these things and then in just a second we'll see that, you know, you've got to add users to your system before you can do anything with it. You're working with a headless virtual machine. You need to do some configuration that's specific to you. So, this is all stuff you just got to do. And stuff that you can script out for yourself very easily to automate. So, we're going to configure the host. If config, we're going to add some things to rc.conf in our host so that basically what you want to do on your host is lock down anything that would listen to, that would bind to all IP addresses on the machine. You want to bind everything just to the host machine's IP address. So, common sense here. You just, you don't want something listening on IPs that could be listening from jails. Let me see. Also, some little tricks, send mail, submit is a real handy little new feature and RPC bind enable, you can get rid of it and IDD flag, blah, blah, blah. That's all on the man page. Then you want to configure your host's SSH statement to only bind to the host's IP address as well. Because then you get into this condition where if SSH dies and you chill the host could be answering on that IP address and it gets real confusing. So, you want to just be real, real careful on the host machine when you're configuring it, real minimal. And then we're going to configure the jail. So, we're going to call, make the jail call. This is the man page again and we're going to do a bunch of stuff in it. We're going to follow this list of simple things that are the same things you do when you install and create a new operating system. So, we make the jail call and suddenly it's kind of like it's kind of like a single user boot on a machine. You just got a root shell in this jail and nothing's running except your process. Except the shell process. So, check the host name. We just gave it, you know, chick.diverseform.net If you check who we are, we're root and we're in the root directory. We're going to run system install and be lazy with this, but also to see that, hey, the program just runs just like a standard and free BSD install. And go through setting a root password, adding users to the system, and setting your time zone. Ta-da! Network services. We want SSH to be running. We've got to figure out a way to get into the jail, right? It's got no head. It's got no anything. We need to have an interface to it. So, hey, SSH, cool. And now if we check the jailsrc.com file, we'll see that, hey, neat SSH is enabled. Let's add a few more things. This is all again checklist from the man page. Et cetera, et cetera, et cetera, et cetera. Anything important here? No, not really. It's all in the man page. So, what do we just do? We have all, are usually built devices mounted. We just populated this with specific data. Our users added ports if we needed to, added other softwares, whatever we need, just like we're setting up a stock brand new system. So, we're almost there, right? We configure it, we call a jailed let me see, yeah, we're finished. So, we exit out of that shell, the jailed shell, and we're back in the host machine, chicken, and we're going to add our IP alias. We're going to give chicken .200 address, following again the man page. We go through it, it's got 200, our host has two. We're doing pretty good. Now we're going to mount the process file system in jail, so that jail has a procfs to use, and there it is, mounted. So, what do we just do? Mounted proc and gave our host machine an IP address for this jail to use. So, start tangent. Here's a script about starting a jail. It's pretty easy to walk down the script to see how simple it is to start this thing. It's easy so you don't have to keep typing the path to your jail to just define it as a variable. Define the IP address, define the host name, and we just mount, make sure we've got our IP interface. We mount devfs and procfs, and we just make the jail call and run RC. So, the jail booting process starts after init and at RC. And here we go. So, we're going to make the jail call. Here's the start. We're just going to start it manually. Does everybody understand the line that they're looking at that's highlighted in red? Pretty simply. I mean, we've got our jail, our path to our jail, a host name for the jail, the IP address we're going to bind it to, and we're going to use a shell to kick off RC as the process is running inside the jail. And kaboom. Here we go. We hit return. WEMO. Hey, this looks like starting up a freebase demachine for the first time. We need to type a bunch of random junk to generate entropy so that our system creates its first keys. If anybody's, you know, going to record this and wants the image of this jail so they can crack the keys on that later, just give me a holler. And hey, it started. Wow. And it started and that looks like any regular boot demessage. It started SSH. Important so we can get into it. On the host machine if we execute command JLS list jails, we suddenly get this little return of the jail, the IP address, the host name, and where it's living. Nifty. So let's SSH into our jail. Let's get into it. And if you notice the IP addresses changed a little bit in the slides, but that's okay. Still .200 for the jail and .2 for the host. So this is just a regular SSH login. There we go. Ask for a password and we're in. And we have our standard login message that we see scrolling down here. We just logged into the system and we, you know, added that user Ike. Logged in as Ike. The host name is Chick. We are in Ike's home directory. Neat. Alright. There's the computer. And it's just like any new server that you could think of at all. You know, like right here we're going to run top. Here's a video of running top. Like there's all of our processes running. A little send mail that we set up running. Submit only SSH. Pretty simple. Pretty cool. So inside the jail we also have root. So we can do whatever we want inside the jail. Pretty cool. Now that we're inside the jail, how do you know you're inside a jail? This is useful for honeypots. This is useful for a number of things. There's a patch with a URL listed here. Also in the DEFCON CD materials. For how to patch the jails to not ever show that you are jailed. It's a pretty simple patch. But if you're in a jail, SSH control will tell you you are or are not. So let's exit out of that SSH connection and we just walked out of the jail. Will be. The jail is still running. So from the host how do we find processes on the jail? This is from the man page. Some flags you can throw at PS so that you can find the jail based on its jail ID. And the jail ID, just every time you start a jail it just keeps stacking a new integer up on top. So the jail 1, 2, 3, 4, 5, 6, 7, blah, blah, blah. But there's all the processes that belong to the running jail. Pretty simple, right? Now if we want to kill those processes from the host system we want to effectively stop the jail, shut it down. We run kill all with the dash j flag and the jail ID. And it will kill all of those processes. Another thing we want to make sure and do is unmount those proc and dev FS mounts. That's a common mistake that trips people up because they will very silently stack up and you'll have several mounts of dev and proc FS in the same place. Which will only cause problems for applications intermittently and in very weird and surprising ways that you may not find for weeks or months at a time. That's a little gotcha. Okay, so we're going to use the start script that we were looking at earlier to start the jail. So that's way it's a one line start. And really at the end of the day, let me just say something off to the side here, it's really, really important to treat your host machine like a platform that you never touch or log into or use for anything other than running your jails. You only want utilities in your host machine that are for the purpose of running your jails. You do that, keep really hardcore uptime, which you can absolutely do with FreeBSD, then suddenly your jails are the things that you're always dealing with starting, restarting, stopping, running applications out of. And it ends up being a consistent platform. Okay, so we're going to start our jail. Boom! We've got that shell script, so we go for it and there it goes. Started. Really, really simple. And again, there's SSH started up as part of the process. And there again are our processes as seen by the host system. Now JLID6. Pretty cool. So running processes. We're going to use JExec right here. JExec is a utility that you can execute a process within the jails process tree. So right here we're going to run PS to get the processes from the inside of the jail. Using JExec, let's bring it up right now, is generally a bad idea in a lot of contexts. JExec makes a call to a low level system called JailAttach, which basically anytime you're executing a process that's controlled by a process on the host that you're executing inside of your jail, that right there is a vector for something in the jail to escape the jail and get out of the jail. So if you do something with JExec to something that's been compromised or set up for you to do in the jail, like if you commonly grab some kind of file out of the jail and your jailed user finds out about this, they could craft something very malicious for your program to return to your host system. So you want to use this intelligently. Let's see. So in this case, if PS had somehow been compromised and changed around, tweaked, in some ways take advantage of the JailAttach call, it would be screwed. So also in the host's process file system and in the jail's process file system, the status file contains as its last field the host name of the jail of the process which runs or just a dash for all the rest of the processes. A lot of people who don't run jails in previous days don't even really think about the dash. So back to this comparison, the host can see all the processes, the jail can see its processes, and when you look at a given process on host, in this case, what are we looking at? I think I'm looking at SSH on the host, SSH in the jail, and SSH name in. The jail has the host name as the last item in what's returned. The host has a dash, wild excitement, tedious details. Also in PS, you'll get a J, if a process is jailed or not. So you can do all kinds of things to script this stuff. Does everybody get it? Everybody gets it. You get your virtualization. Okay. Excuse me. Is there any way to stop that process information? Yeah. Patches to the source of free BSD. I would actually email the guy who did the patch before to dis allow host SSH control information leak because that guy does a lot of honeypot stuff and he may have already done it basically. So we get it. So the best practices and opportunities. Okay. And while we're with the metaphors, anybody who knows who PHK is ever thought of the similarity here? I did. Okay. So first and foremost kind of idea. Nobody's broken out of jail yet. PHK would love someone to do so. He wrote the jail feature originally around 98. And in conversations and just kind of discussing this with him, he kind of feels that I would agree that it's still kind of an esoteric thing. Like there's not a critical massive of Unix or BSD users who are running a lot of jails all over the world. So maybe it hasn't been a real big target for people. But the things that it's built on with your CH route and your TCP IP stack on free BSD sure as heck are. If someone breaks out of jail PHK really, really, really wants to know about it. That gets him really excited. So best practices SSH indoor jails to manager processes. We already got that. You can always see the jailed file system in user land from the host server but be careful when you're tiptoeing through your jails because you got a lot of things that are soft links and all kinds of other stuff that are part of your stock operating system. Like can anybody in the room yell out what slash home is on a free BSD system? Like what actually is that? User home. Right. Soft link. So if you're tiptoeing into user home in your jail, that and you have the same perhaps user name like I've got Ike in my jail and maybe Ike in my host. If I go tiptoeing into that, suddenly I could do something to a bunch of files on my host machine when I meant to do it to my jail. So it's a really bad practice to really spend a lot of time doing stuff to your jails from the outside. So you design your jailing system carefully and be creative with core UNIX utilities. I mean there's how many stock system calls? What's the cut? 1200? 1200 stock system calls. That's a lot of stuff. There's a lot of utilities in there. You can do anything with what's in the box. Batteries included. Use your highest secure practices for your host server. So this means everything, everything, everything. This means big, fat SSH keys. This means every kind of lockdown you can possibly think of, apply that crazy hardcore painful rigidity to your host server so that you liberate your jails to really be used and not be locked down that hard. Okay? So great utilities. In FreeBSD4 it's really worth mentioning or anybody using that JPS, Jkill, Jtop, they're all imports. They're kind of pearl based. They've got some rough edges, but boy they're useful. Five and six onward, built in, PS and all have jail flags. Plus, JLS for listing jails which we've seen. J exec, 10 minutes. Wow. J exec for excusing stuff in jails, day attached. I'm going to jump through some of this because he's going to read this stuff. Additionally handy or other stuff. Look common weak points in jails. Host name lockdown. If people change the host name of their jail this can get real confusing. Say they know the host name of another jail in the box and you're doing something where you're trying to either stop or start a jail and you don't know which host name it is now and things can get very, very confusing. So you can lock that down. Other things, resource attacks, partitions, disk images, another one, fork bombs, memory hogs, there's solutions to deal with that. Direct, which we'll talk about, direct diver access, flags to melt, FFS and PROCFS. What about this stuff is unique to jailing or this virtual machine environment. Like this is a bunch of special stuff you've got to learn and deal with if you're going to secure your jails. The only thing that's unique is host name lockdown. All the rest of this stuff is classic UNIX stuff. All right? So you don't have to learn a lot of new stuff that's going to be just specific to jailing if you want to run jails to run virtual machines. Comments on isolation. We're going to start whipping through all this stuff. Cool thing called the open route project that a guy named Evan Cermanto did. People talk to me about that later. There are a number of ways to perform resource-based attacks on jails. And there's a number of ways to mitigate them. Let's jump through where they come from. Okay. So process attacks. There's something on the DEF CON CD that a friend of mine, Brian Redman, gave me to use in testing out jails. He made it a long time ago. It's called hog. It's a few lines of source code. And what hog does is hog just a certain chunk of memory. Real simple little program. Well, you can create a fork bomb by running a while loop and taking a hog, in this case hogging 99 megs of memory. So this is going to fork off an infinite number of processes all hogging 99 megs of memory until swap is, you know, burning. Well, you can lock this stuff down. Brian, here's where Brian made it. And he's blown up a lot of really big systems with this little piece of code. Memory and process attacks. Here's how you deal with it. You just lock down maximum processes and how much memory gets used by the processes. And let me see. Then you want to set immutable flags on the file. And then you could start up the jails themselves in a higher secure level so that those immutable flags root can't change these files, these very special files. So root becomes a little less than root, but in a trivial way for most purposes. Or you can actually set the assist control variable on the host machine so that no jails jailed root can change flags on a file. Change immutable flags. Honeypots. Okay, so this is that little comment on honeypots. We already talked about disk resource control. Put at least your systems again on a separate partition to isolate them from your host machine. And file-backed disks are really, really, really cool. Disk images. A lot of people use Apple machines and are familiar with disk images there on FreeBSD and the handbook they call the file-backed memory disks using the MD config utility. Cool stuff. Very simple way to create partitions without truly creating partitions. And they're fast if you compile them into your kernel. Which is a pretty simple little flag. And we won't get into how to use them because the FreeBSD handbook does a great job of telling you how to use them. Automation. Tar balls or your friend. If you're deploying stuff to a lot of jails, updating jails. You want to be aware of your Dev and Proc mounts. Be aware of SimLinks. And you can also use FreeBSD ports. Hey, cool. You can... Let me see. Oh, not for the ports collection, but say you're in a big enterprise environment and you have a lot of specific custom jails you want to roll out. Like, you know, hundreds and maybe thousands of jails in various environments. Well, you could actually use the ports mechanism not taking and putting a whole jail in the public ports tree. That would be stupid and irresponsible. But here's my custom OS. No. But, you know, in a big enterprise environment it's worth actually looking at how the ports mechanism works because you could deploy all kinds of custom jails using ports locally. And then you can also stuff stuff in CVS or Subversion. Upgrading's really cool. You could just use build world. Upgrading's cool on FreeBSD in the first place. You just dive into the jail when you first run what was I going to say? I just looked at the time. I'm going to keep moving on faster. It's in the handbook. It's real simple. Anybody has questions, ask me later. syscontrol.com. There's a bunch of syscontrol variables which are important, like set hostname allowed, right? So that the jailed user or rude can't change their hostname. In ForcedatFS, this is so that the jail can see only particular details about file system mounts. By default, the jail can see all file system mounts on the whole operating system yet it may not be able to touch files on them. This is a bad thing. This is information leak. Like hey that guy's jail is on my machine too. Ha ha ha. Now I know that. So you can lock this down to a point where your jail has, can just see its mount points or see none. Allowing raw sockets for ping etc etc. All of these things have consequences and they're pretty self-explanatory. Access to routing sockets. System 5 IPC allowed. PHK put that in so we could run Postgres. Thanks man. It's a pretty insecure thing though. And CH flags allowed again getting into whether or not the jailed can change their stuff. And then of course whether or not we're jailed. Firewalls moving on. You can't run a firewall from inside a jail. You want to talk about it? Talk to me later. Start script with the disk image. Well there's a few lines added for all of this stuff. Some little miscellaneous for crontabs just so that your logs get really really nice and quiet inside of your jail because you don't need to save entropy and you don't need to adjust the time and it will just work and log it. Miscellaneous gotcha. You want to make sure that you change flags if you're trying to remove this jail directory. Quick words on true virtual machines. Look this is from the VMware catalog, excuse me from a brochure of VMware's. Here's a structural comparison. VMware's are really classical virtual machine and there have been a lot of them over the years. A ton of them. VMware's a popular one nowadays. And a pretty good one. A lot of people use it to run stuff. Well you got your OS, your hardware, your OS and the software that virtualizes the machines that then get loaded into it. Very different model from jailing where you just have your hardware, your OS and isolated processes. So once you get that, let's look at attack vectors real quick. VMware's got a couple problems that jail doesn't in the context of creating really secure services. One of them is, hey, your operating system is of course a risk and that's a risk in all cases. The software itself is something that has an unknown developer headcount and it's closed source. Which all of those issues well those are discussed in a lot of other lectures here. In a similar vein though, if we want to talk about attack vectors for jail, people need to really root, they need to really crack CH root and TCPIP sockets on jailing. I'm getting the kill switch here. So I'll just finish this sentence and kind of flip on and go. So at the end of the day basically what I'm getting at is a lot of people in the world rely on FreeBSD's TCPIP sockets and CH root. Therefore there's a lot of hands working on jail without even knowing it. Then we get into Slayer zones which by the way are being used in capture the flag this year. They have little problems that can be overcome but zones are cool. Zen is a whole other ball of wax which you don't have time to talk about. Then you have Microsoft doing things with hardware virtualization, bad bad bad stuff. So look, here we are. We're almost to the end. So look, at the end of the day jails are this really simple thing that you can combine with anything else on your BSD system. Which means a lot of other really cool technologies that are upcoming that just work with jails. Because they're all modular, independent isolated things from each other. So with that stated you can do a lot of really cool stuff with jails. We missed a lot of fun stuff but that's it. Look, quick key special thanks to Winermute. I mean who taught me to jail in the first place. And reality. By the way, by Winermute a drink he's up here. And also PHK and Robert Watson. There will be stuff, a rata from this that didn't make it on the DefCon CD like extra slides, up a nice bug website in the library and I'm Ike and thank you all very much. There's that.