 door to door. I need you guys to be louder. That's from my benefit. Okay, cool. So my name is Brian and today we are going to talk about shipping Ruby apps with Docker. I did start a company called Co-Pine. Have you ever heard of Co-Pine in here? Oh wow, that's awesome. Okay, cool. How many of you use Co-Pine? We'll work on filling the gap between the first hands and the second hands. Come see me afterwards. We'll talk about that. Okay, so we're going to talk about Docker just to kind of gauge what people's familiarity is and I can kind of try to tune how detailed I get. Who's heard of Docker? Okay, almost everybody. How many people have installed it and played around with it, like either locally or somewhere? I would say that's about a third. How many of you are running it in production? Cool. So I'm really excited about Docker. Like really, really excited about Docker. I think Docker is going to be something that all of you are using within the next two years and so my goal in this presentation is to try to kind of paint the picture of why that is and get you excited about it too and we'll look at kind of what the options are for getting started with it today. So Docker is a generic way to run any service anywhere where the any is a little bit of an asterisk and it needs any Linux service. So it won't work for say Mac or Windows services, but if you're deploying a Linux, which I mentioned many of you are, you can use Docker and it's a generic way to ship things around and that's where the name comes from. It's a shipping metaphor. So you think of like cargo containers. Docker is based on that metaphor and you put your application into a cargo container and you throw it to the ops team and they can run it anywhere. So it's worth two big components. It's container based virtualization combined with a generic package format that you can use for any service. We'll get into exactly what that means in just a little bit. So container based virtualization is kind of what makes Docker awesome and this is the big DevOps idea of the last year or two and I think going to be the big DevOps idea for the next couple years. It kind of snuck up on a lot of people, myself included. So like a year and a half ago I didn't really know anything about what container based virtualization was but the Docker folks through a conference a couple weeks ago and they got a bunch of presentations from people who are using container based virtualization and apparently everybody who knows a lot of stuff about ops has been using us for a while. So all of Google runs on container based virtualization. Pretty much all of Twitter, all of Facebook. So the guys that are dealing with DevOps at scale are pretty far along in terms of migrating stuff to container based virtualization. I think that's one of the big reasons why it's going to become so prevalent and it scales up and down. You don't need to be running near-based mini servers as Google for this to be valuable. We're going to look at how it can be useful even if you have only one server today or if you're just developing on your laptop today but you know that if you use an approach like this it does scale up. I mean that's as big as it gets right so you can start now and then move up with it. So there's a project that's been around for five or so years called LXC which just stands for Linux containers and this was kind of how Docker got its start. For a while it was a sort of a wrapper, a higher level, easier to use layer on top of LXC and LXC was kind of doing the hard stuff under the hood. LXC itself was built on other layers but Docker was sort of the user interface for LXC. It's CH3 on steroids so it means you get a shared kernel and isolated resources so you can only be running one version of the Linux kernel right when you're using container-based virtualization. So there's no like okay I'm going to run this kernel with these kernel models over here and then I'm going to do something completely different over here. You can do that with a full VM sort of system that a lot of people run on things like AWS right on AWS can pick your kernel. You can't do that with container-based virtualization but it turns out that if you're using a modern kernel that's fine for almost everything right you don't really I mean me personally I don't really want to be picking what kernel I'm running I'm not smart enough to actually do that so I just want a good kernel that works and then deal with things that are a little bit higher level than that and then the resources when you're using container-based virtualization like LXC or Docker are isolated so you get a sandbox for resources like PIDs, networking and files so the root this is like the CH root part the root on your container well is not the root on your host operating system and the PIDs are different right so PID 1 within your container does not have anything to do with PID 1 on the underlying host operating system and then you can you can do all sorts of fancy things with the way you configure the networking for each container you can give it access to the host networking through the network bridge you can say okay this doesn't have any networking at all or the port mapping which we're gonna see a little bit and you can also do resource limits so you can assign CPU shares and memory shares to your containers to control how they work when they're competing for resources Docker is from a company that was called used to be called dot cloud and they kind of have a platform as a service that was akin to Heroku and they were running this for a while and the conclusion that they came to was well you know we're never gonna be more popular than Heroku or whatever but they sort of realized that the technology they developed under the hood in order to support running that was more valuable than the service they were providing themselves so they made a pretty drastic step and open sourced this project Docker which is the next evolution of their underpinnings of their platform as a service and it's all under it's either MIT or MIT or BSD license very permissive licensing and gave it away and said okay what we're gonna try to do instead is become a unified standard for how people use container-based virtualization if we can do that we'll figure out how to make make a bunch of money doing it so they rebranded the company to Docker Inc the open source it's on go and it consists of two primary components release user-facing components which is a command line their face and a server daemon that you run which provides a rest API and those is talk order HTTP which has some really nice advantages oops under the hood dockers at the top and then there are a few pieces that kind of make this work on the left side a UFS stands for another union file system that's a component that Docker use uses in order to store Docker images basically the actual file systems that make up the applications that you're gonna run need to be stored somewhere and a UFS is a layered file system that makes that efficient it used to be that Docker would talk to LXC which we looked at a little bit earlier and then LXC uses C groups and namespaces which are some kernel level features that's really like the weapons grade stuff that makes this work that's like the hardest components to get absolutely right I think and then but but more recently Docker has switched to a component they open source called lip container which obviates the need for them to use LXC right now you can switch back and forth and live containers the default but I think it seems like with containers the direction that everything's moving so that just simplifies the stack a little bit okay so what do you get when you run your Ruby application with in a Docker container isolation we already talked about it's sort of an ephemeral ish in the sense that if you're running a Rails app usually the process that you're going to be serving your request from whether it's Internet so passenger or unicorn or something like that it doesn't really need to keep anything over the on the lifecycle of that process right so we kind of move to these pseudo share nothing architectures where we're connected to a database writing things there but the process themselves are supposed to be disposable and Docker works really well with that it doesn't work to try not to delete anything that you might need later but basically you can throw away containers and they're gone one of the big advantages with container virtualization versus say full VM virtualization is the low CPU and memory overhead so if you think about you know let's say you fire up virtual box on your laptop and then you think about how many VMs can I run simultaneously on my laptop it's like well each of those needs what like half a gigabyte of RAM at a minimum to kind of like run a full system and they're a number of gigabytes each so you can run a few of them and that works fine but with Docker you can run hundreds because you're not actually needing to store full copies of all these file systems and boot full instances of an operating system it's more like running a process it's a special magical process that's isolated then it is running an entirely new operating system and so they also boot up much more quickly if you are booting up a regular VM you have to wait for all of them on to to do right that's just kind of the way it works but with container virtualization and Docker the amount of time it takes to boot a isolated container is measured in milliseconds you basically can't perceive how long it takes to start up a new Docker container or throw it away you get the small images with the UFS layering which we talked about and this lets everything be very very high density which means saving a bunch of money and also saving a bunch of time because it can run all of infrastructure on half as many servers that makes everybody's life easier right there's always one of the things that need to be dealt with on a per server basis and having fewer servers is a win so Docker has two components there's images and containers there are two main concepts this is a little tricky at first I kept getting them confused but the image is a saved version of something that can be booted into a container so you think of when I say image can think of like a package Docker image is like a Docker package and then when you run that package you get a container which is like a process so it consists of a root file system that can be any Linux distribution that's in there and a lot of times people are basing their Docker containers on a bunch of you don't have to and then also metadata within the Docker package of how to run this so there's configuration that'll say okay when I want to run this package basically this is the executable that I should run right pretty simple it's just a little packet of JSON that's restored there and then they get stored in a registry which is kind of like a package distribution site Docker runs an official one you can run your own just like RubyGems.org you can push packages to it and they become publicly available or you can run your own Gems server. So if you want to give Docker try this used to be kind of a pain in the butt but if you're on a Mac you can install boot to Docker which has everything you need to get started with Docker on a Mac it actually runs Docker within a regular like a virtual box virtual machine so Docker itself can't run natively on a Mac so the hack is well okay we'll just run one VM which will be our Docker host and then we'll set up the client on your laptop to talk to that VM because it's a client server architecture the fact that there's a host running in a VM doesn't really matter so boot to Docker will take care of all that you don't need to worry about it if you're running a Linux laptop or Linux development environment you can like a bunch you can install it with the official Docker packages but long story short it's very easy to get started with Docker these days so I mentioned there's a command line interface I'm not gonna go into a ton of details about what you can do with the CLI but just at the high level give you the broad strokes of the main operations that you perform the documentation about this stuff is pretty good I mean it could always be better but Docker build is kind of the key command these days so Docker build is a way to kind of compile a package from a directory so if you have a Rails app you're going to go into your Rails app directory and want Docker build and the end of that process will be a Docker image or Docker package that you can then deploy to any other places I'll actually show you that in a second Docker run is the next step so okay you've got this package now you want to actually execute it somewhere this could be on your laptop on your staging environment or on your production environment you can customize just a few things so just the things that should vary between those places right so if you have bought into this idea of kind of environment variable based configuration Docker works really well with that people who use Heroku might be familiar with the concept of a 12 factor application that plays very nicely with the way Docker works and where that ecosystem is heading so what you want to do I think is when you deploy your application to staging you deploy a package which you've previously tested on CI byte for byte test of that exact package and you promote that to staging and then when once you test that package on staging you promote that exact same Docker image or package to production byte for byte and the only thing that varies between staging and production is the configuration in terms of the environment variables or potentially some of the things that your ops people need to control so port mapping is for example you might run your production instances on a different side of ports than your staging processes Docker run you give it at minimum you just give it a Docker image and it will run with the defaults but also it lets you provide environment variables and networking configurations and resource limits so you can set a RAM limit and then your passenger process will never be allowed to consume more than that amount of RAM by Docker PS works just like you would expect display running processes and there's just a bunch of other things that may be working with Docker images and containers a little bit easier there's a public registry of Docker images so if you need to run something like say Redis, MongoDB, RabbitMQ it's likely that somebody's already gone through the work to package that up into a Docker image which you can just search for on the public registry and then Docker pull will download that to your Docker host so that you can run it anytime you want then Docker containers have a lifecycle you can stop them which will send signals to the processes to kill them off you can look at the standard out and standard error using the logs command that's kind of one of those 12 factor application concepts you can look at all your images and you can inspect what's on your system so we mentioned Docker build how does Docker build have any idea how to specifically package your application into something that can be deployed anywhere the answer is a Docker file so this is a really basic syntax that's been developed to almost like a DSL for just the few instructions you need to give to Docker to let it do its job when you're ready to package up an application so this is an example of a complete Docker file for a Sinatra application there's not much to it which is kind of the point but we'll look at the sort of the few different types of commands which are present here the first thing is that every Docker file starts with that line about where it's from so that that means what image am I starting with you don't really want to day to day be creating an image from scratch where you have a root file system with nothing on it and you need to get everything there for it to work that's a bit overkill so you can start with a either a publicly available Docker image and customize it or you can create your own internal Docker image as a base for your company the fusion passenger guys have actually released some publicly available Docker images which serve as a great base if you're looking to deploy Rails applications so that's what I'm doing here and I'll talk about those images in a bit run kind of does exactly what you would expect it's just run and then shell okay add is a way from moving files from your source code directory your local directory into the Docker image right so they're not something magically going to take all of your code and nowhere to put that into your into your image so you can use the add command to move files between your source code directory and the Docker image add and then dot on the line there in the middle is just going to move the entire application source tree into the home app web app directory in your resulting image and then you need to tell Docker by default what it needs to run if you just say Docker run and then the name of your application you can override what it runs but you want to give it a same default and this sbin myinit is a process that the fusion passenger passenger full image provides that will start up your application so that's just kind of part of using this base image and then everything works by sort of exposing a network port that you you're going to get access to so most Docker files will define that they're going to do a bunch of stuff and the end result of all that if you come to black box right you're just going to say okay when this is done we're going to expose port 80 to the world and that's how you'll be able to use this so to the outside world it doesn't really matter what's happening behind the scenes the you know the network is the interface between these things so this is a talk about ruby so let's look at what it you can do to get your ruby applications deployed into Docker today okay so I mentioned that the fusion passenger guys created some publicly available images to smooth this process I strongly recommend using them especially if you're just getting started with Docker there you can start from just regular a bunch of you but then you're kind of back to square one in terms of well how am I going to get the right version of ruby on there am I going to use rbm or rbm or some app custom app repository that has ruby 2.1.2 it's just not fun to deal with those things right so the passenger guys have kind of already figured that out for you and you can just leverage what they've done they have images that have ruby up through version 2.1 python and node kind of already there so as long as you're just dealing with those things you have to do very little customization they also have a build toolchain and common c-level dependencies like livexml2 so you don't have to worry about making sure you have the right version of that stuff installed and the amount of overhead that you get through building on top of this image there are some processes that it's going to spin up by default when you use this this image as your base but I'm all told that's about 10 megabytes or so of a fixed cost which compared to the RAM usage of most production rails processes is pretty small these days so when you run it I mentioned it has some processes that are going to take some RAM these are some of the things that it can give you out of the box so the myinit process is what we saw in the docker file as the default executable myinit is just a little python script which handles booting up runnit which actually runs the other dependent services and then there's this edge case that the passenger guys have sort of found where long story short if you boot a docker image and the group process in that image is your application then that is going to become a PID1 and that PID1 is not going to be taken care of one of the responsibilities of PID1 which is dealing with morphin processes so they wrote this myinit shim which which does that for you but for all intents and purposes you don't need to really worry much about what that's doing but so the other things that that image provides are core facilities that you kind of need everywhere right syslog is important to be running because of your Linux distribution emits logs you want them to actually go somewhere you don't want them to just get lost in the ether cron is often important if you want to do something like log rotation or something system level that needs to be done periodically you can also just use something like clockwork and deal with that at the regulator which is what i do and then sshd is kind of an interesting piece right so if you're running a docker container it's not necessarily clear how you go into it and sort of look around right like what's the inspectability you can run a new docker container with and you can select a command like bash which is very similar to if you've ever done like haroku run bash you get a bash shell inside of a haroku environment you can do the same thing with docker you can docker run bash but you're not going to be able to look at a specific container and say what's this thing doing right now unless you either run sshd or kind of jiggle around with some of the lower level container virtualization commands in order to attach to it which is a bit tricky right now hoping that docker will add a docker attach command where you can say docker attach and then get a shell within a container so you can inspect it but that has not happened quite yet okay so now i need the demo gods to smile upon me and let me do a quick demo of how this works let's see can you still hear me the podium makes work hello yes okay um so this is just my laptop here and since it's a mac we're going to use boot to docker in order to get this whole environment up and running i've already started that up but basically similar to vagrant you can run boot to docker up instead of vagrant up and it will just make sure that everything's in its right place and it will actually give you a environment variable that you need to set on your local shell in order to configure for the docker client how to talk to the server right so this address here and this port is where the docker server is listening over hdp for commands when i run docker ps it just does a get request to the server and says okay what are the running processes and then the command line just gets that back and prints that out but because the docker server is running within a vm it's not going to show up it's not going to show up inside of my process tree it's not running on my local box it's inside of the virtual box vm so what i've got here is just a really simple application that's a synatra app and what i'm going to do is package this up into a docker image and then run a version of it for my own purposes so let's see here here's the docker file that i'm going to use this is very similar to the one that i showed on the slide the only thing i really changed here is just an optimization on these lines which makes better use of docker's build cache to not have to do bundle install if the dependencies don't change right so everyone knows like bundle install can kind of take forever and you don't change your gem file that much for reasons that aren't particularly interesting if you push the gem file into your docker image before you run bundle install then docker's able to cache that more efficiently than if you just omit these lines it would still work but it would do a bundle install every time so let's start a docker build what i'm going to do is this force rm which tells it to oh wait i'm going to have it not use the docker cache you can actually see what this looks like if it's going cool so here we go cool so it's gonna run it's running the bundle install but i'm just going to scroll back up to the beginning you can see with each step it's going to run the command and then save an intermediate container so it starts with the passenger full image it's going to remove this file add this file over here each time it's actually caching the result in a immutable image so that when we go and run this again it'll be much faster when it gets to the bundle install it runs that now it's showing you the output from the command that it ran and it finishes right so now just to show you if we run this again it's going to be really quick because so i don't know why i actually went back to bundle install but not that interesting what we'll do now that we have a docker image we can see if we say docker images we can see the hello image is sitting on our local system we can actually execute that and the dash p argument tells docker to map all of the ports that the docker image is supposed to expose and the dash d option tells it to run in the background as a demon as opposed to you in the foreground and wait around for it so sorry for the let's see if i can make this a little bit easier to read um docker ps will show me what's running so you can see that the image i ran was the hello image it's running the espin my init command it's been up for 14 seconds and it actually assigned a dynamic port to map to 80 so the docker file said i'm going to provide 80 that's where i have interesting stuff and then docker when you booted it said okay i need a port to make that port 80 available because obviously if you ran multiple docker images that all provide port 80 they're going to conflict right so it's kind of up to your ops team to figure out what ports they want things to sit on in prod right most application developers don't really care about the port numbers and they shouldn't care they're just going to that's kind of the contract is the app developers provide something that exposes a port and then they ship it to ops who figures out how to actually run it in prod so now with this so if i hit the root there's no route there so that's just the sinatra not found page and if you hit slash hi it says hello ruby people it works um so what i'm going to do quickly is just make a quick change change the code you can see i've modified the application i'm going to run the build process again now it worked with the cache so each one of these steps you can see it says using cache so it's not actually running that that's why it's really fast including the bundle install use the cache and then when it got to the ad it did not use the cache because the ad step was adding a different version of my code into the docker image so this is just a fast recompile i'm going to run the same run command again and the ports don't collide because it's going to assign another random port uh to the um the new container you can see we have this container over here it gives every container a name uh it doesn't you can specify the name or it will give you a randomly generated one so in this case we have uh romantic hopper and goofy yoneth um my favorite name is it like often i seem to get angry torbolts which uh makes a lot of sense um uh so you can see this one's been up for two seconds but it's running on port 49154 on port 49153 it's still saying hello ruby people but if we change the port to 49154 hello ruby people um and what you would do if you were using this to deploy to production is you'd probably keep the old processes around uh boot up the new ones switch your load balancer to serving up requests from the new port and then remove the old port from the load balancer config uh and shut it down so that would just be a matter of doing docker kill and it will shut that down now it's going to be gone or you can also address it by its name instead of the container id and it's all the same so now it's not going to be available cool that is the demo so okay um so i'm super excited about this and this is the reason why uh as a ruby developer i feel like for all the years that i've been doing this i've never really had a unit uh to deliver my application in right so for a while uh i worked on an application that was in j ruby and we built a jar file and gave that to our ops team and for as many things as you can say about java and it being scary and enterprisey that was kind of awesome because it didn't really matter what was in that jar file i could test the jar file i knew that it worked i gave it to ops and they could run it the thing is if you're on c ruby that concept doesn't work at all um i guess some people have done work to package their ruby applications up into like dot deb files i mean it's just files on disk so you can kind of package them up anyway you want but it wasn't ever particularly easy uh to do that so if you're looking for a delivery unit you run into all these issues right how are you going to deal with things like bundler and what if your dependency uh your ruby gem dependency actually requires something like libxml2 that needs to be on a system there's no way in your bundler uh config in your gem file to say make sure you have this version of libxml2 installed but you can do that with docker um war files or jar files we talked about that uh you can't use those with anything except the the java platform and if you use docker you can get all this for any service right you can put java code into a docker uh image there's no problem with that at all but you can also put anything else you want into it right so people are packaging up things like mongo db as a docker image people are packaging up rails applications as docker images and it's all just homogenous from the standpoint of whoever needs to run it so this opens up a whole world of change in terms of dev ops processes you can use the same image and container for local development that you push to your continuous integration environment um one idea that we're starting to play with is in our docker images in addition to by default running the code and providing the application creating another executable inside the image called self test so if you say docker run hello and then you customize the executable and you say okay i want to run self test instead the docker image is self testing so it's going to run your suite of tests and put out an exit code of pass or fail whether that image is doing what it's supposed to be doing which is a really powerful concept then you can do things like taking that image building it on ci running the self test signing it on your ci server as having passed the self test promoting it byte for byte up to staging verifying it with manual qn staging promoting it byte for byte up to production and really only having the environment variables and potentially your port configurations changing between those things so that whole problem of well it works on my machine or i'm not really sure what this is you know why this is failing on ci kind of goes away and you can really get this benefit of building it once and then running it anywhere within your sort of linux ecosystem so the fact that it doesn't run on windows really doesn't bother me all that much this kind of meets all my needs okay this changes everything um it's you have to sort of stare at it for a while but why i'm really excited about docker is once you change this one component of your infrastructure it sort of unlocks all of these other benefits through all the other layers so immutable infrastructure is an example of that um a lot of people how many of you have used chef or something like that to sort of configure your servers yeah about like so chef's great but there's this fundamental question that's been nagging at me which is why do we ever run chef against the same server more than once right that only kind of leads to trouble the first time you run chef it's great because you run it and you know exactly what state it's in it ran the script it's deterministic that server is in a known state and you're fine then the problem is you're like oh well now we need to upgrade the version of this package so you change the version of the package and you rerun chef and sometimes it's fine sometimes it fails and you need to iterate on that a number of times but through that whole process you're creating uh indeterminate state on that vm so the reason we run chef against the same server multiple times is because it's too expensive uh to actually throw it out every time right to throw out your vm every time on aws after every chef run people do it um but it's not uh it's not free at least in terms of time right um with docker containers and docker images they're so fast that you can actually throw them away every time it makes no sense to you know uh ssh into a docker container and apt get install an additional package because now you need this extra package you're just going to throw that container away and build a new image which has the packages that you want so you know exactly what's in all of your containers at all times which is a really powerful concept um then it gets a little more crazy right so if you're running all of your code inside of a docker container if you're running MongoDB, Redis, memcache, passenger, engine x all inside of docker containers and there's a host vm right there has to be an underlying operating system that all this is running on and a lot of people are using Ubuntu for that but if you think about it why do you need Ubuntu underneath right like why do you need a package manager at all if your packages are actually just docker images maybe you don't need all of that and maybe not having that can actually lead to a simpler and more secure system um so you've already got a way to run whatever code you need the host operating system is really just a matter of executing the docker demon and keeping things running along smoothly so there's a linux distribution called coro s which is exactly this these guys kind of rethought a distribution from the standpoint of everything running within container virtualization and they stripped out almost everything it's actually auto updates so you never like just like chrome auto updates you can have an auto updating host distribution on your servers and it has almost nothing on it except for docker and some basic configuration management stuff which is really powerful from the standpoint of security right if you can just get an over-the-wire update of your host system it's not likely that that's going to break anything if there's very little running on the host itself so it doesn't have a package manager where you're going to be installing things and updating things and dealing with all that complexity you just get an entirely new image and drop that on top of the old version or actually next to the old version of coro s and then it just changes a pointer so that when you restart the the box it'll boot off the new version of the operating system this eliminates entire class of issues okay the linux distributions actually kind of become unnecessary in the containers too right so the fusion passenger base image that we started with is based on ubuntu right now but does it need to be right there was nothing in my docker file that was ubuntu specific and there's nothing fundamental about the passenger base image that requires ubuntu so you could run something that's much smaller than ubuntu and get the same benefit somebody could take a micro linux distribution like say busybox and add ruby into it and the few c-level dependencies that are needed and now the size of the image for your application drops from hundreds of megabytes down to maybe you can get that down under 100 megabytes which is really powerful right so now you're like okay we're using docker and we don't need linux distributions on the host and we don't really need traditional linux distributions in the containers docker just kind of shifted everything dramatically it goes further you don't really need configuration management if you're running coro s on the host operating system there's not much to manage at all there's a single config file which has things like the users that you need to have available in the few different customizations but that's pretty much it and then for building the images the docker file in many cases is all you need you can shell out to a bash script so if you need to install a bunch of packages you can create a .sh file and run that and it can do a few package installs but for most cases you don't need to be using something like chef or ansible within your docker file to install dependencies because it's pretty simple right we saw how many lines of code that was and then if you go further downstream this starts to affect things like the ROI on service-oriented architecture so a lot of people are talking about microservices these days right and which is kind of a new formation of these service-oriented design concepts they're great for a lot of things but they're not a free lunch especially with respect to deploying them into production deploying five or ten apps is generally harder than deploying a single app but with if you're deploying on something like Heroku it gets a lot easier right so i've worked with teams that use heavily service oriented architectures on Heroku and Heroku is kind of one of the reasons they're able to do that because Heroku takes care of managing everything you just say okay push this to Heroku now i have my service running docker is going to unlock those workflows for any environment that you want to deploy into if you wanted to play on your own hardware you can start to get some of those benefits you can do behind the firewall install so i know multiple companies that have products cloud-based products software development tools that they have customers who are asking hey can i run this behind my firewall for example like the github enterprise problem right that's traditionally been a huge pain but if everything's running on top of docker in production you can actually just give them a docker image and say okay you're going to download this image and you're going to say docker run code climate for example and you're going to get a port that exposes code climate that's the contract is it's going to expose port 80 and port 443 and you'll be able to get a working code climate from just that so it really simplifies behind the firewall installs and then the kind of final frontier is a concept that sometimes referred to as data center computing so the idea of data center computing is that you're just going to plug in a bunch of homogenous resources into your computing pool and then they're going to be able to run heterogeneous sets of work so when you need more resources you should be able to call up your ops guy and say hey we need to double the number of you know cpus and ram that we need but what it's being run for should be abstracted away and just dealt with as a commodity so there's projects coro s has a project called fleet built into it which does this sort of thing you can say fleet run instead of docker run and if you have a fleet cluster it'll just find a place that has the appropriate resources or the appropriate configuration to run that and there's also projects that are a bit more robust like mesos which is a top-level Apache project used heavily at twitter where you can build a mesos cloud and to say okay mesos i need 300 passenger processes they each need to have two gigabytes of ram available and two cpu cores and i don't have any idea i don't even want to know where those have to run within our data center but you've got you know a thousand machines figure it out and just mix them together into whatever resources are available and then give me back a way that i can address them and add them to the load balancer so there's a bunch of interesting stuff happening in this data center computing space right now mesos aurora is a project on top of that fleet and there's a few others as well that are going to make it you know we're going to get to a point where even on small scale projects you're just plugging resources in and then defining in a declarative way what resources your application needs and as long as your resources are always more than the hardware resources you have are greater than what you need for the software you're fine you don't have to worry about it okay so what's the status of docker they released 1.0 finally a couple weeks ago it's in production at a good handful of companies it's pretty stable at this point if you want to deploy let's say a rails application on top of docker there's some there's still some sticky bits it's not necessarily that docker has rough edges so much as docker has a limited scope and what you'll find is when you go to deploy things on docker there are other problems that are not within docker proper's scope that you're faced with so if you are for example spinning up a bunch of docker containers and you need to add those to a load balancer and then you want to do a deployment like i did on my laptop where you're booting new containers you need to reconfigure your load balancer and drop out the new ones that's not really a problem that docker is trying to solve you need a tool that's higher level than that to deal with that so this is where a lot of the activity is happening right now there's a whole ecosystem of projects that are being developed that build on top of docker and make that type of stuff possible for the load balancer example there is a project called hipatchy which is a redis or etsy uses redis or etsy d to configure dynamically configure a load balancer based on whatever you need to be available so you can just talk to a redis instance and say okay these are the ip is of and the ports of the application right now and reconfiguring your load balancer becomes as simple as running some redis commands so you can still do that with say cap astrono right as long as you can connect to your redis instance you can reconfigure load balancer so there's kind of pieces being filled in at each layer of the stack but the ecosystem is not quite mature enough yet where if you're going to run something on docker you can just say okay i'm going to use docker and these other three components i'm a big fan of coro s i think that that's going to be kind of part and parcel with the default docker stack and coro s also provides a component called xcd which is a distributed configuration service akin to apache zookeeper so you can keep all your metadata for where everything is and the environment variables you need inside xcd but even doing that is not trivial right now so there's kind of a few like the exercises left to the reader bits but the reason i wanted to get up here and talk about this now is i think you can definitely use it for deploying for developing locally you can definitely start integrating it into your ci environments and the ecosystem is getting better and better every month in terms of how you know it's really the issue is not having options for deploying to production it's that there's too many options so so many people have created projects on top of docker that there needs to be some consolidation i think in the ecosystem where it's more like rails and there's like this is the happy path and that's going to be i think happening over the next year but i will be very surprised if and the vast majority of people in this room are not deploying stuff on top of a docker based system within the next couple years so it's a good thing to start looking at and playing with and getting getting your head around because it's going to have some really big implications all the way up and down the stack that's it thank you very much for your time and attention and we can do some q and a so i'd like to talk about development because it looks like when you modify that value you redeploy it to docker niche so it feels like they are watching it well actually yeah so in so in development right so what i did there was akin to say deploying something out to a staging environment so i had a docker image that i built from a version of the code i changed the code and then i built a new docker image and deployed that out if you're doing local development what you would probably do instead is create a docker basically make the files that you're coding in available to the docker image so there there are ways that you can wire it up so where you're actually editing your source code you can make changes and then the docker the docker image can kind of just access that without having to rebuild every time you save a file so it's not necessarily something like vagrants or shareholders uh yeah so you're going to um it depends on if you're on a mac or linux the way you would set it up but yeah you can expose through uh if you take let's let's put the mac thing aside if let's say you're developing on linux uh you can do what's called a bind amount with um docker which is the dash v option and that just lets you take a directory on the host file system which would be your laptop uh and expose that into the docker container so you just bind mount in the directory where you're working on the code and then that would be available inside uh to everything that's running within the docker system if you have if you're on a mac and you're using um vagrant then you have to do like another level of mounting right so you mount from the mac system into the vagrant vm and then from the vagrant vm into the docker container but it's also possible so i have a question is how far would you take the abstraction for docker do you separate how say a database from uh applications server in yeah so um i've seen it done a couple ways uh we have a image that we can build for code climate which runs everything within the same image so it runs um redis mongo db memcache uh engine x passenger all within a single image and you run it and you just get uh code climate right um that's one model for production that's not particularly useful right because you want to manage your database separately potentially on separate physical servers uh and it doesn't really deal with things like high availability uh and that sort of thing but it is useful for um for example shipping things behind a firewall uh what i think is likely to be the case is you're gonna have kind of different potentially slightly different um package uh roles depending on where you're deploying to um and then in production like staging and production will kind of be the same uh so sometimes people will say you can run one you should run one process per docker container um i think for most use cases that's a bit extreme uh i would think instead about like a role so you have a role of app right um and that probably runs maybe it runs an engine x process which also has a passenger process attached to it uh but if you discover that you need something like sidekick or you want to run clockwork and do a ruby based cron system you could create separate images for those things and be able to deploy them all separately um but in most cases you don't need that flexibility so i would lean towards um keeping things simpler to start which in generally involves putting more into a single docker image to begin with and then splitting things out as you have need uh because that adds a little bit more complexity so is there a tool within docker to bind those boards together or yeah so um yeah so docker has the concept of links which is relatively new so you can say okay docker run mongo db uh and then docker now run my application but make mongo db available to it uh and that's called a link that only really works if they're running on the same docker host so uh i think for many use cases it it can break down pretty quickly but it can work for like local development so you can run mongo db in a separate docker container uh and then use links to expose that to your um to your app container uh so it kind of just depends on whether you're going single host or multi host is a big variable but then you can always do you know your own um your own thing yourself right so this is kind of one of the sticky bits is like how do you tell everything where everything else is um docker lets you do the dynamic port binding that we saw in the example but it can also do static port binding so you can just say okay docker run mongo db and expose port 27017 from the container as 27017 on the host i'm gonna fix that port now if you try to do that twice you're gonna get an error but it does simplify things dramatically because if you don't need to then you can address your database from your app server and say well it's on this host and it's port 27017 which is the default mongo db port so you don't necessarily need to jump to fully dynamic configurations for everything right off the bat and so when you want to go to fully dynamic configuration what's the way you're saying patching those um yeah so when you want to go to dynamic service discovery there are a few options there's a project called sky dns um there's a project called uh there's well there's so there's xcd which can can be used to sort of insert information about where services are located and configure processes but you kind of need to glue that together yourself uh and then they air bnb team released a couple projects that are used for kind of doing the same sort of thing uh hepachi works for sort of tcp layer load balancing but not necessarily everything um can be done like that so that's that's probably the um area where the with the most options right now in terms of how you're gonna do it i'm interested to see what the docker team does in terms of kind of unifying that a bit more and it sounds like they're starting to work on that i would just share a few things that i guess in my name is just some questions there when i'm on the development tool there's a project called fade it's like a little just python tool that uh you can write an animal file that just lists like a few types of containers like cd's and all the shelf res containers say for instance this there's really nothing you need to configure just expose the rest and then you can tell it through the animal configuration also just build your application software file so you put that animal file here in your project repo just run the fade and it'll let you run all the services potentially where you just you could run the fade on docker notes and production i guess um or script it as ryan said you kind of lose some of that stuff together production still i think um the other thing that's nice to notice is you uh i don't know i'll try to do you later something about uh fret it's matric open and cache just working on a fused implementation uh for docker so that that's pretty good for uh osx it'll be supposedly completely transparent like docker that should be excuse me not volume shared volume that's that's the biggest thing that's missing for us as a workload development workload thing right now we still need jagan to be our docker ghost and share our volume um but can be really nice so that comes along where it's just a limited area completely um but they say it's close thanks i have a question yeah um so when you talk about uh having docker right for different images so for example if i have a database image then would there be any um like you know cases where i might delete the image and all my data will be gone ah yeah so um that's a good question so with uh docker by default it will try to save things for you so if your um your docker container generates a bunch of files on disk it will keep those around until you tell it to um delete those uh but with the database which is a great example right so database obviously you need um file storage that is has a longer life cycle than just the that time you ran the database right so for that you would use um bind mounts or volumes so the dash v flag uh so you would just create a directory on the host system uh maybe it's a network attached storage or you know high performance disks um you would create a partition for that uh and it would have a path right so it would be like slash data for example and then when you boot your mongo db uh container you would say okay bind slash data from the underlying host onto the um into the container and then everything just writes there so when you stop mongo db uh it'll still be there you can use all of the um conveniences that you might have available like snap uh file system level snapshot backups all that stuff just works because it's all running in the same uh kernel right so if it was different kernels then you'd have to get into things like um nfs right to like share files back and forth but there's nothing that prevents you from bind mounting the same directory into multiple um containers and the kernel will handle that uh because it's basically just a process that's sort of sandboxed so bind mounts yes uh if i'm running nojs on a docker apple or on a docker container how close is the metal line super close thanks Aaron um brine i have a question uh i so are you running docker in production and and are you facing any challenges uh yeah so um for us in production we do have uh that behind the firewall use case that i described um we are we have shipped code climate behind the firewall with docker uh and that is in prod um so for our individual customers uh that is like back on docker 0.6 which is a little bit uh crazy we were early adopters um for uh our production instances code climate dot com uh that's a project which we're kind of actively spinning up right now um to to move everything over to that and we'll probably um try to use uh coro s for that as well so we just have coro s boxes and at least at the app tier which is important um that's actually an important thing to note if you're going to start using docker you don't need to run everything in your entire environment with docker right so for us we have maybe 10 physical servers or something like that a couple of those are database servers or a handful those are database servers those are probably going to be the last thing to switch over to docker because the benefits are just not quite as extreme um it's not like we're actually building we would be building new mongo db images and deploying them out on a regular basis so those can just sit they have host names and they have static ports and the app tier can still be moved to docker and then address those the same way that addresses them today uh so we're going to start by doing everything in the app tier which is the you know the workers that do the uh analysis on our uh code and then the the passenger processes thank you any other questions so brian thank you