 Can you hear me in the back? Awesome. Thank you, Jay. So my name is Michael Rivnak. I work for Red Hat. I've been there for three and a half, closing in on four years now. I work on a project called Pulp. I'm going to give you a little context for that so you understand where this talk came from and where these ideas came from, how they've been valuable to me. Pulp is a big web application written in Python. It has a lot of moving parts, somewhat complex, long-running asynchronous jobs, an additional REST API, web handlers. We have to integrate with a lot of other kinds of services. So it's a kind of work that I do every day. I've found Docker to be just useful to me, just as a developer, not even necessarily to the rest of my team. So I'm going to share with you some ways that it's been useful. So Docker is very popular. I looked at the schedule for today. And there are 15 talks about containers. Just did this conference just today. This is just the first day. And that's pretty incredible. Entire new conferences have come in, new existence. New companies are in existence. Docker's obviously one of them. Containers are just really hot. But there's a real trend to the discussion. That is that it's mostly focused on deployment. I looked at, again, the schedule for today. And you look, you see these fantastic, brilliant technologies, Kubernetes and OpenShift are the ones I'm most familiar with. All these different ways of deploying applications in new paradigms. It's wonderful to think about and talk about. But there's a problem with it. And that's that not everybody can do it yet. Actually, very few of us have the freedom to take apps that pre-exist. And we're designed in somewhat a different mode. And use containers to deploy them. So we're going to talk about something very different today. Besides that, we're going to talk about what can one developer just use, this is a developer conference. So what can you guys do by yourself with just your laptop? How can you take advantage of what Docker has to offer just to make your life easier? So to get started, we have three skills to look at. And one piece of knowledge. Piece of knowledge, this is fairly basic. But I like the basics. Here I've run a Docker command twice. We're going to parse through this because you're going to see a variation on this command throughout this talk. So we're running Docker. I want to run a container. The i means make it interactive. The t means allocate a TTY. Dash dash rm means as soon as this container is done running, just delete it. I don't want to keep the history. I don't want to keep the logs. I don't want to keep the history of the file system changes. Just delete it as soon as it exits. CentOS is a repository. And CentOS 7 is a label within that. And of course true is a process. What does true return in bash? Zero, of course. And false, we see returns one. So this is just to illustrate in the most basic sense many things that we've been doing in bash we can continue to do in very similar ways, but just wrap this containment idea around them. And we can really get some extra benefit out of that, some extra value. Skill number one is bind mount. How do you get data into your running container? Here I've made a directory. And I've created a file and given it a little bit of content. Now we have a very similar Docker run command. But you see in green I've added this dash v option. This is to bind mount. So I've given a path on my local laptop here, my home directory. And on the right side of that colon is a directory inside the container. This directory doesn't have to exist. It'll be created and whatever's on your host will be bind mounted right in. And you can see here that I can read from that file and of course write to it. So this is a great way for example to get your source code that you've been hacking on bind mounted into your container to do something with it. We'll look at some examples. And the next skill is links. One container by itself sometimes isn't enough. Sometimes this isn't very interesting. So here I've run a service. This is an application called Crane. It's part of the Pult project I work on. What it does isn't very important except just to know that it's a web service that exposes a single port, port 80. What's different about this Docker run command you can see I've given it a name Crane. With the dash dash name you put anything there but I wasn't feeling especially creative apparently. So I named it Crane. And then we can see in the output of Docker PS and for those of you who are especially observant you've perhaps noticed that I've truncated the output. I've removed some columns that were not valuable but were definitely causing a horrible line wrapping problem for me. So my problems there wasn't anything interesting there. You can see that we have this running container now that is exposing something on TCP port 80. Back to our Docker run command. You can see in green I've used the dash dash link option. And all I've done is referenced that name that I gave to the first container that I run. I ran. And now I'm in my bash shell inside the container. I run a curl command and I'm just referencing Crane by name as a host name. And this is brilliant. When I discovered that Docker could do this the documentation actually leads you a little bit down a different path to say when you link together two containers look at the environment variables. It's all these helpful permutations of including the IP address and the port and what protocols and these kinds of things. When I saw that you could just reference it by name with the link and then it resolves using normal DNS mechanisms that we're all familiar with is really just brilliant. Nothing could be simpler. So it's very easy. And then we can see I got some expected output. For bonus points, anybody recognize this API endpoint? Nobody. You're just being shy. This is the Docker v1 registry that we had to reimplement in Python. And last, environment variables. You can inject environment variables into a running container at runtime. You see here in green I've used the dash lowercase e and I can set an environment variable foo to a value of hello. And inside my bash session I can see that. So it's a great way to inject just a little bit of information at runtime such as I don't know the maybe the name of some network service that's not managed by Docker but you want to interact with. So these are the three basic skills. With these skills are very simple. We're gonna look at simple examples now. Docker is full of functionality and full of lots of more advanced options. You can do things that are better and more robust than what we're gonna look at right now. But the goal here is to see some real simple, easy, like lazy coder kind of examples of how Docker can be valuable to you just using these three skills. So unit tests, this is a development conference. We all like unit tests, probably most of us at least. Here's an obvious, an obvious statement. They're only valuable when you actually run them. It sounds silly but it's true. Here's a problem statement. We have a large test matrix. So this project I work on, pulp. We have to run in at least two different versions of Python, two six and two seven. A little bit of our code has to run in two dot four. That's pretty painful but we're working on it. We have a variety of different versions of libraries we work with. We work with at least two different message broker technologies. The combinations really get rather explosive and it's difficult to be able to test all of that on your own, just on your own laptop as a developer. What we often end up depending on is something like continuous integration, Travis or Jenkins or some other custom thing to run and then your entire test matrix after you, for example, submitted a pull request. What if you could do all of that just on your local laptop very conveniently? The answer is of course, you can make a docker image for each combination. It's actually very simple. So let's look at a docker file. I think this is the only docker file I'm gonna subject you to today. So we'll just get through it in four easy steps. So step one, I'm starting from a CentOS seven container and I've added information about who created this. Now we just install some dependencies. Don't worry about the details of what's in there. It's just a bunch of Python dependencies that we happen to need. Do whatever setup is necessary at this point. Maybe you would compile your source or something. I don't know. In my case, I had to create this directory and incidentally, this was a bug I discovered just by going through the exercise of containing this particular application. I did not realize that it was depending on the existence of this directory at runtime and it should not. So I think we've fixed that by now but it was a beautiful discovery to find that by going through the exercise of containing it in a minimal environment. And then last, step four is you do whatever's necessary to actually run your tests. So it's really very simple. Any questions? Very informal, so just raise your hand, yell at me if you have any questions along the way. Yes, please. Great question. The question is, is Docker going to run this script every time I run my Docker container? The answer is no. The answer is this Docker file is what we use to build a Docker image. And now I have that image. Ah, and this leads to an excellent, my last point on this slide, is you may ask, where did my actual code get in there? I didn't install my actual code I was hacking on with YUM and the answer to that is you bind mount your code into the running image. So you have an existing image that's been built upfront. You Docker run whatever I named this thing, bind mount my source into it, and then we're off to the races. So you can very easily expand your matrix then and go for CentOS 6, maybe someday CentOS 8, different versions of libraries potentially if you want, maybe different architectures if Docker supports that in the future, and be able to maybe with one bash or one four-loop run, four or six or eight different combinations in that many different containers just all at once. So very fast. Any other questions before I go on? This is another fun example. Complex network topologies. So we had a problem that we needed to solve with a new messaging project. Maybe you're familiar with AMQP, probably many of you are not. So we have this message broker in the middle. We use a message broker is essentially a network-based persistent queue. So a FIFO queue in this point, in this example, and we use it to keep track of jobs, for example. We have workers that sit around watching these queues, waiting for work to do, put a message on the queue, a worker gets the message and says, ah, it's time for me to get to work. We do some other communication that's not important to get into the details of. Suffice it to say, in this graph, we have these clients on two different sides. And we have this broker in the middle. What we needed to be able to do was have these clients in remote locations, maybe physically remote locations, and maybe thousands of clients and aggregate their traffic back to the AMQP broker service on one link. And be able to potentially have that link be redundant or highly available. So some folks at our company came up with this dispatch router solution for us. And what we needed to be able to do now is test it. How can we simulate this kind of a complex topology without standing up a bunch of virtual machines or a bunch of physical hardware actually and setting up virtual networks and a bunch of advanced routing and this and that. What we did instead was we just put, we already had the broker in a Docker container. It's pretty easy. I'm pretty sure others have already done that too. And we just put this new dispatch router is what it's called into its own container and used the link option to specify just endpoints. So each of these blue router graphics, these cylinders you see up there is just where we've run that image. Docker run this, link it to this broker and this client. That kind of an idea. And just real, real fast. You can put this stuff together, test it, and then we were putting messages in one side, watching them come out the other side. It was great. This is really handy. And really, it was very, very simple. It just couldn't have been easier almost with that link option. So build environment. This is something that I've come across at least a couple of times personally. Certainly it's better to have a formal build environment and build infrastructure. We use things like Koji at work. I'm sure you use similar or other things. But there are occasions when you just need something real fast or you just need something real simple. Or you don't have access to your company's Koji and you're waiting. Or again, you want to be able to locally on your laptop maybe when you're on an airplane flying over the Atlantic on your way to a conference. Be able to try building an RPM or building some other kind of package in a variety of different environments all just on your laptop. And you don't want to have to run a virtual machine for every single one of those environments. Solution is, just create a Docker image. It's proven real valuable to me as a packager trying to package things for Fedora and Apple, for example. Trying to reconcile one RPM spec file that can work across multiple different versions of the operating system. Having a Docker image for each one that I never even plan to share with anybody else. So I just use it myself on my laptop to be able to run it through five or six different kinds of build environments just all at once and see what happens. Which is really, really incredibly helpful. Saved me a ton of time. Simulating load is a common problem a lot of us face. Have you ever done something like run, you're working on your web service, you have a REST API, something like that. And you want to find out what happens when you have 100 clients connecting to it instead of just your one, manual one. I've run Curl with XRs before. It's a nice way in some kind of a loop to see what happens when you get 100 processes. It's a real cheap way to get a lot of requests going at once. Sometimes that's not good enough. Sometimes you just need a little more flexibility or power. Example we had in my project is we have an agent that runs on client machines and manages them. You could think of it like Puppet Agent or something like this. They don't like each other. They don't like being on the same machine with multiple instances. They want to really stretch out and own the entire machine on its own. So putting them in containers is a natural way to run a lot of them on one machine very cheaply without having to go through, again, the expense of virtual machines that take up a lot more memory, a lot more disk space, they need maintenance, you have to boot them, all these kinds of things. It's just so easy to make one Docker image and run it 100 times and see what happens. You can do really whatever you want. I think it's very, very simple. State reset is one of the most valuable things I think of Docker. The idea that you can run a container from an image and do whatever you want to that container and then blow it away and run it again and be right back to your original pristine state is so powerful. And this is what that example is all about. So the problem is that you need to quickly reset some database state, for example. So perhaps it's running integration test suites, something I've done in the past in Python, the Django framework helps you out with this. You can run tests in a transaction. So you would open a transaction, make a bunch of changes to your database through whatever your test is doing and no matter what happens, whether things go good or things go bad, at the end of your test, roll back the transaction and your database is ready for the next test. Sometimes that's not viable. Right now, the project I work on uses MongoDB. We don't have transactions. So Docker gives us a fantastic way to have a pre-baked image with your database ready to go, maybe you even pre-bake your data into that image. And then you can, after every single test, throw it away, start a new one, throw it away, start a new one. I've even read about, not actually done this myself, but others have created essentially a pool of available database instances ready to go. You could keep a buffer of maybe 10 or so available. So as your tests are finishing, they just grab one that's already available and running and throw the old one out in the garbage. It's really just a fantastic way to take advantage of Docker's ability to quickly reset your state. A demo of your application. So I was telling you about pulp. It has all these pieces. It's fairly complex architecturally and in terms of all the different kinds of services it has to interact with and thus the different libraries it has to use and so on. It's actually fairly challenging to think about how we would deploy pulp as it is in containers, like using something like Kubernetes. It should be doable, but probably we'd want to change the way that some things are designed, change the architectural decisions. You have specific challenges like shared storage. That's very important to us. Shared configuration across all these different components. How do you get secrets from one place to another? I know there are lots of new tools that are showing up to help with that, but still it's a challenge. Some people are versed to running certain services in containers at all, like databases, for example, although that's gotten easier as of late as well. And then, of course, updating all these parts is a challenge. Do you have a base image that you update and then rebuild the other pieces on top? There's all these challenges to actually do it right. So what happens if you want to containerize your app but you find also this challenge of, man, it's just a lot of work to do it right? Just do it wrong. So this is my solution. We have a demo available of our application that you can run and it has red flag warnings all over it. Don't do anything important with this. Don't use this for real. Don't give it real data, but you can find a bash script on our website and run it and it'll fetch the Docker images, it'll start them up, and you'll have an entire running pulp with like eight different containers all working together using the bind mount for shared storage, using linking to let the services talk to each other as appropriate. I think it even does use environment variables to inject a little bit of runtime state in there and then you can try it and use it and throw it away when you're done with it. I like to use it to reproduce bugs or do demos of pulp or all kinds of things. Because again, it's so easy to, that one, it may take several seconds for it to start. We have a database that has to start and this kind of thing, but even the idea that in a few seconds, I can throw away an entire deployment of my application and have a new fresh one back where I started is just incredible. So there's some tools that you can make, that we can make. Maybe I'll make some of them. Actually, I have made one of these. One is an MTA, mail transfer agent, I think is what the T stands for, right? Have you ever worked on an application that sends email? And if you have, I probably a lot of us have, I know I have, have you ever done what I did and actually, okay, you're hacking on the email feature, you wanna see if it works? Have you punched in your real email address into your app, create a user, real email address, do whatever the thing is that triggers the email and then go check your Gmail account or whatever and see what happened and see what it looks like, right? I've totally done that. So funny story, I have a colleague who did exactly this, he was working on a different job on a big Django application and not only did he punch in his own email address, he volunteered a couple of his colleagues and punched in their email addresses too. I mean there are developers on his team. Something he had done went wrong and he I think ended up sending like a quarter million messages to each of them or something like this and this company was using exchange server at the time too. So that poor thing basically just melted and it was, it was at least a day or more to recover this entire company's email infrastructure because he made this mistake. So what could you do with Docker instead of that? We could have, this gets back to this general concept of it's so cheap to have a Docker image sitting there. You may be a few hundred megabytes of storage to keep one around and that's it. On like a virtual machine where you need gigabytes of storage probably and you have to boot it to use it, you have to maintain it, what if it breaks or you screw something up? Now you have to actually fix it unless you did configuration management type things and made it easily reproducible but then you need to reproduce a whole VM. We can have this tool bag of images sitting around and this is one of them. You could make a Docker image that has something like post fix or send mail in it and receives this mail and does something useful with it like write it out to a file or just throw in a database that you're gonna look at or make it available by IMAP so you can actually go browse and look at it and see what's there. That would be just the most useful thing that you could fire up anytime you need to test the email feature of your application. Now there's proxies. So we sell a product to customers. They deploy it in their infrastructure and they use a variety of proxies it turns out. It's not all just squid. So we have to ensure that what we do can operate both behind their proxies and our access out to the world or even within works through their proxies and that can mean a lot of variety. So do you wanna have a virtual machine sitting around for squid and another one for something else and another one for blue coat and this? Do you wanna be an expert in how to deploy these things in a reasonable way or configure them in the way that your customers normally do? Probably not. So one solution is you or your team could have a pre-baked Docker image that has one image for each of these different kinds of proxy technologies and be able pre-configured. It's ready to go. Everybody understands how to use it but nobody, well somebody, but not everybody has to understand how it was made. You just run it, use it, throw it away when you're done. Just super easy. Fake APIs is a nice easy one. If you want to, if you integrate with something like GitHub's API or something, we integrate with things like the Docker API as you saw, the Puppet Forge API and the Python Package Index API and these kinds of things. So if we were going to write some tests and perhaps integration tests but didn't want to depend on the internet and all the things that come with that, Docker image is a perfect place to run a very simple fake API. Have it available, very quickly run it. You can use that link feature to link your, maybe another image with your actual integration tests, link them together. Really, really simple stuff. It's really powerful. This is the most recent one that I've been working on. So my application, we've made some changes to the way we interact with the database and the way we've defined the schema and we recognize that this is a little bit risky. We have to be very careful that we don't screw it up. And in order to do that, we want our users to help us with their actual real data. Let's see, on our development team, we don't have a version of Pulp that's been upgraded eight times from Pulp 2.0 to see what that data looks like. But we have users who have. So we've made a little script that interacts with data, pulls some data out of MongoDB, ensures it can use it and read it sanely and write it back. And now we're creating a Docker image to run that in and a separate Docker image to run MongoDB in. So we're gonna hand our users a very simple little script that will help them dump out their database, their real live production database, dump it out, load it into this MongoDB running in a Docker image, and then fire up this little upgrade test script and try to read all that data and see what happens. And that's gonna give us tremendously higher confidence that what we're doing is gonna work for all of our users and it's super simple for them. They, everything's contained. Thinking of Docker as this kind of packaging technology, if you can ship stuff around and hand it to somebody and it's real easy to run, is one of the really most powerful aspects of Docker. So those are all the ideas I'm gonna share right now. There's some next steps. There's a lot of great information. There's already been a Docker 101 today all the way through very, very advanced Docker things. For more information, right on Docker Hub, there's great, great tutorials and documentation. If you wanna get started there. The registry, we have our own registry where we have our Docker image. If you're into using our stuff, you can use our Docker images as well. And then a lot of different distributions have a lot of pre-baked images ready to go. Fedora's the one I'm most familiar with on a daily basis, but things like MySQL and Postgres and MongoDB and Message Brokers and Apache Web Server and Nginx and Development Environments for Python and Ruby all have been pre-baked and available for you to just go run them and play with them and make your own tool bag of little things that you can use in your everyday life. So with that, that's all I have. I'll take questions now. And also if you guys have, likewise, any similar ideas to this or an example of something that would be useful to you, you could tell us about it. We could talk through how it might work. So what do you have? I missed a couple words. Could you just repeat one more time? Okay, so the question is, if you have one machine, have I tried to run containers on that one machine with my colleagues? No, I've not tried that. This is the boring answer. I basically just do all this on my laptop. It doesn't take much. A container is really just, you're just running another process just like you would, but wrapping it with some extra stuff. So we've never really had a shared scenario like that. I see. So yeah, so he's saying he has one powerful piece of hardware that you'd like to share among them. Docker runs great inside virtual machines. Open shift. Or yeah, install open shift on that sucker and use that. Yeah, what else? Yes? Sure, so the question is essentially on my team, who is responsible and how for creating things like Docker images out of our application? The answer is it's a shared responsibility of the team and it's not very important. Like our users, except for this last example I gave of this upgrade script, we don't expect any of our users to deploy pulp in Docker containers. If any of them are, I'd love to talk to them and find out what they did. It's actually pretty challenging. So as we find it useful to do so, we can create little tools like this and do that, but otherwise it's essentially a task like any other on our team. We put it in an issue tracker and somebody picks it up, who's qualified. Does that answer your question? Could I tell you more? Yeah, I agree 100%. He's observing that there's a lot of ambiguity now and uncertainty in the development cycle of who's responsible for what. I agree, there's actually a lot of risk. Something I worry about is, this is going a little off topic, but that's okay. It's the end of the day. For example, the exercise of going through traditional packaging like RPMs or Debian packages gives us a really nice formal checkpoint for things like the file system hierarchy and just other general best practices of deploying your app. Have you done reasonable things? And if no, somebody's gonna review it and slap your hand and say, don't do that. So the idea that your development team, I know developers, they're not always the most responsible employers of things, that your development team can just shove their app into a container, however they feel like it and hand it to the ops team and say, here, run this. It's kind of scary. I mean, it seems like the future. And the future's now. Like that's happening and it was working for people. But people are acquiring a bigger skill set to some extent. A unique aspect of where I work is because we write software for systems engineers. Our software engineers tend to have higher than average systems engineering, at least awareness, if not outright skills and experience. What else? Back on topic, yes. Ha ha ha. Mm-hmm. This is the million, the million crown question. Like, what country am I in today? Yes. The answer is, it's very difficult to answer. The question is, where can you get trusted Docker images? How can you trust what is in your Docker image? If you're just downloading it from somewhere, even if that somewhere is the Docker hub, even if you think very highly of Docker as a company, as an open source project, how much trust do you have in their infrastructure and in the, just any user, anybody in this room can put an image on Docker hub and claim any name that is available as a username. So if my name, if I hadn't already signed up, you could post as me and start posting Docker images. So what I would suggest to you is don't do anything important with images you get from some strange place on the internet or even some very normal place on the internet. What I know as an employee of Red Hat is we, that's part of the value we offer to our customers, that we have our Docker image that is trusted and designed to be that. Yes, in fact it is, yeah, I mentioned right there. So we offer that, we may not be the only people offering that but that's what I know and am aware of. So I would look for something like that or roll your own, you can make your own. But that's a very important point. And part of the reason why everything I've gone through today is simple and easy and don't, not mission critical. Like if you screw it all up or it tries to do something bad, hopefully it's not the end of the world. I development in a veyor and VM, you know. So all of this is meant to be disposable. And doing more important things that brings up those kind of questions. Speaking of questions, yeah, I've got a few more minutes. What do we have? Yes, yes. The question is he'd like me to repeat and perhaps elaborate slightly, re-explain this idea that for testing purposes, for integration tests where you need a live database, you might have a pool of available running Docker containers running that database that you could pull from. So the idea starts from the simple case of I have a MongoDB Docker image. And for every test that I run, I run a container from that image and I write some data and read some data and do whatever the test does. And when the test is over, I stop that and throw away that container. Next test, start a new one and repeat. That introduces a little bit of latency. You know, hundreds of milliseconds, I think is probably approximately right for what it is to start a container and throwing them away. It probably has some small expense as well. So if you're worried about that sort of thing, if you have very small compact integration tests that run quickly, maybe this would be valuable to then keep a pool of maybe 10. So this would definitely involve you writing a small tool to do this. It could be as simple as a little Python or a Bash script to just manage this for you and keep a few of these running. And you essentially need something to start a bunch and then something else that gets one and looks for the name. You remember that I named when I was doing the linking, I gave it a name and then referenced it. You can let Docker will assign randomly generated names if you let it. So you could just look for an available name. I have something that looks for an available name and hands it to the test and then at the end deletes it. Then looks for another available name, hands it the test and then deletes it. And maybe starts new ones off on the side, something like that. Is that good? Okay. We're probably done. Two minutes. Couple questions. Or we can just go get dinner. Going once. Going twice. One, yes, right here. So, which component are you talking about? For example, database. Oh yes, a database. Okay. Sure, yeah. So the question is would you for something that you do want to run on bare metal or well that you want to run uncontained and not in any kind of Docker container, would you use some configuration management to integrate with these other ideas of spinning up integration tests, use something like Ansible or otherwise or Vagrant? Yes. Partially. We run our whole app, our development environment, most of us on the team at least, in a Vagrant virtual machine that's easy to destroy and recreate. We don't tie that process necessarily to anything except crap I broke it and it's not worth fixing so let's just throw it away and rebuild it. We should probably quit right at the time. So thank you very much. Oh, please fill out the survey. If you guys enjoyed this especially, give me some feedback I can share with my boss. Thanks. Oh, I did forget to hand out the swag. Ah, let's see, this guy over here, they'll ask some good questions. Every 10 minutes. Okay guys, so we'll be starting the lightning talks in like 10 minutes, So we will not have enough time to test, so if somebody wants to test slides for the talks, come by and test that the projector works, otherwise this will be taken out of your time.