 So check. Can everyone hear me? Good. All right. So I'm going to talk about a project I've been working on called Codetainer. And it's a browser-based sandbox. And I'm going to put that in quotes, and I'll tell you why in a minute. I built it using Docker. A little bit about me first. So I've spent my career as a software engineer in security, basically. So I started off as a kind of security analyst and moved more into software engineering. I worked at security companies primarily. Symantec was one. And Mandiant, which was kind of a high-end incident response company doing R&D there. And I'm also an entrepreneur. So I co-founded a company called Threadstack, which is actually based here in Boston. We're over by the seaport, actually. So I actually left Threadstack recently. And now I'm working as an EIR at a venture firm called Accomplice. And I just realized I tell people what that is. And people don't know what EIR means. Basically, it's an entrepreneur in residence. And I just tell people I don't really have a real job, because my job is to kind of look at interesting stuff that's going on in terms of new startups and what they're doing, new technologies, and how that's changing, how existing companies do things, and if there's any opportunity in the market to build something interesting. In addition to all that, I organized the Boston Building Meetup Group. We meet in Kendall Square every month. And we usually have interesting talks about Go. We're going to be doing some more kind of hands-on, like introduction to Go classes. So if you're in or near Kendall Square, I recommend that you come by there. And I think that's it. All right, so what is Cotainer? So as I mentioned, part of my quote-on-quote job is to look at interesting technologies. And one of the things I've been interested in as an engineer and also from a market perspective is Docker. And obviously, you heard in the last talk, Docker is being used for the various use cases. One of the really interesting use cases is the microservices use case, the ability to kind of package up all your code, all your dependencies, in this kind of tightly knit unit that's easy to deploy. It can displace a lot of configuration management activities. So that's one interesting use case. But the other interesting use case that I've been looking at is, well, a container, in some respects, is also kind of like an isolated boundary. It's like a kind of miniature virtualized system. Most of you have probably used real VMs before. And containers are a little bit different. So one of the use cases that I was looking at was the whole try X in your browser, where X can be, for example, learning a new tool or learning a programming language. A lot of these are implemented using kind of like a virtual version or like an emulated version of a programming language or some kind of custom sandbox. So here are a couple of examples here. There's a kind of a try get which runs you through a get tutorial. The one on the right is a little program that allows you to try a tool called bro which is used in network intrusion detection. So I was wondering, is there a way to build something that's more generic, that can be reusable and composable for multiple use cases? And my inspiration for this was this project called CodePicnic. It's actually a company. So if you haven't looked at this, it's actually really cool. You basically can log in. It's a hosted SaaS service. And basically you get this kind of browser-based Docker widget that you can put in your browser and use it to do things like demonstrate APIs, let the user play around with some code. So it's pretty cool. But I was looking at it and there are some problems that I had from a security perspective. For the most part, the containers are running as root on the really code picnic systems. And that allows you to basically install any tool you wanted on these containers. And the other problem is, well, if I can install any tool and this has network access, I could probably use it to abuse other systems. And CodePicnic would be responsible. So I could install a DDoS tool and spam someone else. So if I wanted to run something like this on my website, I wouldn't want the ability for a user to do that. I want to be able to isolate what the user could do. Maybe if I'm demonstrating a tool or a programming language, I don't need the container to have network access outbound. I don't need to run as root. And Docker actually has the capability to implement some of these restrictions. So just talking through some of the more of the use cases we talked about most of these. I thought another interesting use case would, when you think about training people, there's a lot of tutorials on how to code. There's less tutorials on how do I use this sort of Unix tool to maybe debug something or I want to learn reverse engineering. There's no real tutorials on doing that that you can interact with. And I'm the kind of person that likes to, well, I can read something, but I just totally forget it. So I actually need to interact with it to be able to learn things. So I thought a widget or a tool like this would be useful in implementing some of these debugging training and other different use cases. And another use case that is not really covered in the CodePicnic use case is the remote management. There's a browser. You can have a browser connection inside your container, basically. So that might be interesting for some people. So the requirements for this project. I wanted to be flexible and powerful enough to support these multiple use cases. I wanted to be programmable, API driven. I wanted to be able to host it myself so I can actually do the third thing, which is to be able to secure the told Docker and host, Docker host instance around it so I can turn off networking. I can apply like an app armor and SE Linux profile to limit what the container can actually do and generally have control over it. So I built this as an open source project. So let's talk a little bit more about the technology that makes it possible. How many people here know what Docker is? Pretty much everyone. How many people have used it in some capacity? About half of you. Pretty good. So as I mentioned, Docker is a kind of virtualization, right? So the way I like to think about it when I explain to people, most people know what VMs are. It's virtualizing the whole system. But the way Docker works is basically virtualizing a process. So a container is really a process that runs and it has its own virtual file system. So it thinks some certain path is actually its root file system and it takes advantage of something in Linux called namespaces, which allow you to segment off what each process can look at. So one of these things, for example, is networking, right? Maybe you don't want this container to see the same network activities that another container can see. The other things would be certain system calls. There's a whole slew of things that the Linux kernel supports in terms of namespacing. So more, a little bit about, is this not working? Well, I'll just talk through it for now until I can fix it. So another advantage of Docker versus your typical VM sort of virtualization is that it's very lightweight. So when you're shipping around a VM, you create an image, that thing could be several gigs. It's gonna take several minutes to actually boot up an operating system on it. And it's not good for use cases where you want something to spin up fast and maybe die really quickly. So in terms of processes, processes start up really fast, usually in seconds or less. And the file system changes just because of the way Docker was built can be measured in terms of kilobytes or megabytes instead of like these huge gigabytes. And the same thing with the memory footprint, right? It's just however much memory that process is using is what the container's gonna use. Images are kind of layered and reusable. So another cool thing about Docker is because it's shipping its whole file system that's basically this huge tar GZIPT set of layers. So you could basically build your custom application image on top of an existing image. So if you guys have used Docker, you probably heard of Docker Hub, but Docker Hub is a registry for people to do exactly that. So what people can do is actually inherit from an image someone else has built. So someone else has built, let's say an Ubuntu 14.04 base image. You can just inherit from that in your Docker file and actually work from that and build your application on top of that. You don't have to, you can take advantage of that other person's work building that operating system image. And the other cool thing about Docker is that there's powerful introspection and management API. So I can really script the creation, the destruction of containers, the pulling of information outside of it, looking inside a container, which is harder to do with traditional virtualization. Cool, it's working in. So just a little bit more of an introspection. Here's the API. It's published on Docker Hub and it's actually pretty comprehensive. So, Cotainer is a system, as described before, it's like code picnic. It's basically a browser widget that allows you to connect to a container and run certain actions. You can program that widget for certain use cases like doing a tutorial or teaching someone how to code in a way where you don't have to emulate the repel of the programming language. You can just use the actual language and limit what the container can do using Docker's security properties. So the way this works is there's an API server, that's basically the core of it, and it just sits in front of the Docker API and it's spinning up these Cotainer processes which are really just container processes that you can apply certain profiles to that limit what the process can do. The API server has various routes that allow you to do things like spin up Cotainers, register an image in Docker for use in the Cotainer and stop them, et cetera. And so, just some of the components. Whoops, a Cotainer is just a Docker container. I think I covered all of these. The image is really just a Docker image that's registered for use in a Cotainer. And a profile is a profile whose language sort of matches the Docker API configuration for its security. Other tools I used to build this include Go. So I just have ended up doing a lot of programming in Go recently because it's pretty, I don't know. I feel like it gets the right balance between flexibility and speed of development. I like that it's statically typed. I like how uniform the code is written, how uniform the code looks when it's written. Like you look at one Go project, they all look the same. It's just really addictive to write in. And also the Docker APIs are written in this so it would be easier to take advantage of them using the same programming language, et cetera. X-Term.js, which allows me to render terminal characters in the browser. And SQLite, which I use for storing basic metadata about the Cotainer, like what Cotainers are running, what images are registered for use in Cotainer, what the profiles are, et cetera. So just to give you a little kind of feel for how it works. You start the Cotainer server, which is the web server that I described before. And there's also some command line actions you can do while I show you later. First step is creating an image. And you can do that with the API here. It's just, you just do a post to, I don't know why I have animations on there, but I did. But basically you just do a post to slash API v1 slash image. And you're just basically registering a Docker image that you have on that machine for use with Cotainer. So you can see here I'm registering a TCP dump demo image that I made with Docker. Step two, to launch a Cotainer is really simple. You just basically post to API v1 Cotainer, and you want to supply the image ID and an optional profile. There's an optional name as well. Let's just kind of use for tracking within the database, really simple. Step three is interacting with it. So there's a few APIs that are around interacting with it. The one that you would probably use in your browser would be the view API, which basically renders an HTML terminal view at full, every size is automatically. So the way you can use this is the same way you'd use it in code picnic, just basically slap it in the iframe. Unlike code picnic, there's actual APIs that to interact with it a little bit more granularly. You can actually attach the web socket that allows you to send and receive data from the Cotainer terminal, and send is kind of just a wrapper on top of that that allows you to send individual keystrokes. So more, so there's APIs to actually interact with the files inside of a Cotainer. So you can list files from a particular path, download a file or upload a file. So this would be really useful if you're building a code editor example. Maybe in your application you have your own code editor, syntax highlighting, validation, you have a run button, you just basically copy the file over to the Cotainer, run it within the REPL there or however you wanna run it, and then get the results back. So I mentioned there's a command line thing too, and basically just wrap some of the APIs so you can use them on the command line so you can list running containers here and then register an image. Some of the challenges with building this were dealing with things like missing APIs in the Docker API. So the Docker API is pretty thorough, but there were some things that were missing, like I couldn't list a file from a particular path in the file system. I could actually pull files off of it if I knew where they were or upload files to it, but I couldn't really list things. So I really wanted a way to do that. And I also wanted a way to be able to minimize risk of abuse by sandboxing what a Cotainer can do. As I mentioned before, Docker has these knobs that allow you to kind of mess with the security and isolation properties of a container. So this is actually straight from the Docker API documentation. So you can see things here, you can specify the number of CPUs you want it to use, memory, what ports you wanna expose if you want networking on or off. There's a whole bunch of different things that are very useful when you're trying to sandbox what a container can do. So I basically took advantage of this and I created a spec which I call a profile. And it's basically a JSON file that has a similar format to the Docker API JSON that allows you to specify specific things you want triggered for a container, a Cotainer. So the way you use it is you just register it with a name. So what I'll do, it'll just slurp off that JSON and then you can pass its ID as a Cotainer profile ID when you actually create the Cotainer. So going back to sort of the missing API problem, the way I thought I could solve this was basically Docker, obviously, you can execute anything you want inside a container using their exec APIs. So I was like, well, why don't I, for each Cotainer that I start up, just mount a directory at a certain reliable location that has the tools I wanna use. So for the file listing thing, I just created a small little tool in Go that listed files from a given path. It returned some JSON and then I just built an API around that and bam, I could do that for every Cotainer that I was running. And this is just kind of how it works. There's that create exact thing. I don't like showing a lot of code, but there's code there. Okay, so I'm gonna pray to the demo gods because I'm gonna try to do a live demo. Let's see how well this works. So I'm gonna demo two things. One, just basic like how do you create a Cotainer? Like those three steps that I described before. And the other one is like a little app I built that kind of is a tutorial around LSOF. So let's see here. So we're gonna basically, the first step is we're gonna build this Docker file. So if you look here, this is a Docker file that has Htop installed. So I'm gonna build a Cotainer that's his whole purpose is to demonstrate Htop. So, I'm gonna do that and it ran really fast because I've clearly built this before on the machine. Usually it would download different pieces of the file system, but there we go. I've basically, if you look at the command line, what I've done is I've used the Docker build command to build my container or my container image really. And I tagged it with a name of Htop. So what I'm gonna do here then is register this for use with the Cotainer thing. So I'm gonna use that register command, image register, Htop Latest. So it registered an image, great. So now I can actually use this to actually launch Cotainers within my API server. So now I'm gonna actually spin up a Cotainer and hopefully this will work. So I'm gonna use the create command on the command line. Again, this just kind of is a easy way to do it rather than using the curl or using the HP APIs. And I'm gonna specify the name of the image I wanna use and then the name. Looks like it succeeded. So let's see what it looks like. Just kind of this, this is the, it's my other demo. If you can get, see I knew this one. Copy and pasting from Tmux is not easy thing. Barely works, there we go. All right, so you can see it created my Cotainer and it's just a browser widget and there's htop and you can run it there. Yay. You know, the cool thing about this widget is like you can resize it to whatever, you know, whatever it auto resizes, which is one of the most annoying parts about building this because I'm really bad at HTML and CSS. So there's that. So just to see what a more full use case, I started putting together this tutorial which, you know, isn't that great because again my HTML, CSS skills are not awesome but you know, it's meant to be an LSOF tutorial, right? So you can see the, I have this Node.js app, what it does is it makes a call to my Cotainer API server and if there's no, you know, Cotainer ID in the session, it creates a new one for this browser. So I opened up a new one and like an incognito window would create a new one for this new session. So you can see here it's 07.4, this one's A6A. It's got a different host name because it's a different container. So each session gets their own sort of sandbox to play around in and you know, so some of the cool things you can do, like, this is, again, there's like two things in this tutorial but one thing is like teaching them like what the LSOF-I command does and even though this stuff is running in an iframe, right, normally I wouldn't be able to, you know, interact with it but because there's an API behind it and I've proxied that through my server, I can basically send it commands so you can actually confirm the output of this. You can type it yourself if you want to but if someone gets stuck, they can, you know, they can just run that command. You can have clear steps. One other thing I wanted to show but didn't have time was like the whole file-save use case. Like you could have a file editor in this window, ship it off with the kind of file-save API and then run it in the Cotainer and show the output there. So back to the slides. So the status of this project, it's a little bit alpha, it needs some more stuff to make a production ready, like auth I think would be important. Documentation, testing. Oh, one of the other things I didn't demo in the LSOF, I actually used a profile with that to limit. Usually in a Docker container you can't run LSOF but I was able to create a custom app armor profile that gave my this particular Cotainer permissions to run it. So either profile thing is really powerful to allow you to tweak what the Cotainer can do. So that's the GitHub URL, it's all open source right now. So if you wanna contribute, let me know. I am going to start putting more issues, like there's a bunch of stuff there already I need to kind of finish up and start working on. Or you can just ping me, that's my email and my Twitter handle. And yeah, if you have any ideas for it, I'm more than welcome to hear them and welcome collaboration. So on the back end, what were you running the Docker containers in? What do you mean what I was running it in? I was running them with Docker or? Oh, I mean yes, but you just use on a regular Linux system, no other? Oh yeah, I use Vagrant on my machine. I do it on my dev in there. Awesome. Yeah, it's just regular, I mean Docker has its own requirements, like you need basically a modern kernel. And this project in particular, I'm using the latest version of Docker because I want all the APIs, they've released pretty, they've released new APIs pretty frequently, so I don't know if it'll work with like 1.7, so you need 1.8. It's 1.8 though. Yeah, 1.8 is a kind of modern version to use this. Hi, it's a great presentation. My mind's a little bit blown right now, but you mentioned there were some missing APIs in Go, sorry, not in Go, but in Docker. And I was wondering, have you talked to the Docker team about it? I started looking and then creating a pull request and I was like, I can't be screwed, like it way easier just for me to mount a volume with the tools I want. I'm assuming if you look at the Docker issues, there's literally like hundreds, maybe thousands of open ones, and they're really responsive, they're not just sitting there, but I haven't dug through to see if someone already had that request open or not. Like I said, with each release, they add like a bunch more APIs, so I expect the whole file listing thing will be added soon or later. Great, thanks. Yeah, this is a Docker question. I've used Docker in previous companies, and I'm not sure if this is still a limit, but it at least used to be that if you ran multiple Docker containers on a system, you had to specify sort of a hard-coded memory amount that they would take, and I noticed in your configuration it had memory swappiness. So if I've got a couple Docker containers and one of them is being quiet and the other one's busy, do they still, is it still hard-coded? They each get the memory they get? I don't know if it's hard-coded. I would guess I'd have to see how they implement it using C-groups. I don't know if that reserves it automatically or what. I know that this parameter here, there's memory and then there's memory swap that you can also configure. It's a different parameter. Yeah, I honestly do not know specifically that. Okay. Baffle you all. Ha ha ha ha ha ha ha. All right, cool, yeah. If you're interested in Go or Docker, come grab me. Like again, come to one of our meetups. It's pretty fun. Yeah, and if you want a job working in Go or Docker, I'm working on some interesting stuff that will come out soon. So probably shouldn't recruit from the constant contact people, but there are any visitors here. Question. So what are you working on next? Especially in this area? How we say stealth. Stealth right now. Yeah, if you like early stage company, like yeah, just come talk to me. If you want to work with cutting edge tools and things, I have opportunities. Thank you very much. Thank you. Thank you. Thank you.