 Now we have Ed Scotton, who's working on Cloud ABI, a way to make this a lot more easier by building a sandboxing-friendly POSIX runtime environment. Please give him a warm round of applause. Thank you. It works. Yeah, good. So it's actually nice to see like such a full room over here. There's only like 10 or 20 spare seats left. The last time I gave this talk in Germany was back at the Foscon, which was in September, I think, conference that's in a place close to Bonn, and what was a bit of a shame was that they planned my talk right at the end of the first day, right at around the time that they started firing up the barbecue, which meant that like I only had 10 or 20 people attending my talk. So it's really the opposite experience of what I'm having over here. So I'm, oh wait, my laptop turned itself off. Now it works again. Yeah. So before I start this talk, let me first sort of introduce myself quickly. My name's Ed Schouten. I'm an open-source hacker from the Netherlands. Back in 2002, 2002, when I started studying, I became really interested in how open-source operating systems worked. Before that, I was still a Windows user, but then I started using Linux, and a year or two years later, I also started using FreeBSD. I used OpenBSD for some time. Most of these sort of commonly used open-source operating systems, I've even tried running them once. Some of them I liked a lot. Some of them I didn't really like, but the ones that sort of stuck with most is FreeBSD, as you can see in my t-shirt, and I also use Linux on a day-to-day basis as well. So back, late last year, I actually lived in Germany, and I had my day job where I did a lot of soft-working over there. But there was something that really sort of caused an itch that wasn't that I couldn't scratch. That I couldn't sort of fulfill at my previous employer, which is the reason why I left and decided to start my own small, small scale IT consulting company called NUXI, which is just Unix with byte-ordering issues. So enough about me. Let's talk about cloud ABI, which is what I've been developing for the last year. Before I'm going to explain what cloud ABI is, I'm first going to explain what I think is wrong with Unix, and lots of people have different opinions about what's wrong with Unix. So, you know, this is probably just bike shedding, trolling. You know, we'll see at the end of the talk during the Q&A whether you people agree or not. So even though Unix is an awesome operating system that I've used for more than a decade now, there are like two issues that have never really been solved in my opinion. So the first of all is that the system itself doesn't really stimulate you to run software in a secure fashion, and what I mean with that I'll explain in a minute, and it also doesn't really stimulate you to write reusable testable software, which I'll explain afterwards. So let's look at like a very simple use case. Say you have like a Linux server running somewhere and you want to run a web server on it. It could be Apache, it could be Nginx. It doesn't really matter. Just some web server. This web server only needs to do like a handful number of things. There's a really tiny number of things if you really think about it. So it needs to accept incoming TCP connections on which it will receive HTTP get requests and, you know, somehow process them. Maybe it needs to access some files on disk stored in some kind of directory. Maybe if there's like some kind of, you know, for example, a Python or a PHP script in there that connects with some kind of database back in, it also needs to open a connection to that and then perform some transactions, computer response and send that back across the TCP connection. So this is just really, if you sort of look at all of the possible things that a Unix program could do, this is just a really tiny amount of functionality. But what you see is one such a program is compromised and the attacker can basically just do anything that that user can do that the web server is running as, which is quite a lot of stuff. So first of all, could just create a like a parable of all the world readable data under slash and, you know, if your file system permissions are set up in a correct way, then this might not be not be really damaging, but, you know, getting them set up for your entire file system correctly is in practice pretty hard. And also what is world readable within your company might actually not be world readable like for attackers on the other side of the internet. So there are also some other nasty things that an attacker can do. So for example, it can still invoke set UID utilities that are provide quite a large attack surface. So exploits in those programs can be, of course, you know, used by an attacker to gain even more privileges. And even if that's not the case, an attacker can still do some really nasty things. So say an attacker would manage to break into the web server, install some kind of, you know, back connect service where it can, you know, create login sessions on that system. Even if you would just update your web server to a version that's no longer vulnerable to a certain exploit, an attacker could have just registered a cron job as that user and just spawned it up every couple of minutes and be back in again. And even if all of those kinds of things, you know, accessing the file system is, you know, really well fenced off, an attacker can still turn the system into a botnet node, you know, and take part in sin flooding attacks, sending spam, that kind of stuff. So there's a huge disparity between what a program on Unix should be able to do and what it can do because that's just how simple the Unix security model works. And you see that over time, people have introduced sort of frameworks and new features to the kernel to sort of make it a lot better. And in my opinion, you can sort of divide them in two different categories. And the first one of them is based is adding more access controls to the system. So the mindset being traditional Unix permissions are sort of not granular enough, not precise enough. They cannot be used to express the actual restrictions that need to be placed on a web server, for example. So we're going to extend that. And on Linux, you see that one of those frameworks is SE Linux, but another one is App Armor. And App Armor allows you to create security policies for every application. You can create sort of an additional separate security policy that's stored somewhere in slash EDC, and that can be used to sort of restrict the application even further. But in my opinion, that's not a real solution to the problem because what happens is that a software engineer just develops some kind of web server, releases it, throws the code over the fence to the people at the distributions, and there are the people that have to write these security policies. And they actually don't have the full understanding of how the application works. And in some cases, end users might actually need to modify these policies to get their application to work correctly. And I think that this is sort of not how it should be. It makes security a lot harder than it actually needs to be. And the problem is that, for example, going back to our web server, for example, again, if you would change, for example, the root directory of your web server in your configuration file, you also need to update it in your App Armor security policy. Well, what happens is that some user tries to do this, and for some reason it doesn't work anymore because he forgot to update the security policy. So then he googles around a bit, and then he just sees like, you know, this is caused by App Armor. I had the same thing on my Ubuntu system. So what does the user do? App get, remove App Armor, problem solved. So in my opinion, these kind of systems, they simply don't work right. And then there's a second class of systems, and, you know, over here I've called them capability space, but they're not necessarily capability space. But, you know, just for the sake of the argument, let's just, you know, throw a sec comp on Linux and Capsicum just in one big bucket. I'm now going to explain how Capsicum works in a bit more detail because as you'll see what we need to disknowledge later on. What happens with Capsicum is that you just create your web server like your Apache server or your Nginx. It starts up like your regular Unix process, and the first thing that it does is it parses the configuration files and the configuration files that list, for example, the IP addresses that the system needs to listen on, the path names of all the root directories, and it sort of opens up all of those directories and all of those network connections that it actually needs. Then when that part is finished, it calls a system, a special system called a cap enter, and this sort of flags sort of an annotation bit in the kernel on that process to say that this process is now running in capabilities mode. That's how it's called. And what happens is that from that moment on, any sort of dangerous system call starts to return the error code enough capable and to sort of explain what that means. So for example, that network socket that we opened because it was sort of the IP addresses was mentioned in the configuration file, of course, it's safe to call accept on that to accept incoming connections. You know, when you get one of those connections, it's safe to call read and write on them to sort of interact with the system on the other side. That's all safe. It's also safe to open files underneath directories that you've opened. But what's not safe is opening an arbitrary file on disk, saying open slash etc slash password and writing stuff into it, for example. This is really not allowed, of course. Just opening any files is not allowed. And that's how Capsicum works. It turns the system into a system that uses capability-based security. And that basically means that there is no global state that a process has, but the process always sort of has a bag of capabilities, file descriptors in this case, that it can use to access resources on the system. And this is already used by quite a lot of applications on FreeBSD at least. So here you see a list of programs, the DHCP client ping TCP dump. You know, TCP dump is a really good example, if you think about it. The only thing that this program has to do is, when it starts up, it has to open the Berkeley packet filter to capture network traffic. It needs to make sure that it has a file descriptor open to your terminal to actually write messages to it, which packets were received. And when it has done that, it can safely call cap enter. And if an attacker somehow manages to trigger a buffer overflow inside of TCP dump, there is little to no risk at all. And this is actually far more likely than you think, because TCP dump is full of all sorts of packet parsers. It can parse dozens, maybe hundreds of different network protocols and all sort of handwritten parsers to just find all of the specific interesting fields inside of all the packet headers. And there's a fair chance that there might be some buffer overflow in TCP dump. So what you can do as an attacker is you can just sort of send all this random traffic across the network and just wait for a systems administrator to run TCP dump. And when that happens, you can actually just trigger a buffer overflow in a process that's running as the root user on the system, because everyone runs TCP dump as root. So this is a real good model for hardening all those unix utilities, in my opinion. I really like it. So my experience is using Capsicum. Late last year, I started playing around with it. And it really works as advertised. The implementation of 3B is quite robust and the design behind it is all right. But there is something that I noticed that really makes it hard to sort of use this at a larger scale, something that's larger than just ping or TCP dump, because those utilities, they are fairly small if you sort of look at your average unit system. So my observation is the following. Code doesn't really like it if you suddenly start to disable functionality that it depends on. Code, so the best example that I could come up with, I'll first give you the best example that I ran into. There is this crypto library, two crypto libraries even. I won't share the name, but what happens is that they try to open DevView random and what do you do if that fails? Do you give an error message or do you terminate? No, not really. You actually just run getTimeOfDay and then get the user ID or something like that and then just use that as your initial state for the random number generator, because the time is random enough. So that's actually pretty bad. So if you're not using Capsicum, this code is completely safe because opening DevView random always works. But as soon as you call CapEnter, then it no longer works and you go through a different code path that was sort of never exposed before on a regular unit system and you end up with this completely unsafe setup. Now, there's also some other annoying things that I ran into. So for example, time zones, if your program starts up and you call CapEnter and then run local time, it uses UTC as your time zone. The reason for it is it can no longer access EDC local time. But if you call local time, at least once before calling CapEnter, it does use the local time zone. The same holds for localization. Once you call CapEnter, you can no longer do any conversions of pieces of text in different character sets. It's just a real mess. My observation is if you just take a really big Unix application like Apache or Nginx and you put a CapEnter call somewhere in Maine, then the program will just explode in ways that are really hard to cure. It's basically just you hacking on it for months just to get it working. There's no proper guided way of porting a larger application over. So what you see in practice is that smaller Unix applications and free BSD, for example, are being ported over. And sort of in-house maintained code like, for example, the code that runs in your Chrome tabs is being run in SecComp. But it's not as if most Unix applications make use of this. If you just run PS on your system, there's maybe only one or two processes on there that actually uses Capsicum or SecComp BPF. So that's really what I sort of dislike about Capsicum and SecComp. They work for these really sort of artificial or, well, self-developed use cases, but not for sort of the long tail of software. So a second problem that I have with Unix security is that Unix makes it really hard to just run untrusted programs directly on top of them. So if you just download a random ELF executable from the internet and run dot slash run on it or however it's called, then you really mess up your system. Don't do this. I really wouldn't do this. Running it in Jail, FreeBC Jail, running it in Docker is a bit safer. And the Docker people are really convinced that Docker is safe enough to run arbitrary executables. But I guess most security experts really wouldn't dare this either. Running it inside of a VM is safe. Well, also not always. I guess there's likely another talk at this conference that sort of debunks the safety of systems like Xen or KVM, but it's safe enough for most people. And this really makes me wonder why can't Unix just safely run third-party executables directly? Why do I first need to set up a VM, spend an hour installing VMware, et cetera, or virtual box, putting a distro in there and just to run a single executable? Why isn't there like a same simple environment that I can use to run these sandbox executables? So as I mentioned before, the second problem I have with Unix is regarding reusability and testability. And when I first wrote these slides, I had a really hard time coming up with the right way to sort of explain what I think is wrong with reusability and testability. So eventually I came up with a way of sort of coming up with an analogy. And instead of looking at Unix, let's sort of make a step back and take a look at sort of software development in general, namely programming. And let's just take a look at Java programming, for example. If you would write your simple web server in Java, you would probably write something like this. You'd start out with a class, class web server, and it has a constructor, and this constructor sets a couple of internal members. It initializes the class. And what does it do? It creates a TCP socket that runs on port AD and it has some kind of root directory where your files are stored. This works, but most people would agree with me that this is not sort of the tidiest code you would write. If you would just write this for your employer and send it out for code review, then hopefully one of your colleagues will just come over to your desk and slap you in the face. Because this web server is completely not flexible and reusable, of course. I mean, it always listens on port AD. It always uses the same directory. So what you do is you sort of extend the constructor, make it possible to pass in a couple of extra parameters, namely a port number and a root directory. This is already a step in the right direction, but then your sort of veteran Java programmer colleague comes to you and says, like, no, this is still wrong. You need to use interfaces or come up with nice abstractions and layers in your application to make it more easy to test. So most Java programmers would actually write something like this, namely instead of just creating the TCP socket inside of the class, you just make it possible that a sort of an abstract socket type can be passed in. And the same thing holds for the file system access. Instead of putting all of that file system access logic inside of the web server class, you could also just create an interface that, for example, has a function get file content and you pass in a file name or something like that and it just returns also file contents and some kind of buffer. And what you can do now with this class is you can test the entire class without actually constructing a single networking socket through the operating system or accessing a single file on disk. This is something that you can test really easily. Now, if we take a Unix applications, I just showed you like three different ways of solving it. If you now look at Unix applications, you see that they always sort of stick to one of the first two examples that I showed. It's either the case that the behavior of the applications is hard coded or they write stuff to hard coded locations on disk or assume certain locations where they can access services. For example, Unix utilities, they all sort of somehow know that they can open var, run, name, server, caching, daemon, whatever to access like a name server caching. There's no actually way to sort of override this behavior really easily. So, if they don't hard code that kind of behavior, it's typically stored in configuration files that are stored at a hard coded locations as well. And what you see is that applications always require the resources on behalf of you instead of having them passed in. So, your web server configuration, you don't start up your web server and provide it the network sockets that it needs to use. Instead, you write in the configuration files on which IP address and port number it should listen. And this is really bad in my opinion because it doesn't allow you to sort of customize the behavior of the application. Say you want to use a TCP socket that has custom TCP timeout parameters or retransmission logic. If you want to add support for this, you actually need to add it to the web server because when it parses the configuration file, there needs to be an extra configuration attribute in there that allows you to specify the parameters or any options and those then need to be used to create the TCP socket. So, the web server just builds up this huge baggage of all these configuration options that could have been sort of placed outside of the web server. There's no reason why they needed to be in there. So, Unix doesn't use dependency injection and I think that this is a double standard. We're really hammering towards having testable and reusable software, but apparently we don't care about this at the application level. So, here's an example of a reusable testable web server. I won't spend too much time on this, but what happens is instead of just creating the TCP socket yourself, you could just assume that standard in is a TCP socket, for example. And then you can just start up this web server in any way. So, for example, it doesn't matter whether you use an IPv4 or an IPv6 socket, it still works. It can use TCP, SCTP, any kind of streaming protocol out there. It can listen on any address or port number. The fun thing is you can even create the socket once and spawn a whole bunch of web servers all using the same file descriptor. And then you've implemented concurrency without any additional effort. And this web server is, of course, testable because you can just use a Unix socket to inject requests and capture the responses again. So, this is, in my opinion, sort of a better model than what we see in Unix right now. So, now I've spent a fair time introducing or explaining what I think is wrong with Unix, not Cloud API. Now, I'm going to explain what Cloud API is and, you know, as you might have guessed, it's something that sort of attempts to solve this. So, Cloud API is a new Unix-like runtime environment that is, think of it as POSIX, the Unix standard, plus all of the stuff that's provided by Capsicum, minus all of the stuff that conflicts with Capsicum. So, this makes it a lot easier to write software that works well with Capsicum. So, if you would now write OpenDevUranum, instead of just, like, making the program start up and fill at runtime, it now just fills to compile. And it's, of course, annoying if code doesn't compile, but it's a lot easier to fix, you know, compilation bugs than it is to sort of track down these issues caused by the security framework. Because what I explained before of the crypto library that it tried to OpenDevUranum and then filled back to weak entropy, that's something that you really don't notice. The only way you can discover this is if you sort of trace the application and sort of take a look at the system call behavior and see what it does. That's how I, you know, discovered this issue. But if you remove all of these interfaces, then it becomes really obvious where sort of the problem points in your application are. It makes it a lot easier to make software run in such a sandbox environment. So, what's also nice about such an environment is that applications can no longer just create arbitrary TCP sockets or something like that or open arbitrary files on disk. There are no more global namespaces as they're called. And this makes it really hard to hard code all of that kind of stuff in the application. So it's sort of, you're really forcing the application to be written in a testable way. And I think that's good. It shouldn't be like this unconditionally because there's sort of, of course, a very large amount of legacy software existing unique software that needs to work. But it's not a bad idea to also next to that has sort of a clean slate Unix environment where testability and security is sort of a prime focus. And that's also like the point that I want to make at the bottom of the slide. Often when I give the stock at conferences, there are people that sort of walk up to the microphone and say like, this is nice, but it wouldn't work for, you know, traditional Unix use case X. That's of course really not what I'm focusing on. This is really a clean slate approach. So just to sort of rehash what a Cloud ABI program can do by default, if you would just start up the simplest Cloud ABI process possible, it can still allocate memory, it can create pipes, it can create socket pairs, it can create shared memory. In other words, it can do IPC with itself. It can also spawn threads and it can also spawn subprocesses. So it can fork or yeah, creating threads is not necessarily forging, but it can also fork. And it can also interact with some clocks, it can also even get random data from the kernel. Those are all things that are, you know, really not harmful in any way. If you're just sort of a small, tiny program running on some kind of computer, there's really no harm in getting the time of day or getting some random data from the kernel. But what the programs cannot do is open arbitrary paths and disk creates network connections and it can also not just send kill signals to other processes on the system. That's all really fenced off. So the simplest Cloud ABI program, if you would just run it with dot slash whatever, it wouldn't be able to do any harm on your system. So then you want to sort of grant additional rights to this program to actually make it functional and you do that in the form of file descriptors. So you make sure that your program is started up with the right set of file descriptors and that's really critical just to make sure that you use the right set of file descriptors to start up your process. And what you can do is you can just use file descriptors to direct you to access parts of the file system, which I've already mentioned. And this is really nice because in practice this means that most of you people are probably familiar with the change-root system called Unix where you can lock up a process in a single directory. This is sort of change-root on steroids. It allows you to create multiple change-roots. Every file descriptor on its own is its own separate change-root that you can access files underneath. So it's really powerful. Sockets to make a program network accessible. What's also really nice is that Unix supports file descriptor passing. If you have a Unix socket, so it doesn't work for a TCP socket or anything, but just a local Unix socket, you can actually push in a file descriptor on one side of the socket and it sort of pops out on the other side. And this sort of allows for some really complex constructs where a program starts up, it doesn't have a lot of writes by default, but if it really needs sort of a specific write later on, it can sort of send an RPC over to some kind of process like, hey, I need to deliver this email for user X. Could you please give me a file descriptor to that person's mailbox? And that other process then sort of sends a file descriptor back to you and you can sort of write an email into it. So this is really powerful construct. And also something that supports the so-called process descriptors. As I mentioned in the previous slide, you can't just send kill signals to arbitrary processes. So what Capsicum did and what CloudABI also supports is process descriptors. Namely, there's a special for call and if you invoke that call, you actually get a file descriptor to the child process. If you call close on that file descriptor, it automatically kills that child process. So there's also no way for you to sort of leave resources behind. It should be noted, it's important to mention at some previous conference, someone made a smart remark, yes, but what if you can pass process descriptors through unique sockets? Because you could, for example, allow a process descriptor to be passed to the child process itself and then the child process can remain alive indefinitely. So that's not possible. Process descriptors are not possible through unique sockets. That's a restriction that's placed on this model. File descriptors also have permission bit masks and they allow you to sort of really granular deterrent of certain privileges. So you could create a piece of shared memory, a shared memory file descriptor, which is read-writeable for you, but then you duplicate the file descriptor, but on that second instance, you remove the right bits. And that file descriptor, you send it over to another process and then that other process can only read from the shared memory space and not write into it. So this is also an important concept. Without this file descriptor permission bit mask, then the system would be a lot weaker than it is right now. So the secure web service, how would you model it in Cloud ABI? Well, it's pretty straightforward for every sort of bullet point that I had in one of the previous slides. You just replace it by a file descriptor, essentially. So for example, you can use an AFINET or AFINET6 TCP socket for all the incoming HTTP requests. You can use a read-only file descriptor for the directory containing all of the webroot files. So it only has to read capabilities but not write and not truncate, et cetera. So that means that an attacker can never actually modify the files that are stored in your web server root directory. And you can also give this web server an append-only file descriptor that only has the right capability set to it, meaning that the only thing that an attacker can do in the worst case is append garbage to the log file. It cannot truncate the log file, it cannot override its previous entries. It can only add more stuff to it, which is nice. So when I started hacking on this, I observed that Unix becomes really tiny, really small if you remove all of this legacy craft from it to begin with, but also all of the interfaces that are incompatible with the security model. So Cloud ABI only has 58 system calls, and which is pretty tiny. And especially if you compare to Linux, for example, because Linux has 300 and FreeBSD even has almost 400, I think. So it's a lot simpler to implement, and what this means is that you can add support for Cloud ABI to existing operating systems. So I started adding support for Cloud ABI to FreeBSD, and it only required me to write, I think, 6,000 lines of C code. And that is sort of enough support to run all of those Cloud ABI system calls. And now I'm also working on adding support to Linux and NetBSD. The NetBSD port is really robust because it's sort of similar in structure to FreeBSD. Linux requires some more work because sort of being able to run multiple binary interfaces is something that's not sort of a core concept of the Linux kernel, doesn't really support that, so I had to add some hooks to that. But that means that you only patch up the operating systems by adding a couple of thousand lines of code, and then you can run programs without recompiling them, which is pretty sweet in my opinion, especially for sort of cloud computing cases where you're just a hosting provider and want to run binaries that are provided by other people. This is sort of a real killer feature, in my opinion. So now I'm going to explain a couple more things, for example, how can you develop software for Cloud ABI? So first of all, cross-compiling in general is pretty hard. This means that it's typically not that easy to build software for Cloud ABI out of the box. And of course, I've been working on making this a lot smoother and I'll explain how I've been doing that in the next couple of slides. But the problem is that the tool chain, sort of the entire tool chain for building Cloud ABI software depends on a lot of sort of separate components. So there's a compiler, assemble a linker, then you need a standard C library, C++ library, exception-handling library, math library. It's just a whole list and then you only have sort of the bare minimum of building Cloud ABI software. Setting that up is pretty time-consuming, of course. And in addition to that, you also need to patch up any piece of software that you want to use. So removal of all of this capability, I know where APIs really breaks the build in a couple of places. And at the same time, I'm also trying to cut down on sort of some non-unix extensions that are either obsolete or don't make a lot of sense where the C standard has already caught up and provide interface that are a lot nicer. And also really annoying some build infrastructure like Autoconf doesn't even support Cloud ABI to begin with. It does now. But that means if you have any source table from before March 2015, if you run .sache configure on it and want to cross-compile something for Cloud ABI, it will simply say, I don't know this operating system. So that's a bit annoying. So to mitigate this, I've been working on something called the Cloud ABI port collection. And what it is, it's just a collection of build scripts that allow you to build a whole bunch of open source libraries. And libraries include Boost, which is really nice if you're into C++ programming. Curl, if you want to do some HTTP access. Glib, which is part of a lot of sort of GNOME desktop-centric applications. There's even crypto, Libre SSL. And also I've been starting to work on some scripting language support, Lua in this case. I'm working on Python in the meantime, which is going to be a lot more exciting, of course. But what's nice about Cloud ABI ports is that it builds those packages once. I built them on my Linux workstation or my freeBSD server. And then it turns them into a bunch of native packages for different operating systems. So it automatically generates freeBSD packages, Debian packages, et cetera. So what you, as an end user, only need to do is you can just go to my website. I'm going to give you the link at the very end of this talk. And it has some instructions on how you can add a couple of lines to your ETC app sources.list or your freeBSD package configuration. And then you can just use app.get or PKG to fetch those packages. And they're identical across operating system. They're like byte for byte identical. And this means that you have a really consistent development environment like that you can use to develop software. So it doesn't matter if you're compiling an application or Linux or BSD. In theory, unfortunately not in practice, the checksums of the binary should be identical, which is really nice in the heterogeneous development environment. I want to explain why the binaries aren't exactly identical, but if someone wants to grab a beer with me, I can explain that it's horrible. So to clarify, these packages don't contain any native build tools. These packages are all built on my freeBSD or Linux system, and they don't have any cross-compiler that you can run on openBSD. These are just the cross-compiled libraries. And the goal is that the actual native tooling needs to be provided by your own operating system vendor yourself. So if you're a package maintainer for some kind of distro operating system, I'd love to talk to you because I can always use Cloud ABI packages for more operating systems. So here's just a real quick explanation of how you install such a cross-compiler toolchain on freeBSD. I've picked freeBSD here because the steps are a bit easier than on Linux, and I want to make it look pretty, of course. So on freeBSD, you first just run this command package install Cloud ABI toolchain, which gives you a copy of the latest version of Clang and the latest version of Binutils targeting Cloud ABI. And these packages are provided by freeBSD themselves. Once you're done, you already have a compiler, but it can't compile anything because there's not a single Cloud ABI library installed on your system. So what you do is you just add a couple of snippets to and then you can run package update followed by this long package install command. And what you see over here, this is the name of the architecture, x8664 Cloud ABI, and this is the name of the package, namely cxx runtime. So if you install this package, you get a C library, a standard C++ library, enough to do standard C and C++ hacking. And then once that's done, you can already just invoke the cross compiler and compile any software you like, even including Hello World applications. So now that I've explained how you can sort of develop your Cloud ABI software, we're going to look at starting them up. And it's actually sort of more interesting than you think. There's more to it than meets the eye. If we're going to take a look at a very simple Unix application, so LS, for example. Many people have used this application, I won't need to explain what this tool does. Anyway, what we do is, instead of just calling opendir on dot or what LS does by default, it opens the current directory and fetches the directory entries. There's no way we can do this because the application has no working directory anymore. It's simply not there anymore in Cloud ABI. So we call fd opendir, which allows us to open a directory by file descriptor. We then call readdir on it in a loop and then just print all of the file entries that are in there. You also see that there is no standard out in this runtime environment. It's really just, we need to assume that standard out happens to correspond with file descriptor one. So that's what the fd open call us for. So how can we invoke this application in a bit of a sort of a less traditional way? We first compile it, run it through the cross compiler, then we load up a kernel module that's needed on FreeBSD to run this. Nice thing is this is already just integrated into FreeBSD 11 by default. So if you just download the latest development snapshot of FreeBSD, run KLD load Cloud ABI 64, it works out of the box. No packages needed, no patches needed. It just works. Pretty awesome. And then when you have the binary, you can just run dot slash LS, but we need to provide it a file descriptor to a directory on file descriptor zero. We just run dot slash LS, smaller than slash EDC. Which works. It prints all of the entries in the directory. So this is like a Cloud ABI's Hello World application in a certain way, LS. So even though this works, this is not really a natural way of starting processes. It can really get out of hand. Starting up a web server with 20 different file descriptors to different TCP connections, et cetera, it really doesn't work. It's the shell is not meant for that. Even to make it worse, there's not a same portable way in which you can create network sockets. Through the shell, of course. And how do you know the ordering of the file descriptors? If you have a web server that uses 20 file descriptors, how can you be certain that file descriptor 13 was actually the one that corresponded with the network socket? Maybe that was a log file, or it gets out of hand quickly. So please don't use Cloud ABI in this way. It's a mess. Also a problem with this approach is you really lose the existing paradigm where you can specify complete applications through a single configuration file. Right now, you can just go to slash EDC, make changes to a configuration file and restart the servers and it works. If you would just start it from a shell, you would have maybe a separate configuration file with program attributes, but then you'd also need to start up the web server with all the different file descriptors. So it sort of turns it into two different ways of configuring the application, namely your sort of configuration attributes and file descriptors. So I've been thinking about a way to sort of streamline this process to make it a lot easier and safer and saner to start processes. So I came up with a launcher tool called Cloud ABI-run, which is only a couple of lines of, sorry, 100 or 200 lines of C code big, it's fairly tiny. And what it does, it sort of replaces traditional string command line arguments, so arc V, by a tree structure that semantically looks a lot like YAML. And I'm just going to sort of explain in the next couple of slides for what it looks like and then it sort of becomes more obvious how it works. So assume that you sort of have a traditional Unix process, so not a Cloud ABI process that has a YAML configuration file. You could, for example, have a schema that looks a bit like this. So you specify a host name that's returned in error messages or log file entries, number of concurrent connections if it has some kind of thread pool inside of it, an IP address and port number which it needs to listen, a log file, and a root directory. With Cloud ABI-run, what you do is you sort of annotate this file in a special way, instead of just specifying these path names directly, like I did in this slide, or like those IP addresses over there, use a special tag. YAML is apparently a type language. I didn't really notice before I started working on this, but I've just created a couple of custom types, tags, that you can use to annotate which strings are actually path names and which strings are actually IP addresses on which you need to bind. So what happens is that Cloud ABI-run parses this YAML file, it scans over all the entries in there, it sees all those socket and file tags and then just replaces them by the actual file descriptors to those directories and network socket addresses. So eventually you end up with something that looks like this, just like a pre-processing path that gets rid of all of that stuff. And this is actually what's being passed on to the application. So if you're writing a program, instead of using int main, you can now use an alternative prototype called program main, and there is a set of functions provided by an ARC data header, and these allow you to traverse over this data structure, extract file descriptors from it, extract scalar values like integers and strings and booleans, also allows you to iterate over all the maps and dictionaries and all the things that are stored in that YAML file. And this is a really nice compromise, in my opinion, because it doesn't make it harder to configure services than the way it is right now. You still have a single configuration file that allows you to sort of both add configuration attributes but also resource dependencies that you want to list, and it also makes it impossible to sort of get the ordering of the file descriptors wrong, of course. And what's also really nice is that Cloud API Run also ensures that all file descriptors that happen to be open at the time but weren't specified inside of this YAML configuration file are closed, so there's no longer any accidental leakage of file descriptors into processes. For software developers, it means that there's no longer a need to write configuration file parsers. You can just use this ARC data structure to sort of browse through all the configuration parameters. And there's also no longer a need to sort of write code to acquire all those resources and start-up, which is nice. So once you're in Program Main, you can already start working instead of first having tens of thousands of lines of parsing and configuration of file and acquiring all the resources the way you need them. So now quickly going, like in the last remaining minutes that I haven't going to explain some use cases for Cloud API that I've been thinking of. Maybe the audience can think of some other cool use cases and I'd love to hear about that. So at one of my previous employers, I was working on some kind of network security appliance and one of the things we had was a spam filter that was running on this. And this spam filter, we bought it from some kind of third-party vendor. I don't even know the name of that company anymore. But the problem is that this is just a unix process that's just running on your hardware appliance. You know, it's just a third-party binary blob that you're running. And what if this application was implemented as a Cloud API program and was modeled in a really simple way where emails would be sent in on a pipe, for example, and the application would then just return a true or false response whether it was a spam email or not. That means that if there's some kind of buffer overflow in this spam filter, email spam filter system, then there's almost no impact to it. The attacker can just sort of consume some more CPU cycles on your hardware appliance, but that's all it can eventually do or maybe falsely return that the email wasn't spam. But the question is whether that's a really bad thing. It's still better than just, you know, eventually maybe even becoming rude on this hardware appliance. Same old for network appliances. There's a lot of research right now towards doing packet filtering in user space. For example, NetNAP is one of those projects where you can really efficiently get network packets into user space and implement all of the firewall logic in user space and then send a message back to the kernel whether the packet should be accepted or rejected. Something like this could also be run as a Cloud ABI application to just make it a lot more secure. So high-level question management. This is also a really interesting one in my opinion. Cloud ABI applications have the property that the dependencies of them are sort of really known up front. It's not as if they start up and then acquire the resources themselves. You already know what they depend on. So for some kind of cluster management system this would also be really nice. You have a couple of web frontend applications, a database backend, some batch jobs, and all of those have the dependencies that you can sort of express in a graph. And based on that graph, the cluster management system can start them up in the right order and ensure that they're sort of started up close to each other. So not that, for example, the database backend is started up in a data center in Japan while the web frontends are started up in a data center here in Germany. You can really improve the locality if you know all that information up front. It also makes migration of processes a bit easier because you actually know what the process depends on so you know what you need to migrate to the other system as well. And this is really looking really far ahead and the question is whether it's ever going to happen, of course, but still I like to dream about this. There is this service called Amazon EC2 and Amazon EC2 is really nice, in my opinion, that you can just run arbitrary Unix programs in the cloud. It doesn't really matter which program language they're written in. You can just create your own Linux VM and just run a web service written in Rust or whatever you like. Google App Engine, on the other hand, is sort of a more managed system where you just build your own application either Python, Java, or Go. You just throw it over the fence and they start it up for you and automatically scale it up. And already sort of if a system goes down they automatically start it up on a different system. So it's a lot more managed. And I think that Cloud API could be used to sort of combine the two of these where you sort of had a more managed system where you don't actually care about individual Linux or Unix systems anymore but it still allows you to just build programs in any programming language you like. So maybe we'll see something like this in the future up here. Would be pretty awesome. So this is all I have to say for now because we're sort of way out of time, I guess. Here are a couple of links that are interesting. So first of all, there's a link to my sort of consulting company's website but on that site you can also find documentation of how to use this. All of this is open source so here at the bottom are a couple of links to my GitHub page for my company. So the top link is the C library that you need to use for Cloud API and the bottom one is the package collection. So if you browse through the package collection and you see that some kind of package is missing that you like, well then if you're really into that stuff then you know you can always send me pull requests on GitHub. And there's also an IRC channel Cloud API on EFnet so if you'd like to lurk around then that's the place you can go to. And that's all I have to say for now. Thanks for attending my talk. Thank you very much Jack. If you have questions please line up behind the four microphones we have here. Are there questions from IRC by now? No questions from IRC. No questions from IRC. We'll start with the microphone in the front left. Okay, thank you for your talk. One of the problems you mentioned was that a lot of primitives we use for instance in C libraries that you are dependent from a global state of the system for instance at C local time but maybe at cresolve.com or everything like that. Sorry could you repeat that question? Yeah, so how do you solve this problem of being dependent of a global state of the system in at C or for instance at C local time that you mentioned? Yeah, so that's a really good question. So first of all I do realize that systems will always have global state in them and trying to eliminate that is just far too optimistic. But what I think is at the very bottom if you look at the core primitives that are exported by the operating system they should not be bound to any global state. Global state is easier to introduce than to eliminate. And if you have sort of an environment where there is global state for example there is a workstation that has a login session that has a menu bar at the top or something like that it's always easier to introduce it in a programming user space than it is to sort of already let that trickle through through the core APIs of the operating system because that's the reason why we need virtualization it's sort of really trickle into the APIs and now applications really depend on that global state. So I'm not saying we can eliminate it but it shouldn't be part of the core APIs. Did it answer your question? Yeah, so if you develop a cloud API application how do you get the local time? You have to pass manually the Etsy local time in the file descriptors? Yeah, so for example it could be the case that your configuration file for your application has like a time zone colon Europe slash Amsterdam or Europe slash Berlin et cetera and then inside of your application you use that time zone instead of being stuck to a global time zone that's specified in slash EDC. Okay, thank you. The microphone front right please. Yes, over there. Hello. Hi. Can you hear me? Okay. Yes, I can hear you. So you mentioned you started with the clean slate it was definitely partial clean slate Cherry OS is a Cambridge university project they made their own custom risk architecture with a custom memory management unit which can protect a region of memory known as a capability and then they have a custom operating system and a custom programming language which then every time they instantiate an object it's coupled not with just information but also authority capabilities to perform some action on a resource and this actually gives you a multiplicative attack surface area reduction because at every abstraction layer of your program you're reducing attack surface area. Yeah. This is like kind of like halfway or partial way there it's definitely implements some capability object capability security but not the full thing. So it's really awesome that you brought up Cherry so I actually know that the people working on Cherry pretty well I'm very thoroughly involved like I chat with at least a capsicum people once every couple of weeks so I hear all that news coming in and so what the Cherry people are doing is they're building a CPU that has capability registers built in so instead of having operating system level capabilities where you have file descriptors acting as capabilities you can actually say you know I here have a tiny piece of memory a buffer where an application can write into but now I'm removing the right bit from that piece of memory and passing that on to some other thread or coroutine inside of the process if I understand it correctly and I mean it would be really awesome if like Cherry and Cloud ABI had children that would be the most awesome thing ever and it's really cool because maybe you know Brooks Davis who's a free BEC developer as well actually I work on Tahoe Lafs which is a cryptographic capability system okay and it has a different security model that works over networks so you're saying that Cloud ABI should have kids with that yeah yeah potentially okay well we should chat afterwards I just wanted to like mention that my different perspective on yeah yeah I mean just yeah it's really cool that you mentioned it because I mean just look it up Cherry it's also developed by the University of Cambridge really awesome project cool next question from the front left microphone followed by the front right microphone front rear left microphone okay rear left microphone no first first the front left then the rear me or okay oh so the question about time more remark than the question so I don't know about your threat model but actually providing the ability to read the time can be a dangerous because you know clocks are kind of unique in hardware hardware details and it can be used to identify the machine yeah yeah so this is also really good remark so I have to say that you know famous last words even cloud ABI is not perfect so for example the fact that you can access those clocks and the reason that I have for this is the following so POSIX has already a nice standardized interface for dealing with directories as file descriptors so open at the system call that I mentioned in these slides is actually part of POSIX 2008 and with that in mind I can sort of think that there's a reasonable sort of way I can expect people to adopt this the problem with clock APIs is it would be really awesome if there was also really a way where you can say I want to pass in a fake clock to this application this application is not running in 2015 but in the future or the past could even be useful for testing 2038 compliance bugs you know but the problem is there's no sort of standardized API for this so right now to sort of make the adoption go a lot faster and not sort of cause a lot of bike shedding and trolling I decided to just stick to a subset of unix and unfortunately that doesn't provide any way to inject custom random number generators or random clocks or that kind of stuff yeah but it's a really good remark thank you the left rear microphone please okay there you are how does your capability-based security model play with IPC especially when there's no parent-child relation between the processes sorry could you repeat that question okay how does your capability-based security model play with inter-process communication especially when there's no parent-child relation between the processes well I mean so now I'd need to bring up like a list of all the individual requirements or you know Xeomas that sort of specify what a capability-based security system is but there is a way of if my interpretation of capability-based security is correct eventually what it boils down to is that every program on its own has its own bag of tokens that it holds on to its own capabilities in cloud ABI case file descriptors and every process can do a couple of things with those file descriptors namely it can discard those file descriptors at free will you can just close your own file descriptors and you can actually pass them on to different processes and there might be some other different requirements or some other properties in there as well but is this not a capability-based system that's my question what's in your view missing can you use the capability-based security system to secure or to control inter-process communication well I think you're referring to the fact that there's also this thing that it might be possible to sort of steal tokens from other processes where you can say I now want to sort of disallow this process from accessing X and there is no support for that at all so you start up a process and you granted these tokens but there's no way to actually retract them later on that's missing from this environment but if you just make sure that you start your process up with the right set of file descriptors where you don't leak in any things that the process shouldn't have access to then I think you should already be quite safe okay thanks we have a question from IRC we are a signal angel yeah how is this in rights different from Docker which is at its base also only the TH root stereost sorry I didn't understand the question completely you hear me okay yeah how is this rights management different from Docker which is at its base also only TH root and stereo rates so I have to confess there are a lot of different operating systems out there that all use capability based security in a different way I think what makes cloud ABI unique is that first of all it's based on the POSIX APIs for which there's already a whole bunch of existing software available and compared to Capsicum it has the advantage of sort of having less food shooting so yes there might be some other systems that also sort of have a huge overlap and functionality with this but sort of exactly focusing on making POSIX stuff work is actually where sort of the niche market is in this case we have a question from the microphone front left most of the things normally people today would do with namespaces here so username spaces network namespaces or mount rebinds in containers like docker does docker screws up only one thing which are username spaces which are coming soon since two years now and actually the kernel hack to introduce username spaces into the kernel has to touch 150 pieces in the kernel and one of our developers is working on this but it's not yet finished so normally I would say if I have a critical application I would try to do it with a different user and take away the rights of this user and this is very similar to this docker kind of things of containerizing applications then I have all the other things that I do not even have to modify the application which might be hard so if you have you mentioned java give me a java binary which really accepts this file descriptor then everybody would be happy but I've not seen it so far yeah so I think there's sort of like a divide or like a like a division that sort of puts people into two camps right now so first of all there is this focus on switching towards capability based systems which I've explained in this talk but then on the other hand there's also sort of a movement that's moving towards virtualizing namespaces so that includes both freebies the jails but also docker and the entire linux namespacing movement so my concern with with the entire namespacing stuff that's going on in linux and all of the other systems in general is that it actually removes transparency a lot because for example the user id and user credential namespacing how annoying is it that if you run for example ps on one system you see that the process is running as user 1000 and then you just run ps inside of that container or that single namespace itself and it turns out to be one million and three it really removes transparency also the worst thing that you could even come up with is pit namespace virtualization which is a my opinion not even needed if you think about it a bit more because then the problem becomes that like pit 75 in one environment is not pit 75 in another one and it really puts the churn on systems administrators in my opinion and that's the reason why you know this system tries to solve it by actually cutting off access fat instead of adding more shrink wrap around it to sort of make all of the existing stuff work there there's this overly strong mindset that for some reason there is no single way we can ever change any applications in user space and I think that's a true shame I think you know times are changing unix is already what 40 years old 45 years old by now why can't it change that's that's really what I'm sort of trying to expand experiment with here but but it's some really good remark but it's actually changing so you if you um put all these things into system d's and you only need to configure your system d properly and every every pain goes away and then yes but most people don't like system d but yeah exactly yeah but but I mean it's it's a it's a different solution to to solving a similar problem it's either by adding more shrink wrapping around it in the kernel just you know adding more and more virtualization or namespacing virtualization features in the kernel to work around the problem or just saying like we're going to remove stuff and use the space that actually contradicts with this model so it's two different mindsets yeah we have time for one very brief question from the front right microphone what does this mean for for scripting languages in the shell like if I if I whip up a quick shell script for automating something in the file system do I need to fiddle with the capabilities to get it right or do I just run it all powerful and lose the advantages so um the unique shell is something that is really something that's sort of orthogonal to what I'm developing over here and I'd be amazed if uh you ever get to the point where you would just have a shell a real like a born like shell opposites like shell where you can type in stuff that actually interacts with cloud api programs well for scripting languages I think it's actually different I mean I'm working towards having python ported over to this and then you actually have a python interpreter where you can just run all your standard python scripts in there as long as it don't try to open arbitrary files on disk and python actually already has nice interfaces for dealing with directory file descriptors for example in the latest versions of python that is so um it will break a lot of stuff I'm I'm and that's why I'm saying it's going to be a symbiosis and not like an assimilation it's really two separate environments and we'll see where it goes okay thank you thank you very much Ed Skalton