 It's already like three years when I'm working for FreeBSD, so yeah. So our today outline, I will present you a Capsicum itself. I will describe what it is and how it works a little bit. Then I will show you how we capsicumize some of the base tools. And then I will present you a new debugging infrastructure, which was presented last year, which is connected to the Capsicum. Then I will tell you a little bit how the CASPER works and what it is itself. And I will tell you a little bit about the future, so outgoing work in Capsicum world. So Capsicum. So in the dark ages of computer science, our process had something called ambient authority, which means that they had access to every single thing in our computer. That means that can create a connection to whatever they want. They can open any files from our disk. Of course, there are some restriction about the users and so on, but mainly the process can do anything they want. So Capsicum use its implementation of so-called capabilities, which is the totally different thing from ambient authority. That means in capability mode, we say that process has access only to the thing that the process should have access to. So if process works on a file system, then it doesn't need to connect to the network. So he has some access to a file system, but he doesn't have any access to networking. So Capsicum was introduced in FreeBSD9. It was implemented mostly in Cambridge and by Robert Wilson and Jonathan Anderson, with some cooperation from Google and from Ben Lowry. So the current infrastructure of Capsicum is one simple CCTL, which is CAP ENTER, which enters to the capability mode. After calling the Cisco, we don't have access to any global namespaces. So here are all the global namespaces that we have in FreeBSD. So for example, if we enter the capability mode, we don't have access to a file path namespace. That means we cannot any more open any directories files on our disk. Or we don't have access to a protocol address, which means that we cannot create any connection to the network. So another part of Capsicum is so-called capability rights. Capability rights are rights that I call it like local capabilities. It's stored in the descriptor. So we can open, for example, a descriptor to a file system, to a directory, and that means that we have a capability to that directory. We can do anything we want in that directory. So going a little bit further, Capsicum implements so-called capability rights, which means we can even further limit the descriptor itself. We can say, okay, this descriptor is read-only or write-only. That means that if we have a descriptor read-only and somebody tries to write, then Colonel will say, no, you don't have capability to do that. So here are some of Capsicum rights. We have CapRead or CapAppend. It's sometimes the capability rights are, Capsicum rights are dependent on the type of the descriptor. So for example, we have a CapAppcept, which will be used only on the socket, and CapReceive would also be used only on socket. So we can get capabilities in two ways in Capsicum. One is by getting access to some directories or some descriptors before entering the Capsicum. So before, for example, in the main phase, we open some directories, we open some socket connections, and we enter the capability mode. In that way, we have only access to those capabilities, which we ask it before entering the capability mode. Another way is by delegation. So because we can send the descriptor by using Unix domain socket, we can send the descriptor from one process to another one. So if we have, if some process have access to some directory, for example, or some other resource, then we can ask that process to give us the permission to that resource. So we have, for example, some boxed process with Capsicum, which can speak with another process, which privilege one, which has access to some resource. And we can ask that privilege process to give us the access to resources that we want. So for example, we can ask the privilege process for some more files or some more, or some connection to the network. So it's Capsicum hard. Capsicumizing hard. It's not for a new code. When we design everything in separate, with some separation, the implementation of Capsicum in our program is really, really easy. At real system, we use it for almost our process or for all our demos, but we design this, operate our ecosystem to support that. So what about existing one? Is it hard or not? So we, in 2015, we had a Capsicum implemented in few tools. There are few nice tools like TCP-DOM or DH Client, but most of them are SSHD, but we still was lacking from a lot of base system tools. And unfortunately, it took us a long time to get when we are now. In 2016, we have a little bit more programs that we Capsicumized. So there are simple programs like ES, which is also Capsicumized, but we have also a little bit more complicated and which are not even in base system like IRSE client, IRSE client, IRSE, or package, which also is Capsicumized whenever it's needed. Even if we receive some, if we receive some package from the internet and we want to check the checksum of this package with the most current internet, the counting of the hashes is done in Capsicum, in capability mode. So our story starts from two bugs in 2016 of BS patch. Those are, was very easy, those was very simple bugs, which allow us to exploit BS patch. One of them was, one of them was integer overflow, and another one was, sorry, there was an integer overflow. So Alan Jude, seeing those two patches, decided that, hey, how hard would be to sandbox BS patch? So it was not so hard. So first of all, we need to read some code. I hope you see a little bit, very good this code, because we will have a lot of code in this presentation. So step zero of every sandboxing is to read the code. We need to understand how the code works. It's not important if we are using second pledge or Capsicum or whatever, we need to understand what our code does. So BS patch is an application that for patching the source code or some files. So it's work mostly on two files, one which is patch and another file that we are patching. So here we have some random opens, which I mentioned earlier, that we need to open some files in BS patch. And those are interesting things in Capsicum, because we cannot open files in Capability mode. So step one is to reorganize the code. Like I mentioned before, one of the ways to obtain any privileges in the Capability mode is to open everything that we need before entering the Capsicum itself. So we just reorganize the code and we put all the opens before entering the Capability mode. So we just need to call CapEnter after all those opens, and we are already sandboxed the BS patch. We can go even further and we can see that most of the, we can see how the application is working with those descriptions. Here, for example, it's just doing some reads. And here we have some f6. So after reading that, we can go a step further and limit the descriptors to do only that. So we have, for example, we have what I showed you before, we give the descriptors Capability read and seek. If somebody would still exploit the BS patch, then he can't do anything more with these descriptors than read and seek. So here is a patch for the sandboxing the CMP program, which is a very easy tool for comparing two files. And there in Capsicum, we had some duplicated code, which we was doing all the time, the same, the same in every application. So we went, for example, here, we are limiting the SDA out file descriptor. Most of the programs in base system are using SDA out and SDA in descriptor. So this was done in many, many applications. This is a little bit more complicated code because we not only limit the capability writes on the descriptor, but we also limit the IOCTLs that we can do on those descriptors. And another example, which is that in many applications, we need to use, for example, error functions, which brings us some nice error message. And unfortunately for the NLS, native language support, we need to pre-open some files in the system. So unfortunately, this is not allowed in capability mode. We cannot open any files after entering the capability mode. We need to pre-open them before. So we introduced so-called Capsicum helpers. Those are a few inline functions that allow us to limit some of the most popular things. So we have CAPH limit stream, which limits all the files descriptor for SDA out, SDA in, and SDA error. Or we can call the separate functions like SDA out or SDA into limit only one of them. So instead of doing all that work that I showed you in the previous slide, we can call one function which will do that for us. So unfortunately, like a mesh LPC isn't our friend, and error needs from some NLS files for native language support. We have a little bit more issue with LPC. For example, local time need to read time zone for our operating system. It's done only on the first call of local time, and it's cashed the time zone for us. But when we call the first time the local time, it's pre-opened those files. So if we would do local time the first time after entering the capability mode, then unfortunately we would get wrong time. So we need to somehow pre-open those files. So we do that by introducing more Capsicum helpers like Catpages, which is done for caching the NLS files, or the TZ data, which allows us to pre-cache the local time data. So we can now de-duplicate code, and now the code looks a little bit simple. We just limit the SDA out and use another Capsicum helper to cache the NLS pages. Okay. So now I would like to tell you a little bit about the debugging infrastructure, which we have in Capsicum. So from the beginning, KeyTrace implements supports from Capsicum. So we can KeyTrace our program, and we are getting the information about the traits of the program. So KeyTrace allows us to see which syscalls are called, and what was the exit status. So the support for Capsicum in KeyTrace is basically that it tells us which capabilities are missing. Unfortunately, this approach has two problems. One of them is that it's very easy to miss something. We just don't see, we can miss something in Catrace, and we need also to use Catrace on every sandbox program to know what was the trace of the program. So we need to see which syscall was known, and if the syscall ended with error. So it's very hard, it's very easy to miss something. And it's hard also to cover all the patches in the program, because there are some ifs which can be called or can be true in some condition, and not true in another. So we need to test it in many, many ways. So here's an example of Catrace output. We call the CapEnter, and we try to open some file that is not allowed in Capability mode. So Constantine presents a new system, which is called InotCap, which trap InotCap, which if something is done and it's not allowed in Capability mode, the kernel will send the trap signal to the process. So we will create some core dump for analyzing our process. This is a little bit more handy in working system. We can have like a lot of capsicumized programs, and we can just see if something is starting to be broken in our system. We can also enable that by Pro-CTL in our code, if we want only one process to be traced. But this approach has also the same issue like before. It's hard to cover all paths of the program, and we are not sure if we are not failing in some conditions. So here is the core dump from one of the programs that was failing Capability mode. We can see we have a nice trace of the program where it was failing in Capability mode. So we was calling the open, which is not allowed in our code, in Capability mode. So now I will introduce you to the Casper. Casper is a library that provides us the functionality that are not allowed in Capability mode. So for example, if we want to create some sockets or open some files, which is not allowed in the Capsicle, we can ask Casper to do that for us. So if we have some code that is duplicated in many programs, like for example, I want to open some files, and we don't want to fork and implement everything to all the IPC between the processes that are asking about give me another descriptor to the file, or give me another directory, or give me another connection to the Internet, we can do that using Casper. So the APIs of Casper is very similar to Lipsy. We try to not modify the APIs at all. So for example, if we have GetHosMyName function, and we have so equivalent in the Casper, it's exactly the same interface between the Lipsy version of GetHosMyName and the Casper one. So the idea behind Casper is to make things very easy to separate the programs. And we need to create Casper, because Casper is a fork of our program before entering the Capability mode. So how Casper works. So we have some process, and in that process, we call a function called CapInit, which creates the Casper. And now we can ask Casper to do some stuff for us. But before that, Casper is also working the second time, so called Zygote. Zygote is a very lightweight process that will be used to create our new services for our programs. So now when we call CapServiceOpen, for example, to open the DNS service, we will clone the Zygote and create a service from it. And after that, Casper will pass us the connection to the service. And after that, if we only want to open one service, we can close the connection to the Casper, or we can ask Casper to create more services, for example, a file system service or some other service. So after that, we can close the connection to the Casper, and we can talk directly to the service, and we can ask him to provide us some information like, for example, DNS resolving DNS. So right now, we have five Casper services. Like I mentioned, the system DNS services, which allow us to resolve some DNS names. We have GRP service, which allow us to work with the group operations, like searching information about the groups. We have PWD, which is used for password database operations. We have a random service, which allows us to obtain some random data from a service. And we have also CCDL service, which allows us to get information from the operating system. So what is also very interesting about Casper is that all those services can be even further limited. So we can say that, okay, we want to resolve only IPv4 DNS. If somebody will try to resolve IPv6, Casper would return an error that this is not allowed. Or we can even go a little bit further and say, okay, I want to limit Casper to be able only to resolve one domain, like I don't know, Google.com. And if we would try to resolve others names, then Casper would say, okay, that's not allowed in our system. So this is the limitation in the Casper itself. But we can also go a little bit further and limit the Casper itself and say, okay, I'm interested in creating only system DNS and not others services. So here we have a trace route, we have a patch for the trace route, which use Casper for sandboxing. So like I mentioned before, we create Casper by calling cap in it. It's done on the top of the patch. Then we open service by calling cap service open, we open the DNS service. And we are limiting the service to name and address for resolving. And we are limiting Casper for IPv4 on. And after after we do that, we can enter the capability mode. And this is, we also limit here the, so yeah, we also limit the other things in trace route itself. Like we limit the send socket to be able only to send the packages. And we limit the receive socket only to be able to receive. After that, we need also to replace our get-host-by-name functions called to cap get-host-by-name. Unfortunately, right now we also add the if depths that if, for example, trace route isn't in, it's not in the, it is in the base system, but it's maintaining also by others. So we need to add like if depths that we have or not have Casper itself in the base system. So now I would like to tell you a little bit about the future of Capsicum and what its ongoing work in that field. So one of the things we are working right now is Casper mode. Like I showed you before on the patch, we have a lot of if depths in our code. So the idea of Casper mode is that we will hide all the if depths inside the Casper itself. So Casper will you will only compile Casper once and Casper will decide if we have an, if we are using a real function or not. So we have all Casper functions will be mocked for us. So we will not need to do that in the code itself. There is an existing review of this change. Unfortunately, it is like a third implementation of those mocks. We had a few ideas how to do that. One of them was by just mocking it in the library. So we would have LeapCasper version of library which is not is not doing anything. It's just calling the standard LeapC functions. This approach was dropped because we didn't want to link to the library that don't do anything. So the approach that we are trying to do now is to implement mocks using a lot of inlines and defines and we will not need to link to the library itself. We would just need to include the header itself. So we have a lot of ideas around the Casper. So one of them is to integrate the Casper in the LeapC itself and we would just inside the LeapC we would decide if we have Casper or not or how we want to do like get host by name function. If we want to use the secure way using older process or we want to use the standard path. Other approach we are thinking is make a LeapC in 3dsd more pluggable. That means that we could register a different version of LeapC, a different version of get host by name depending on libraries that we are linking to. We would have structure of the functions and we would just in our we could just change the implementation of the get host by name dynamically. So right now we also need to call cap in it with those changes. We also could do that in the start functions. That means that we would not need to call it in the main function itself. We would just have the Casper and we would just use it transparently with in our program. So we are so thinking about the sandboxing services itself. Right now Casper is not sandboxed at all. It is a privilege process and he has an authority to everything. So if somebody would exploit the IPC between the Casper and the processor he would get access to all of our data. So we also want to somehow sandbox the Casper itself and we think this everything which are we doing here with the Casper is also about reducing the TCP trust code base. So we don't need to now trust the whole LeapC code but we need to trust that our IPC library is safe between Casper and processor. So it also some it is also some surface for the attack but it's still smaller than the whole LeapC. So we also thinking about introducing new Casper services like file system which is missing from for a while and it's important for us because it's stopping us from sandboxing a lot of base tools like tools that are used that are working on multiple files like for example grep. We also thinking about implementing for example services called system TLS which would allow us to create a TLS socket and a Casper would provide us all the negotiation between our process and the internet. We also want to implement the socket services which will allow us to create arbitrary socket to the internet raw sockets without any encryption and so we was discussing also to implement like configurations service which will provide us the configuration of the file. So we would have like this is also very interesting from the operating system point of view because we can have some a lot of different format of the configuration and Casper would be responsible for parsing them and send us the configuration in a unified way. So there are two more very interesting services syslog and plug-in and we will look a little bit closer why we would like to implement them in FreeBSD. So unfortunately right now if we would enter if we will we are using the the sys-tel that send the trap in a cap in our system DH client on the start of the system will tell us that something is wrong. So let's see a little bit closer what is wrong with the with the DH client as we can see it's create a column for us it's like I said it's receiving the trap signal from from the kernel. So if we will see closer in the trace we see that you see is trying to do connections connect to that somewhere. So if we would see a little bit closer to the to the code we can see that it's called from the so DH client is trying to do connection and as we can see I didn't mark that but I hope it's visible for you it's trying to call it's the connection which is trying to do is done by syslog itself. So we can see now the code of the DH client the sandboxing of the DH client and it tries to pre-open the syslog before entering the capability mode. So everything should be fine unfortunately and syslog is very interesting and I think it's kind of messy code unfortunately and openlog don't return as any value it means it's always succeeding so we cannot fail when we try to open log we also can fail when we are trying to log something so it's always succeeding unfortunately this isn't true because as you can see DH client started after syslog D so it turns out the DH client don't cannot connect to the syslog at the time and we cannot even print the error because we don't know about that. So we have to approach to this problem we can change the order of the syslog and the DH client and or we can just implement the CASPER services which will provide us the all the functionality of syslog D and I didn't mention that why the this connection was tried to do and why it's failed it's because the syslog tried to reconnect every time when we try to log something to the interoperating system so if we're trying to send some log and it's not connected to syslog D then he tries to reconnect and because we don't have information that it's connected or not it's failed in the capability mode because he's trying to do a connection to syslog D and this is this allowed in capability mode. So another example is syslog D which is also failing on my machine when I enter the inert cap mode so when I try to syslog it I see in the Dmask that something is wrong in my syslog D so after doing all the work with the seeing the the bugger and seeing what is wrong with the trace we see that there is a part of the code which called gtp pvuclass this code is failing in our in our sandbox this function is responsible for for reading the logging.conf login.conf and etc logging.conf files and unfortunately we can't pre-open those files because before authorization we don't know which file to open because it's a home directory of the user that it's trying to log in so we also would be I'm also working on implementing the system log in to provide the functionality for reading those files and what is also very interesting about this part of the code which is broken in the free bsd is that this code exists only in the free bsd it's purchased maintaining only maintenance only by free bsd developers so I would like to thank a few people that contribute a lot to the project from last from last year Alan Jude which done most of the Capsicum eyes especially the pspash Baptiste, Conrad and Ed which also did a lot of work around Capsicum and Constantine which implemented the cctl enotcap so thank you very much are there any questions so at this point how many of the base programs what percentage of the base programs so at the beginning of the presentation I so yes so this is this is the all that was uh um was sandboxed in the 2016 and the list before is with the all the sandboxed applications so those two lists contains all the sandboxed application right now in the free bsd operating system login service yes it is terrible yeah this was my first attempt to read the syslog code and it was very very messy and I'm very surprised that we use that code still but yeah it's it's very messy um so you're saying that you are trying to implement more services yes for Kessler and you said that you know Kessler actually introduced some smaller a civil service attack yes does introducing new services actually increase the service attack or it doesn't so the services will be the same because it's still the same library that we are using for IPC so the service the services will be the same but we need new services to make a capsicum a little bit more usable in many cases for example we cannot implement a grep we cannot capsicumize grep right now because we are missing the whole file system namespace so we could do like a work around like the fork from the grep and try to you know implement ad hoc the service itself but it would make the code I think that this would introduce the bigger surface of attack because it's a specific implementation only for one application yeah my question is just for my interest I think that now Kessler or Kessler a little bit different rather than open yes it's fresh yes so please give me your idea okay so the difference is in the approach of the of the rights that we are accessing so capsicum says that you have a right to some capabilities inside the program and there are managed by file descriptors so you have like the right to open to open the files in that directory or you have a file or you have the descriptors to connect to the internet and so on in my opinion in pledge the approach is a little bit different and you say okay this is the namespace use that namespace and you get the access to whole namespace so in some cases pledge is very useful for example if you would see the the version of pledge at wc it's great it's just like one pledge call entered you have access to file system I don't care anymore right and it's great but if we would go a little bit further and for example if some developer would miss miss give too many rights to the to the program for example I'm giving you the access to my file system namespace and I also giving you the access to network then the application itself isn't so secure because I can read all your data and send it over the internet so pledge is more like about the about the reducing the surface of the attacking the application but making some some sorry I miss the word making some so it it's easier to use but it's a little bit less secure in some cases I also what I'm also don't like about pledge and I hope open bsd people will figure out how to fix that is that if you would fork from in the pledge application and do exec on any on any program then the new program is without sandbox so you can some in some cases theoretically you can escape from the sandbox itself in capsicum you don't have such issue you always are fully secure because you have only limited rights to do things in in capsicum and this is also why the capsicum is a little bit harder to implement it in implement in some applications so yes we would like to do that one of the things is the missing file system for example that is lacking from long time already which should be done soon I hope and we outside that so this stop us for sandboxing all the programs that use the file system so like like WC and so on this make a nightmare to sandbox such applications and besides that yes we would like to see more patches in in capsicum and more sandboxed application and yeah yes so again it's a bigger surface of attack right because in pledge like you mentioned you have access to a whole file system it's retried you can do whatever you want with the file system itself but in capsicum you have two descriptors you have only those two rights to work with and there are very specific ones and you cannot do anything more out in that case you can settle into an object oriented way whereas with pledge it's just in case the program starts up and inside of the main function you already have to decide which rights you're going to grant to the entire application pledges approach we work well for of the EEC base system where it's almost are relatively complex but in my opinion I don't see scale in situations where you have to work with a lot of third party items where it's really hard to judge from the outside how you're going to restrict the application so for example if you're going to make use of something with pretty complex I don't know open open what's the name again open al open ai some kind of open cp as a means how are you going to judge from the outside which rights you need from the application to make the firing up correctly there's something that's pretty hard to catch but it's pretty easy to catch yes unfortunately there is also to be fair with the pledge right it's also hard with a lot of third party libraries because they for example don't work with descriptors and do like some open hidden calls to the inside themselves I remember we had like one issue with sandboxing I don't remember which application but it was the issue that we are the library itself third party library was opening directly the def random and nobody saw that in the code so we was unable to open the random number generator and we was using like we didn't have enough entropy to generate right correct numbers so to be fair both approach has downsized and upsized and there are different approach to the problem in my opinion one is like as said it's like objective approach when you have access to the one object or few objects to the set of objects and another one is to have access to whole namespace in some cases the access to whole namespaces works right but in more complicated applications it will have the exactly the same if we would like to do it in correct way the secure way it will have the same issue for example I saw the implementation of the moot the the mail client in pledge and they in pledge you have access of course to the file system you have access to the to the network and you have also access the fork and exit and this is done because they can't reload the configuration itself so they just fork and exit one more time that the application itself to regenerate the to reload the configuration this is a big attack surface for the attacker because if I can open the client itself why I will open the client itself maybe I will open net cut with some arguments and just create a a backdoor to to that system so yeah it's it's different approach and in some approaches the pledge was great in some I would be more secure feeling I would feel more secure using capsicum I didn't put it on the presentation but we have a very nice wiki page of the free bsd about the capsicum there is a full list of the applications that are already sandboxed and there is a list of the applications that are waiting and there are that we want to sandbox in the future so the most risky I will not give you the list of the application itself because I don't remember but the most risky applications are those that that are working with some kind of parsers right for example grep is great example when we had a lot of issue with greps because it has a very complicated syntax of regs and it's very easy to find somehow that it's it's exploitable so everything that is like parsers and so on should be should be um sandboxed for example tcp dump right we are receiving the the package from untrusted source we are also doing some analysis of this package it's also very unsecure so we also are think I would also recommend to find application that is working with untrusted input like for example the network one right I guess the applications like wc for example which only counts and don't work with real data itself right it's can go in the in the next step basically thank you very much