 Uh, please join me in welcoming, uh, Ryan Petrich to the stage. Uh, he's gonna be talking about CISCALLME by your name, Sandbox and Guazzman Programs. Uh, Ryan is an SVP at a hedge fund, uh, called, uh, Two Sigma Investments. But previously, he probably, he worked at Capsulate, which I think many people in this room, uh, would, uh, would know of. Uh, please join me in welcoming Ryan. Hi, uh, I'm Ryan. Um, before we get started, um, this is not financial advice, uh, nor the views of my employer, Two Sigma. Um, I just have to get that out of the way, um, uh, first. So, welcome to CISCALLME by your name, Sandboxing WebAssembly Programs. Uh, WebAssembly is a bold vision for the future of computing. It's one where you can safely run code anywhere. This means in browsers, in the cloud, on the edge, in your pocket, in embedded devices, uh, somehow on the blockchain. Um, I'm just gonna ignore that last one. Um, but really, you can run code anywhere. Um, and the promise is that WebAssembly Programs can be run at near-native performance, and then you can pack these WebAssembly Sandboxes together really densely, even if they're from adversarial parties, um, and still preserve these safety properties. So you can run, you know, a bunch of different WebAssembly, all in the same program, and it works and, and, you know, is safe. Um, so this vision is only possible, uh, if the separation between the Sandboxes and the host is reliable and trustworthy. Um, so isolation is the key to WebAssembly's success and its future. Um, engineers building on top of WebAssembly need to be confident in its safety, um, or otherwise they'll kind of stick to more convenient and traditional methods like containers. Um, and so this is what I'm gonna focus on today. Um, the isolation properties of WebAssembly runtimes and how we can better trust in them. So with understanding how this works, I think engineers will be better equipped to, to deal with WebAssembly systems, um, and to operate them safely. So we're gonna begin our journey by examining the techniques runtimes use to isolate WebAssembly programs. Um, we'll expand into some isolation bugs in some of the runtimes, um, and explore what attackers might do with them. Um, we'll shift into some techniques operators, um, can layer on top to defend themselves. Um, and then we'll dive into a specific technique, uh, system called sandboxing, um, that I've explored in more detail. So let's begin by examining some of those techniques that WebAssembly uses to enforce isolation. Um, first off, I'm gonna cover some terminology that needs a lot, so let's clear it up. Um, first we have guests, we have runtime, and we have hosts. So what do these things mean? Host is a program that's running natively on the computer, and it wants to expose some limited functionality, maybe for customization or programmability, to an inter-guest program that's written in WebAssembly. So we've got the host and we've got the guest, and it does this via WebAssembly runtime. So this is a system that's responsible for loading the WebAssembly program, uh, and, and running it and giving it access to only the set of interfaces that the host program chooses to expose. Um, and so we can see here on the slide, we've got kind of, you know, a program, and we've got a WebAssembly guest making a host call that calls back in through the WebAssembly runtime to the native component. And so critical to WebAssembly's security model is that the guest only receives access to the set functionality that the host program wants to expose. Any additional access is a tax surface. It's stuff that kind of leaks out. Um, and especially in cases where a single program loads multiple guests from different tenants that the tenants don't trust each other. It's really important that, you know, this boundary be preserved. And so the complicated thing here is that isolation is actually at odds with runtime performance. Uh, runtimes could instrument every step of the guest program, uh, with security checks. Um, but that would end up being really slow because each check has some overhead. Um, and the computer might even spend more time running the checks than executing the useful work of the guest program. So let's take a look at some of the ways that WebAssembly is designed with this, this isolation in mind. So first off is single pass verification. So WebAssembly is designed, um, as a bytecode that you can scan it in single pass and know that it's valid, right? So this means WebAssembly runtimes are parsing and generating code as they're loading it from the disk or the network. Initiate streaming, right? If at any point the verification fails, runtime can just, like, panic, throw out all the work and say, like, this module is bad. Um, and this lets runtimes load and unload code really quickly, right? So you can make for quick startup times. Um, and this allows service providers to discard idle programs with the confidence that they can just reload them again. Next up is structured control flow. So describing all code through structured control flow in the bytecode makes it easy for runtimes to generate safe native equivalents of the WebAssembly code they receive. Um, so avoiding arbitrary control flow, you know, just makes this translation step where they take the WebAssembly and they put it into native code. It makes that much easier. And again, some runtimes can even do it in, like, a streaming fashion. They can get started on it other bytecode formats like java.net have arbitrary code, and it makes the translation steps a little more complicated with more risk of bugs. Um, so all of this making code simple, you know, the code loading simple means, um, you know, there are fewer opportunities for the runtime to get it wrong, and, um, they can do it a little more quickly. Next is fixed stack layouts. So WebAssembly defines stack layouts without the ability to acquire addresses of data on the stack. If you want to acquire the address of something, you need to put it somewhere else. Um, and so this avoids stack buffer overflows and other cases where important parts of the stack are accessed or manipulated, um, in native code. This often allows an attacker to hijack control flow in arbitrary ways, usually by manipulating the return addresses. Um, so all of this is important as an attacker being able to manipulate the control flow means they can escape this isolated sandbox that has been assigned to the WebAssembly program. Uh, so in WebAssembly deployments that have multi-tenancy, that's a huge problem because now you have the tenants, like, being able to see and interact with the others, and that's, you know, that's bad, and actually multi-tenancy means you're worried about malicious code that you're loading, not just malicious data that you're processing. Uh, next up is function pointer tables. Um, so in WebAssembly, it's not actually possible to directly acquire the address of a function. All indirect calls go through these tables, where you say, like, uh, function n is this particular thing, and so WebAssembly figures out, the runtime figures out how to map that under the scenes. Um, so this avoids the guest program, um, being able to mismatch the signatures and thus leak data from the host in ways that might be able to manipulate it. So again, this is about, like, control flow and making sure that an attacker can't hijack how this runtime is going. Um, another thing is, um, isolated linear memories. Um, and so this is the only general memory storage available to guest programs. Um, and WebAssembly programs cannot access, uh, the host memory, right? So you can only read or write to the regions they've been assigned. Um, and of course, the runtime or the host can reach into the guest's memory space to, like, see what it's doing or to read some arguments or data out, uh, but not vice versa. And again, key to isolating tenants from each other. Each tenant gets their own little pool of memory and can't know about the existence of others. And digging in a little deeper on this, um, is, uh, guard pages and lazy filling. To implement these linear memories without expensive bounds check on every memory operation in the program, which there are many, um, they take advantage of memory management units in modern computers, right? So, um, if you ask the MMU to give you 8 gigabytes of address space and, you know, your addresses are only 33 bits, um, that means there's nothing else in there except, you know, reserved empty space for the WebAssembly program. Um, so that means it can't be filled by other parts of the host program or other tenants, um, and it means that any attempt by the WebAssembly guest to access out of bounds will trigger a crash, and the runtime will, will detect that and convert it into an exit. Um, host calls instead of syscalls, so all access to files, network, other resources, um, are done through host calls directly, uh, by interfacing with the kernel. The WebAssembly runtimes will validate all the host calls to make sure that the sandbox is allowed to perform the operation that they've requested, um, and then they'll go and perform the operation on the guest's behalf. Um, so this includes standard APIs, like WASI, we've heard that before, you know, it's, it's a POSIX-like. Um, and then custom APIs, uh, from the host program to the guest, so the, the host can expose custom functionality, but they should be careful not to expose anything, um, that is, that is dangerous. Um, and lastly, the, this is kind of a lack of a feature, but, like, um, WebAssembly started with no multi-threading support, I understand there's an extension, but this is actually really critical because it introduces a whole ton of other security concerns around, um, you know, the guest being able to modify data as the host is reading it, um, and also things like being able to time the operations that the host is doing and drive a side channel there. So both of these break down the isolation barrier and, like, threading is just a can of worms. Um, so I understand that some smart people are working on how to do it safely, and I'm sure they can tell me what the status of that is. Um, and then this seems weird to call this a technique, but honestly, the WebAssembly community is really diligent, and they've read all the research on how to build safe runtimes. The language runtimes are complex beasts, um, and they have similarly complex security concerns, so, you know, like, a lot of it is just being very diligent, and rust and memory safe languages don't really save you when the job is generating new code that might also need to be memory safe. Um, so a lot of work goes into this, and I'm sure there's tons of details of getting it right on all the various architectures that are very fiddly. Um, and actually, the WebAssembly community has done pretty well on this. There have been surprisingly few flaws that break isolation, so good work. Probably some of you in the room have spent a lot of time on this. Um, but it's really difficult to get right, and, um, so when I went searching for implementation bugs, I did find some, um, and, um, there weren't too many of them, but that's great, but any single bug that violates this isolation model could be kind of catastrophic. So they're really important. Um, the first place I looked was in Wasmetime. It's the most popular, I think, dedicated runtime, and has a rigorous security program with fuzzing and proper reporting of security bugs. Um, I found two CVEs. Um, one let guest programs, you know, access past the end of the guard. We have some incorrect handling of bit shifting. This is hard to get right. Um, this is only applied to x86. Um, and so, you know, that's not great. And then the other one, coincidentally, also was around this concept of linear memory, and, um, it allowed reading sort of before the zero address via some, uh, funky restoring of values from the native stack. So both of these could allow guest programs to read or write outside of their assigned memory region, either past the guard or before zero. Um, and that could lead to a compromise of the host program, you know, or exfiltration of some data. So that's not great, when you want to keep your WebAssembly programs isolated. Um, Wasmr is another one. They tend not to issue CVEs that I could tell. I'm not sure. Maybe I just couldn't find them. Um, but I did read a paper that described two bugs in their Wasi implementations. One allowed deleting files outside of the sandbox. The other allowed, uh, reed-only, so that violates some isolation property. Um, there's, uh, Wasmr, which is a micro-run time, which is designed to be run on tiny embedded systems, and they have a different design. Um, I think their policy is to report CVEs, like Wasmr time, but I actually couldn't find any. I did find a bug on their tracker that, um, showed they had forgotten to check the argument stack didn't get too large, which is a common thing on embedded devices where you have limited memory, like things can move into each other, and that's bad. Um, um, and then of course there's V8. V8 is used in Chrome, Node, Edge, and seemingly every new browser these days. Um, it had some call stack and heap corruption vulnerabilities way back in 2017. Um, and of course there's more bugs in the JavaScript portion. Um, I think the fact that WebAssembly has had very few sort of validates the design. Um, you know, but uh, so these were, and it's important to notice, these were all fixed, they're in the past, and they shouldn't be a knock against any of them. Um, I'm just trying to show that like software has bugs, and some of those are, are gonna be security bugs. Um, it is important and I did come across UVWazzy um, it claimed not to be providing a security sandbox, so I find that a bit alarming. I think, I think perhaps it's just early in the project's life cycle. Um, so what do attackers do with these bugs? Um, it's important to know what they do and what their goals are. Um, and that way we can secure our systems, and this is especially critical on multi-tenant systems. Uh, so one obvious thing is you can read and write data outside of your sandbox. Um, this includes memory from other tenants. This kind of has limited use. I mean it is, it is a danger and you might get lucky, but it's not real, like most attackers are really after arbitrary code execution, right? If you can run code in that host, the process, right? You have full access to like whatever is there, including the other tenants, and that's the, that's the real danger. Um, and so they can do anything the host program would do and this is like being able to, you know, like the attacker can kick up their feet like they own the place. Um, and I'm not sure, but I think this might be possible from some of these bugs uh, previously, it often goes into the devils and the details. Um, so let's take a look at what a traditional exploit path is. Um, I guess a non-wasm program as a point of comparison. So we've got some form of memory corruption, uh, corrupt the heap or the stack and be able to hijack control flow and you reuse little bits and pieces of the existing executable image to add in some new code, um, and then jump to that code, and then usually the code, um, will go and execute a shell. A wasm, it's a bit different. Um, so wasm makes, uh, protecting the guest a little bit tough. They have structure control flow, all the things we talked about previously. Um, but wasm, we have this densely packed web assembly programs and so it's not just a tax on the guest, it's a tax by the guest on the runtime. Um, so cloud providers want to be absolutely sure that a rogue tenant can attack the peers. Um, and so malicious guest code should still keep the host safe. And so here, it's very similar actually. You have memory corruption in the wasm runtime, um, you know, by the, the, the guest code that's loaded. Um, you have Rop payload. Probably you can reuse some of the code that, that like the web assembly runtime generated by loading your code or you can use the other code in it and then again map the shell code, launch into a shell, kick up your feed, enjoy. So how do we raise the, the cost of attack here, right? How do we mitigate these flaws? Um, and web assembly is pretty good, but it's not perfect. So I like to layer things, right? So the first and most obvious thing is process isolation. I'm glad, um, another speaker talked about that. That's fantastic. Um, not every bug leads to code execution. This is really effective at isolating tenants from each other. You just put all the tenants, it's web assembly stuff and it's easy to process and keep tenants in different processes and that's super easy. Um, and it's actually key to a lot of the other strategies we have. Next is C groups. Um, I think Docker is the most obvious and popular way. You can isolate file system network a whole ton of stuff and you can layer this again on top of web assembly run times. And we have reduced privilege and file system permissions. These ones are I guess kind of obvious, but they're easy to overlook. They're easy to accidentally give access to something that, you know, the host runtime doesn't need. And so reducing this surface area is important. Virtual machines is another tactic. Kind of defeats the purpose of web assembly, but I had to put it up here just so people would say I didn't, you know, forget it. Um, and then lastly, and this is perhaps where I'm going to focus on, is, um, system call sandboxing. So you can put it in a sandbox which limits what operations the program can ask the kernel to perform on its behalf. So this is like the next layer underneath the web assembly run time. Um, so the way this works is there's the sub-system called seccomp. You can ask the kernel, hey, only allow me to perform syscalls with these patterns. Uh, and the kernel will validate every call you make. Um, so this style of sandboxing is tricky to get right. It's really cumbersome. You have to know all the syscalls your program has before you, you know, like ask the kernel to limit. And if you miss any, you know, that's kind of troublesome. The kernel will reject you and your program will crash. So that's a problem when you have multi-tenancy. That's a problem in general. Um, but web assembly run times, as complex as they are, they have a relatively fixed set of behaviors, really. They load and jit compile the web assembly guest programs. They instantiate instances of those programs, including some linear memories, um, and then they execute, uh, those programs. So those programs will do a bunch of stuff with their memory and then eventually they'll do a host call. So, once you figure out what the runtime does, um, and what limited sets of behavior in the host calls you have, um, you can set up a syscall profile. Um, Chrome does this for render processes in attempt to mitigate unknown bugs in v8 and the DOM and the web assembly and just all the things that go into the magic of rendering a web page. Um, it does take a ton of effort to build precise syscall profiles. Um, even when you control the software, like everything uses libraries, um, and it's, it's really hard to, to kind of like know what everything under the stack of your software is doing. Uh, but we can write software to generate and apply the profiles instead, uh, using a branch of computing science known as program analysis. So, um, with program analysis, you can yield precise sandbox profiles that tightly limit what your web assembly runtime can perform. And I built software to do exactly this. Um, it's not specific to web assembly. Um, it's called colander. Um, but I'm going to take it here and apply it to a web assembly runtime. So, it does static analysis on the scale of whole programs, libraries, complex control flow, loop-de-loops, switch statements, everything. Um, it generates and applies a seccomp profile that applies to the program. Um, and you do it by prefecting your program with colander like you might do with strace. So, um, I've applied it to wasm times, so it's like colander-wasmtime, right? Uh, to add this extra layer of isolation. Um, so it generates precisely scope profiles without manual effort. Um, and it does this, um, in a really sort of precise way by knowing exactly what operations and syscalls the program performs. Um, so this means we can actually sandbox just like entire web assembly runtimes, um, without actually doing all that hard work to investigate all the minutia of the operations they perform. And so it models the imagined possible states of the program at sandboxing, in this case wasm time, and as it goes, so it doesn't have to, you know, engineers don't have to do hard work. And it does all this to get a complete picture of what system calls are possible even in those obscure error paths that, you know, happen in seldomly. Uh, it's important to note that colander can be more precise than just the system call numbers. It knows what range of values are possible for each argument. And this is much more detailed than a human could be expected to write. And you'll see in a moment why I wanted to apply it to web assembly runtimes. And that's because many security system, uh, sensitive system calls, like those that can add new executable code, um, kind of only happen, like, at startup, and program initialization, um, and when web assembly modules are loaded, and then once the program gets going, it should, like, never do that again. And that is something that attackers generally do during their their attack. So, um, colander applies the program when you ask it to, so it can do it just after the program finishes starting, and it can block any syscalls that are only used to initialize. So here is wasm times, and the text is tiny, so I apologize for that. Um, these are wasm times memory mapping syscalls that it, it figured out. And you'll notice only one of them is prodexec. So prodexec is asking for an executable memory mapping. Um, so it doesn't make memory mappings after it's finished, uh, compiling web assembly code. So colanders can just wait until all that startup compilation finishes, and then tell the kernel, like, allow it to continue to mmap and, and protect, but never with the prodexec option. And that means we can block shellcode just entirely. Like, no shellcode, um, attacker can compromise the guest, and as soon as they try, they will get blocked. Uh, so colander doesn't really have any specific knowledge of wasm time. Um, I only had to tell it, like, here's the point at which the program undergoes a phase change. Um, and it should play equally well in future versions and to other web assembly run times. Um, it is available on github, and it runs on most x86 Linux distributions. Um, I'm looking at other architectures, but right now it is architecture specific. Um, and so with this, um, I hope all of you learned a little bit about sort of, like, WebAssembly's isolation. I'm a huge fan of of just sort of, like, where it's going, especially the component model stuff with nested run times and being able to interop between languages freely is just fascinating and really exciting. Um, and I want to get more people into it. Um, I'm excited to get the, like, isolation properties just as bulletproof as possible, and I think it's going to be magical when we can just, like, have fine-grained isolation, like, between the different parts of our software, without having to do microservices or anything like that. Um, so I'm actually really eager to see what additional steps the community is taking to secure their WebAssembly run times and software, especially if you run, like, a multi-tenant platform that absolutely fascinates me. Um, so thanks for your time and attention. Um, I'm Ryan Petrich, basically at our Petrich Everywhere, and if you like weird, hackery things, give me a shout. Incredible talk. Uh, many questions for Ryan. Over here. Hi, thanks for your presentation. I checked the, uh, DTW report of a calendar, uh, but I couldn't, you know, find the source code and the license, uh, so could you, uh, tell me where actually I find the source code and what's the license of this software? I am trying to figure out the licensing right now. I'm committed to making it open source, but, uh, that'll be wrapped up pretty, pretty soon. Uh, so, so it's not open source at this moment. Right now I only have the binary available, yes. Oh, yeah. Uh, but I think this conference is mostly committed to open source software, so it should be nice if, uh, if you open source software before the conference next time. Yeah, I'm, I'm, the plan is to have it be GPLv3. I just ran out of time for the purposes of this conference. Yeah. Yeah. Thank you. Other questions? All right, Ryan, thank you so much. Thank you.