 Right, so it's me again, who'd have thunk it? Okay, this talk is about security issues arising in OpenJDK. I've called it securing a moving target. That's the aspect of OpenJDK I think is interesting to talk about in terms of security. The subtitle is, what could possibly go wrong? Actually, I'd read that another way. What are the things that might possibly cause problems that are specific to the nature of OpenJDK, the class libraries, the JVM, that wouldn't happen in many other types of applications? That's really the thing I'm interested in. So, I thought when I had talked about security, the motivation for this was actually a B-Sides conference that a friend was involved in a new castle, so I gave a brief version of this talk, but I thought it was really interesting to talk about. It's really motivated by things that happened before I got into the vulnerability group, and that's only compounded since with the experiences of working with all the other people who were involved in that group. Up until we had the vulnerability group, all I'd ever seen were exploits, sorry, were patches. I had no idea what the exploits were, and there were these weird changes that obviously fixed something, but what the significance and importance of it was was really opaque. And then our security expert, Martin Blauer, who's just an amazing expert in security, both in penetrative testing and also in defensive security, came along, and with his help, when we were trying to look at some backpots, so we backpotted to eight and we were wondering maybe should they go into seven? We thought, well, we'd better try and create an exploit to use this patch and see if maybe that exploit would work in seven. And we recreated an exploit. He knew lots about how you could hack your way into things, and I just knew a bit about the JVM, so it was mostly his work. And it was absolutely mind-boggling to see how a little glitch in something in the JDK code could give you an opportunity to open up a big hole and then all sorts of havoc could happen. So that was just really interesting. And maybe think what a weird mindset you have to have to be looking at a detail of the way something works and then to think about a way you might work your way around that and achieve some completely unrelated goal that breaks things open and then it's game over. So it's quite interesting to think about security, and I think it's good for people to understand about security when they're developing code. Of course, there's lots of reasons why you should not talk about security. I mean, there's no benefit in obscurity in not having it visible, but one of the things that the vulnerability group and this has gone historically before that, what we don't do is ever publish exploits. And in fact, when we test a patch and we create a reproducer to make sure a patch that's going into a security release is actually working, we have a before and after reproducer, and we don't actually publish those either. So we're very careful about making fuss about the things that are going in that are security patches. But you can go and look at the patches and you could do what Martin and I did, which is work back and think, how could I do something with this? And I'm not going to go into specific exploits. I'm not even going to talk about the way you could design them. That also will all your fun. You can go and do that. And I suggest you do. And if you read up about security holes and the way they work, there'll be lots of things you can find to go and do it. What I really want to talk about is how OpenJDK gives some unusual opportunities for security holes to arise that are really germane to OpenJDK to the class libraries and the JVM. So what's the problem with security? What's the security breach? Well, there's lots of different types of things. Here's a few lists of things that can go wrong. You can have a desired denial of service. Something happens in the application. If you feed it the right data, it goes into a loop that's either infinite or potentially infinite because it's going to run for 64 bits worth of computation or something. You can drive a program into a program crash in some circumstances, and that's the security hole. There's more interesting sort of malicious activity, which is where it's possible for data to be observed. You can have something that's actually a direct exposure of data because of just buggy code handing something out that shouldn't be seen. But there's also indirect exposure of data with what's called a COVID channel. You feed some input into a program, make it do certain things, and then next time you invoke some other operation, you get a result out that's conditioned by what you've done before. So certain actions can reveal other information about another action of the program does, and you can drive a program to reveal information that wasn't really meant to be intended. That's what a COVID channel is. It's where some actions can side-effect other ones, and you can understand information about what's going on in the program or data. You can actually trigger operations in the same sort of way. You can invoke some behavior by pushing a program in a given direction and end up executing something that wasn't meant to happen because of the bug in the code. But you can also have a COVID channel where you set the conditions up by feeding the program, making it run a certain way, and then another operation actually will go down a different path to the one that's expected. So that's another way where a COVID channel can actually execute things you don't expect. The amazing thing that I saw talking to Martin was this thing called return-oriented programming. You can actually execute unforeseen operations in a program without even having to jit in there. So one of the things that happens when you're doing a graph row of flow, you used to inject code and return into it and start executing stuff, and then the security, that whole was plugged by making stacks non-executable. But one thing you can do is inject stuff on the stack which has return addresses in it that return somewhere to the end of a bit of method code, and that method code can execute an instruction, and then it returns back to where you just jumped from. And then you can do another return to another bit of code with the right address left below that on the stack, and then the load of return addresses on the stack that you've overflowed, and you end up executing instructions for like half a dozen different methods, and you put together a recipe for doing something that gives you access to something that then opens up something else. You can make a jump to somewhere. You can drive the program down a path that wasn't expected. So even without a jit in there, you can actually assemble a sequence of instructions that do what you want. Of course, that's harder to do with jit in code, but not all the code in the JVN, compile C code in there. Now, I'm not interested really in the obvious game over cases where you've got access to the command line or to the class path, or you can load an agent. They're just, you know, not very interesting. The interesting attacks are things that rely on escalation. This is where these little funny weird behaviors in the program can actually end up giving you access to a tiny little hole, and then most of the things escalate that by getting the hooks in, getting a chance to execute something or do something, and then you can actually download a security exploit kit that base you hammer in, and you open up the whole program. So that's the sort of scenarios that are interesting, and where are those sort of things possible to happen in OpenJDK? Now, I'm not there for going to look at things like specter meltdown. These things can happen in any program, buffer overflows. JIT in code makes it a bit hard to exploit things because the code isn't always the same and it's not always in the same place, but the VM doesn't, the VM compile code, you can use all the usual tricks for that. I'm really interested in the things that are specific to the design implementation and what sort of attack surface OpenJDK presents that's unusual that's different from other programs. So what have we got to secure? What is OpenJDK? Well, it's a language implementation, so you start with something that's Turing complete, it's got a file network OK, it's got a presence API, so yeah, you can do anything with it. So there is potentially a big security problem there, this thing is this general purpose programming language, at least it's got static typing. But actually it's a managed runtime, it exposes and manages all those functions and although there are things that you would normally have an operating system do it, it sort of does for you, it presents them in a very controlled way, thread management, memory management, and so on. So it's also a layered managed runtime and the application can do certain things but only through the, some of those things are only available through JDK APIs, the core language and the core libraries and that's all in terms of Java code and there are only certain things that Java code can do, you can't handle raw pointers or whatever. That Java code gets compiled to equivalent machine code and as long as it is equivalent, well then that also has the same safety guarantees and it bottoms out at a layer where you go into native code into the JVM library methods and implement JVM APIs and so on. So there's a nice clean boundary between the stuff that's in Java and that Java can do and also this other code that's written in C in the JVM that has access to things like metadata, the JIT, the GC, the interpreter, all this stuff that's doing the more hairy stuff with a nice clean divide between them and they're really living separate worlds with a fairly clear interface between them. And that's not true in other JVMs, the Java and Java virtual machine like Jix or Maxine or Substrate, the code that's in there that implements the VM stuff has been pre-compiled into an image from Java code but it could just be overwritten and replaced because anything's up for grabs in being recompiled at runtime in a Java and Java machine. There isn't this division where the C code can never be replaced and it just does that job and the Java jittered code only has certain access paths into it. So that's the sort of layering between application JDK runtime and JVM and there's a separation of powers that goes with that. There's a trust sort of model. The app code is the least trusted and in particular the JDK core runtime modules are the most trusted. The module system has really made a much more graded leveling of trust and access to behaviors that are more dangerous and more risky. So the module system is very important in firming up this sort of layered security model and separation of powers. In the very core classes, the stuff that allows modules to communicate via a sort of shared sequence model but that's very constrained so that there's only certain accesses. There's a bit of a sort of spanner in the works there that a JVMTI Java agent can get in there and do things and has a few superpowers that can break this up but that's obviously you can tie down agents so they're only accessible in certain ways. And then the most trusted is the JVM code. It's got the ability to access the operating system. There's actually some native code outside of libJVM that uses system calls but there's also all the stuff most of it is actually encapsulated in libJVM, the stuff that's doing the more hairy things. And then there's the actual runtime stubs that the jitted code calls so there's also other ways into this underlying behaviors. So it's not a completely clear defined but it's done in several places and some of it is via jitted codes so it's not immediately evident where all the access to this capability is. And then there's JVMTI native agents that also another spanner in the works at that level where it's extensible and things can go wrong but you have to control that. So what are the exploit opportunities that this gives that are sort of germane to this setup? So I've picked four things where things could go wrong and I'll just talk about them. The type system, speculative compilation, methanals and the old favorite serialization, the security nightmare of choice. So type safety is really powerful and the structuring of type modules, packages and type access models is a good way of organizing things but the type safety has to be done by verification. Some of it offline, Java C compiles programs into a byte code which is quite tightly constrained and that means you can only actually get certain types of things into the JVM that are limited in what they're able to express and that already starts you off from a base where you can then check things. There's a quick online verification of byte code at class load and then that's followed up by further type checking at link resolve time checking across from one method to a call method and also at compile time a method tree is type checked and there's type consistency. These all rely on each other in a hierarchy though and there's options that things checked at one point don't need to be checked again later on. And this is what I called a moving target. This is actually difficult because the unit of program delivery is class by class. So the JVM is reading in byte code and building up its own internal model of the class base and the type security across that is being patched as you go along. So there's information you have about which types are in the runtime and what sort of accesses are valid and that's always subject to revision because classes can be overridden. So assumptions you make about what's type safe at one point you may actually have to revise or extend as you load more classes into the code base. So it's definitely a moving target there. It's also complicated by the class loader model. Classes can be loaded from the file system the network from different places. It's possible to generate classes actually internally using code generators and class loaders can do that. It's also possible for class loaders to delegate to each other. One class loader loads from another class loader and that can be switched around at runtime. It doesn't have to be a static delegation network. So there's all sorts of room for code to come flying at the JVM from loads of different places. It's got to make sure it has a consistent view of the type model while all this is going on. Could an application benefit from that? Could you somewhere introduce a disparity between the checks that happen at verification time and then a check that happens when a called method is checked against an actually loaded class and the signatures are checked and so on? Could that happen between link resolve time and then the types that are maintained at compile time? You might try and fiddle this by, say, breaking class to class linkage by dropping a class loader and then trying to get classes loaded from somewhere else and see if you could sneak a different class in. Now, the JVM and the JDK should never let this happen and when types are loaded, there's a lot of effort to go on to to make sure that the classes that are used and verified against each other at verification time at bytecode verification time are the same suite of classes that are used when the interpreter is doing link resolution and the same classes that are used at compile time. So that should never happen. But that's an area where this moving class base and this ability for classes to come and go is something that needs a lot of work to make that happen. Speculative compilation, there's an interesting feature. This is the point where the JVM compiles code on the basis of the current knowledge of the type system and how that's influencing execution. So for example, you can also have things which not just do with types like hot branches and cold branches, the JVM will just cut out that code from a compile and compile code that assumes you're only going to take a certain path. You have to make sure that if that changes, you detect it and you go down the right path or you recompile. You don't just carry on using that code and end up having a case that's the wrong case. The same thing if you do use type profiling, you can assume that a call which has a foo argument is always going to get a foo bar. So you throw away the case handling for class foo and you always assume a foo bar. If you deoptimize that and don't recompile the code and don't patch the code and make sure you get a new version, you could end up processing one type as though it's another type and that's game over because then you can spoof types and do all sorts of terrible things. And similarly with invoke virtual versus invoke direct, there's a mechanism there to allow you to make a direct call when there's only one implementation of a virtual method call. That should be bullet proof. It shouldn't be possible for you ever to go and invoke a method on a class of one type when it's really another type. There's something that's done in the JVM is actually setting up an opportunity to go wrong and there has to be a lot of careful analysis to make sure that it actually doesn't go wrong. And we don't think it does, but obviously if you're looking for somewhere where you want to pry your screwdriver in and up and up a hole, that's an area that's interesting. It comes from the nature of OpenJDK. Method handles were the most recent feature to introduce more bugs than anything else really in recent times. It's not surprising because if you think about what was being done to the language, it was a mechanism to allow indirect invocation of a behavior. So before this, if you wanted to invoke a method, it was an invoke in certain bytecode that named the method and the type that it was appropriate to and the invoke was either virtual or direct or a special for a constructor or whatever. But it was actually a known quantity. Here it's possible to invoke a method that's been handed from somewhere else. The method handle is being constructed and the reference to the method is indirect. Now one of the differences about that is that access verification happens at method handle create time. It doesn't happen at the point where the method handle is invoked. So there's a different model for how you secure access to methods with method handles. So it's quite easy to think to hand out a handle to some code that shouldn't actually be getting it. There's a very obvious way for that to go wrong. But there's also some of the details of how method handle works. They're able to be called either with an exact signature in which case you have to have exact matching types and argument types or an inexact signature where certain operations can happen during the call that are performed by the runtime to massage arguments, make sure they're the right type and so on and then eventually pass them through to the target method handle. And you can do things like argument shuffles and so on. This was all implemented in the JVM initially and it was a very, very big perturbation to the JVM because it was such a lot of code was needed and it was later re-implemented using byte code that was generated from JDK instances method handle classes in the JDK and that made it much more secure and it introduced a lot of bugs and it certainly meant that ever since then really big changes to the JVM have been absolutely avoided with trying to do everything as very conservative as possible approach to changing JVM functionality because the lesson from this was that a really major structural change to the JVM was going to have problems for a long, long time. And then of course the serialization saved the best to last. It's the exploit trap that keeps on giving. It's a very old feature. It's always been responsible for bugs there have been loads of exploits. The fundamental problem is that when you serialize an object to persistent storage and you deserialize it or you deserialize it or you deserialize and deserialize through memory you're basically creating an object from scratch from a saved version of it and you're installing all the fields bypassing the normal construction process and you're taking whatever you get out of the deserialization stream so there's lots of opportunities to spoof things there and to give you data and to get things snuck into an object in a way where it's not validated and it's even worse because it's possible to provide custom serializers and deserializers so there's always room for new bugs by people building their own serialization and deserialization frameworks. Andrew will chase one of these for weeks and weeks and weeks a fast serialization library it was too fast. So this one's... I guess I don't know if this is ever good can we deprecate serialization? I'm gonna go. Yeah, yeah, yeah. So there's just four different parts of the way OpenJDK works where you can see there's all sorts of complexity it's really specific to this type of managed runtime and if you can go away and find a nice little security hole there and get a reproducer, send it to us and we'll try and fix it. Don't sell it to some bad guy, please. Right, do we have time for questions? Hey, great, okay. Is anybody going to be first? Hey, Remy. I think in your list you have forgotten the way the security works by walking the stack. Yes. Which is a very, very weird way to do things. Yes, when you're actually verifying something trying to create a method handle it involves looking up the stack to see who's trying to do it. So that's... I suppose it works. So yeah, that's an important detail, thank you. Hey, Ian. Intel have been proposing things like SCADs that separate control and data stacks and CET to kind of verify that returns are going to where they came from. And of course these are ABI changes so they'll need support in the JDK. Is this anything you're looking at at all? I just found out of this from Martin I thought it would be a really neat thing to talk about. I have no idea what people are trying to do to fix that problem. So it sounds interesting and I'd be interested to look at it. I think Martin would probably feel that better than me. He's really good. Yeah, I'm sure he'd love that stuff. It would be most interesting to see what you had to do to the JITs to make that stuff work. Yeah? Also heading to the list, now there is a certain... a number of projects out there that are trying to exploit... trying to leverage from persisting compile code in various ways. There's one from IBM, there's one that Azul is doing and then there's also... in the upstream there's also a native image. So wherever you save a native image and it's being loaded later on there's always... at this point it passes the verifier and you just run native code. So I guess the premises there is that whoever provides this native image file network stream needs to ensure its authenticity but maybe it's often forgotten. Yeah, that's a really good point. So this is true for class data sharing as well that something could... that could have errors in it, it could be tampered with. There's room for things to go wrong there and the grail native images there's nothing you can possibly check. It's a binary that you just run but you build binaries and trust them and run them in other languages as well. I guess there's the problem... the question of is the nature of how that binary is constructed or shared libraries constructed such that you have other trust issues that you wouldn't have with say a compiled C program. I'm not sure what the answer to that is but it's an interesting question. I think you do and this would be a good point for me to point out that back in the days of GCJ we made a tree of cryptographic checksums that were used to match up the pre-compiled program with the bytecode files. Well that would work for class data sharing where you've got separate method code with grail, it's a great big munged optimised image. It's heavily optimised. So that's not an approach that you could really use. I suppose you could check some of the thing to make sure something hadn't tampered with it when you reloaded it but knowing the integrity of all the parts is not really a it's a non-starter. Yeah, but that's true also of a compiled optimised C program. It's not unusual in that regard. Yeah, you have to trust the producer. Yeah, you have to trust the computer. Yeah, you know, these computers. I don't trust it. I'm a Luddite. Anyone else or are we done? Thank you. All right, thank you.