 So, I have the honor to present Dr. Adil Pablo, who has recently joined, or recently, some time ago joined the Java team and helped us with an upload of two of Jetty, and also helped us here with DevConf, make sure we all have a good time here. And you're doing a talk on the system-level JDM, so here you go. So, this is a talk where I try to convince you guys about something, and you try to convince me not to do it and do something more productive with my time. But the first thing is, this is a battle cry, and I got a battle cry, and I know you guys are very sleepy, so the battle cry should wake you up. If this comes through the audio, the sound is... This here has to be on, but I'm going to get it to this. That's from Argentina, it's called a sapukai. It's a type of thing that people scream while they are dancing. So that's not our battle cry. I guess battle cries will be even more scary. So I'm going to talk... Most of the talk is going to be what it entails to build one of these multi-applications JDMs, but I will first start explaining what they are, and most importantly, why we should care. Yes? I have a really, really great point about it, but it comes in like glambar 10. I could have just started with it, but it will be a little bit momentum, so bear with me for this first slide that will be a little boring. So what are multi-applications JDMs? So the idea is to have a JDM that supports what is called isolates, that are virtual machines that allow multiple applications or processes, tasks, depending how you want to call them, basically things that have a public static void main and different class paths to be running on the same virtual machine, and of course, these different applications should not interfere with each other, and that means you will run them through and you will obtain the same result as if you were running them in separate processes. This, you think about it a little bit, I came across the concept of isolates when I was following a project, I don't know, I don't follow it anymore, I don't know the current state, call it JNode, that we're trying to build a whole Java operating system. The interesting thing about isolates is that they are a standard, I mean, they are JSR 121, the application isolation API, you can go to the JCP website and download it. It's a very, very small JSR, what you get is this, you can say I want a new isolate that is going to be running that application, and then you can actually start the isolate, and that will, these few lines of code are equivalent to doing a new process, launching an external process with the JVM. And you can also say this is the input stream, this is the output stream, and you can, things call it links that let you communicate with these things. And this has been around for quite a while, I mean, this is from 2006, so by 2006, this technology was to the level to make it into a JSR. And here's the most interesting part. I, until, very recently I was working in a company that is very fond of large monolithic eclipse-based applications. So right now, many people find themselves running a whole chat client written in Java, plus an email client written in Java, plus an office suite written in Java, and not really Java, but itself, but all written in top of Eclipse. So you are sitting there in your laptop, and you are running three full Eclipse applications, and, you know, on top of that, you want to do something with your machine, but all your RAM is gone, yeah. And you can imagine that as Debian Java becomes successful, people will have more and more applications they wanted to have running on top of their systems. You can be, you know, hosting a few Debian DVD torrents using Azurius, well, now I'll call it buzzy. You can be having your desktop being indexed with Lucene-based desktop search. And you can be doing a voice conversation using CIP communicator. There is even a CIP base that is written in Java. And all of these things will have their own JVMs. And I don't know if you have seen, but in Microsoft Windows XP, when you come to login and you have been running applications, there is this small balloon that says, running multiple applications will slow down your system. Yeah, like you shouldn't really run multiple applications. So, you know, if you're running these six different things, like chat and email and whatnot, you should expect that your system is going to be more slow. But the problem here is, well, with just in time compilation, with dynamic compilation, your code is being compiled multiple times for each of these things. That's okay. I mean, it's waste of time recompiling the code over and over, but it's just a startup. It's something, in a sense, you can live with. The problem is that all these multiple copies of the bytecodes are living in RAM in separate places. And bytecode is very, very small. But the machine code that we saw from that is six to eight times bigger, yeah. And in a sense, this is the same thing as having a different copy of J粒C loaded in RAM for every process you have. Yeah. I mean, Java is that bad. You are ending up with a duplication of, wow, that's true. This really sucks. How can we be doing this? And this is what we are putting our users through. One of the interesting things is, you can say, this is off topic, you shouldn't be giving this talk in Debian. Go and give it in open JDK, Sumit, or whatever. But we are the people who care about this. We are the distribution. We are the ones who put all these programs for people to use. Each of the different applications, the person who write the email client, the chat client, they really don't care so much about this. We are the ones who are giving the user the experience. So building the multi-application virtual machine is really more on the realm of open JDK. But the one pushing that, I think, we are in a very good position to try to do that. So the interesting thing is, this is a solved problem. Yes, I mean, since the 99 on people on Sun Research Labs and other research centers have been working on that, they figured it out. These are some very good papers. All these papers are from UPSLA. So they're published in good places. And the project called Barcelona is even considered a finished project. They build systems that they ship on cell phones that they were modifying their Sun JDK to support the isolates. So this is doable. We just need manpower and decision to carry through. So the rest of this talk, I'm going just to go through three different solutions for this problem. And they are a little bit detailed and technical with respect to how JDKs work. But the whole point is this is not something that cannot be done. I'm personally not an expert on JBMs at any point in time. And I read those papers, and I understood. And maybe that's because I'm not an expert. I believe that can be done. But so the simpler solution, I'm going to go through three approaches. The two of them don't require JBM modifications. And the last one is the only one that actually really works. But the simpler approach is Java has this mechanism called class loaders. The same way that you have reflection, and you can find methods and stuff for different classes. In Java, you can let the virtual machine call your code and tell how to resolve a particular class into an instance or a sequence of bytecodes. So with that, then you can get these type of things. Imagine you can implement that isolate call by loading these things and start replacing classes you already have in memory. So you can start sharing the compile code. Of course, this does absolutely no separation between applications. Yes, everybody accessing system set out will set the output stream for everybody else on the same JBM. So this seems a little laughable. This stuff is not really allowing you to run multiple applications in the same JBM because it's a strong interference. But when you think about it a little bit more, this is really very, very widespread use because applications like Tomcat, in reality, are doing that. They just use this. I mean, the JBM has a strict semantics that you can have all these applications running together and they will not interfere with each other if you use the Java security system. You put a security application manager that stops any use of anything that you know wouldn't work on this type of scenario. I mean, really, there is no real reason why you can have so much restrictions on securities in a Tomcat stuff if it's not work because you want to keep this isolation working. The next, it's okay, so the problem here really is that all these static variables, you can access them from the different classes and you're gonna have trouble. So one of the first ideas that came out is this idea of saying what we need to do is to get the classes that has static fields, we just split them and we have different static fields classes holding them and we can do that on the fly just by modifying by codes, things like aspect J, that type of things. So if you have a class like this that has three static fields, then you, on the fly, you produce something like this where you have these other classes that contains the actual fields. So every time when you were accessing the field before that it was just an access, now you have to go and look into these array counter dollar S fields that is indexed at the total maximum of applications, so you know your application number two, so you're going to get the fields for the application number two and so on. The problem of doing this is that you need to synchronize these accesses because for the moment where you access the fields for the first time, you need to call the initializer. Yeah, so you need to synchronize here and that completely destroys your performance. What's more, this destroys your performance on static methods that people normally use when they want to have something done very quickly. So that paper is such a southern paper. It goes into a lot of, I mean, that's some benchmark analysis and find out that for example, this MPEG audio that is all static methods because it wants to make it very efficient, then it's completely killed. But then they really wanted to make it work so that they don't really need to synchronize here if you can use this double check idiom and you know that your JVM will support it well, and there are ways that if you know your architecture is going to do the right thing, if you check that it's not null and then set it, yeah, then those numbers come back to the regular size. Still, you have overhead, yeah? I mean, it's a little counterintuitive that to have this NBM stuff you will get overhead and that's the little strange part of reading these papers is that they just keep talking about bad news. They don't really spend so much time benchmarking what do you gain from having this NBM stuff, but I would imagine that sooner or later we'll have our users just come in and knock into our doors to have these if we succeed on having a number of desktop applications that are useful. Okay, so you still need a custom implementation of key classes on the system library to support this. I mean, the approach number one and approach zero don't require any modifications to the JVM, but you still need a custom system library. So you get, for example, a system out different for every application. And this part that is particularly tricky for D.V. And is you have different system libraries for different architectures because if you know that this architecture you can check for the value and then set it without getting to race conditions, then you can do that and it becomes, you don't pay these huge penalties. Okay, so the real thing and it's just doing the same thing that it was being done with this bytecode instead of position, but just doing it inside the JVM. And it's really the same thing for each class. You have in the place where before you had just the constant, now you have an array and the array has this different pointer. So basically you are just taking this same thing of having the fields into an array and each field is a different object for a different application. Now it's just that when you access the static field, the JVM internally replace the direct access with a reference on the array. And this is the cleanest way. You don't have to change any source code or any of the code, but you need to change the, you need to access to the internal of the JVM. And in the paper they make a lot of fuss about that this is only worth it if you have access to a very good implementation of a JVM that if you use like some of the free JVMs you wouldn't really be able to experience the total awesomeness of their work. And well, good thing is now open JDK, it's open and we can try these things ourselves. Okay, one of the most interesting things about this paper is this. Besides dealing with the issue of the static fields, they also introduce something that is quite separate from the actual issue of MVMs, but there is no real way to do Java native interface inside one virtual machine and still remain all this lack of interference semantics because your Java native interface segfolds and it will kill your 10 applications at once. And then you get this Windows experience type of thing. So they, in the same paper they introduce Java native interface isolation where they run the Java native interface in a separate process and they have the Java native interface object do copying. So what you get here is the Java native interface object in C is actually a proxy that then does IPC to the actual multiple virtual machine. I was talking with Matias about this on Saturday and he really liked this part. The other part wasn't so much. So hold on to this because maybe at least this is something we can start looking into and start trying to have it. Any questions, comments? Okay, so there are still quite a bit of issues. Initially I was planning to have each of these to be a whole slide, but aren't out of energy. One of the things that they claim is very nice about this is that you can re-haul the whole heap allocation strategy and then have a larger heap for the whole MVM and then when you launch an application with a given heap, you can get extra heap allocated if there are no other applications running. And they also go for the generational garbage collection approach and use the objects that you are sure you want to keep, they just bundle together across different applications because they are going to stay in, you're not gonna garbage collect them anyway. No, the last segment of the multi-generational. There is other bit that is very, very, very tricky and I'm going to try my best to convey it. If it doesn't come across very well, just make faces and I'll try to repeat it. In just in time compilation, every time you access a class, you have to check whether the class has been loaded and initialized but if once it has been loaded and initialized, you don't need to check for that anymore. So the compiler compiles the code with these little pieces of code that check whether the class has been initialized and then it's like self-destructing code. It's like okay, it's not initialized, initialized and then delete myself. And we cannot do that in a multi-application virtual machine because to preserve the semantic, you will still need to initialize in a different application for the same code. So you are still doing that check all the time and that also incurs a little penalty and also means that the just in time code you get is a different just in time code. You still need a few system classes that need to be modified. The system class is always a good case that need to be modified and for reasons unknown to me, they say that they don't support custom class loaders and that for me is a big drawback because OGI bundles are all built on top of custom class loaders therefore you will not gain anything out of Eclipse. Everything will be repeated even with running on the same JVM but I don't quite understand why they did that. I do believe you should be able to do it too. Okay, so now I have put together a few slides about if we were to have something like this within Debian, yeah. So in a sense, this is trying to say, well as Debian Java should we try to push OpenJDK to go in this direction? I'm interested in working on this myself outside of Debian in a sense joining OpenJDK and trying to get this implemented. Maybe other people are interested to making the mistake too. But if we were to have this technology in Debian, it opens a lot of problems too and this is just thinking ahead. What would it mean? Well, one of the first thing is user being Java if you have set it with update alternatives to point to the MVM. Now it's a strange thing. I mean it's now deploying things into a demon that is running there. So the way to think about it is screen versus bash, yeah. And you have extra arguments about the different demon you are launching things against. The other thing is that maybe you want to have an MVM that gets boot up on, that gets started up on boot and is used as a system level. But that brings, again, when it's running under which user and it's really necessary. That's similar to Tomcat. I mean Tomcat you have a system Tomcat and that's running as a Tomcat user. More hairy issue is the bug. Yes, you have this application, it runs fine. Then you put in an MVM and some bug happens. And here you are getting interference between maintainers, yes. And so it's not just, okay, let's add an MVM to Debian and we are done. It's really, well if we add it then we'll have to boot up with these issues, yes. And these are going to be very difficult to find and to track and most maintainers available that's not really my problem because, but in many cases it may be just that this interaction is showing some bug on the code or it's just a bug on the MVM implementation. This is similar to the issue of just having multiple JVMs within Debian. And I understand that as an issue of manpower we're just moving away from having multiple and just focusing on open JDK. The other part is I still don't quite get how are we handling the Java native interface libraries. Yes, I mean if you are only supporting open JDK then you just put the Java native interface libraries there but in this case the MVM as a fork of open JDK should take the same Java native interface libraries as the same native libraries as the regular JDK but I don't know if the location will be different. I mean you guys in the question before I say that you put them in user-leave-jvm-geni, JNI the... We put them in user-leave-geni, yes. Okay, so they are supposed to be shared between different JDKs. Yes. Okay, so that will be better. The part that I found very, very scary is you end up having Java packages that are no longer the same for different architectures. If you go with things like the approach one where you have a different set of buy codes for different architectures because you know that in that architecture you can do things in a certain way and there will be no race conditions. I don't think we... But I mean you could build a Debian package for a particular architecture that still just use Java. It will be really odd but... It's not that far from how we do JNI today. So, okay, one of the best arguments that Matthias had was, you're not talking about modifying one JVM. You're talking about modifying the zoo of JVMs we have for all the architectures that Debian maintains. So, well, you know, landing on open JDK and modifying one of the... I mean the hotspot to be an MVM for I-386 might be doable. Doing the same thing for all the other JVMs and tracking all those bugs and stuff like that is a much larger enterprise. And another very interesting question is, well, but if you want native code and DLLs and all that stuff, well, we do have GCJ. So, an alternative view is just, let's forget about this MVMs and just focus on contributing to GCJ. I'm bringing it up to speed. Many thoughts on that. Okay, so that's the larger ideas. Now, if we just want some of this stuff, the technology, maybe we can start thinking small and say, okay, maybe we just want a cache for the just-in-time compilation results. I don't quite see a global cache for just-in-time compilation results going to really address the memory duplication issue. If you want to address the memory duplication issue, you end up having an almost MVM solution. The only thing you are going to address there is the duplication of the compilation time. Yeah, so the first time you get one of the class, you write the just-in-time compilation results into this cache and then you just get it from there. But still, because of these initialization barriers, you only write there the code with the barrier and you have to load it in RAM and then replace it with the barrier removed after it has been through. The part that sounds very promising is the isolates for the native code. And that's fairly independent from the MVM aspect. They still, the paper still has a gray area that I couldn't really get is in the fact of when you ask a pointer to the actual RAM that, for example, an array has. But in that case, you wouldn't be able to do it in this case. But supposedly, the semantics of the Java native interface object allows you to copy this memory back and forth and that should comply with the Java native interface specification. Would anybody be interested in this type of feature? You know, be able to run a native interface code in a way that if it's segfault, it doesn't kill your Java application, might? So I think we might actually already have had something similar to that. I remember when I used Caffe with some JNI, the, and I had a segfault in the JNI and Caffe helpfully threw a Java null pointer exception and continue running, which is mildly scary, but I assume they're using a similar technique to this and with having a separate process. So I think this might actually have already been done by some people. Okay, so you said that Caffe had that already in there most likely. Well, back to the topic of jet caching, it seems to me that it might be possible to avoid the memory re-duplication by a compiling position independent code and then a memory mapping it. At which point, I would expect the kernel to share the pages. Perhaps there's something I'm missing though and it's just off hand. But you have the static fields, those ones. I mean, those would have to be I guess a separate section or something, but I think if you could, you know, I select the code itself, make that position independent, might be possible to send along those lines. Sure. And the MVM idea is very interesting as well, just a thought. That's a great idea. Do you know much about that stuff? Yeah, I'm not too up on the details. It just seems that in principle, something like that might be feasible. Just wanted to add that it seems to me that the memory data problem is bigger than the memory code problem. And second of all, when you talk about sharing the code, you're also making a optimization for space trade-off because the code can be compiled in this way that's specialized for your application. So just two things, one is a pointer and one's kind of a question. And the pointer is there's a very good Google IOTalk about optimizing Java C for Android where they talk about things like just actually storing the class names is a huge amount of wasted space. So when your memory can strain, you don't wanna have Java Lang String 150,000 times possibly in a piece of code. And the other is just a question is one of the advantages that I find in using Java is that when I get a heap dump, that is the heap dump and having the system level JVM, it's almost like asking the user, well, just dump the kernel memory and I'll take a look because there could be a lot of things in there now. No, but the heaps are isolated in their design. The heaps are separated. So you have your three applications, each one has a two gigabyte heap. Okay, but when things go south, it seems like you still may want the Uber heap, right? I mean, that's what really we're talking about. I mean, the JVM does have, it has a single address space and now we have these multiple processes within them. Yeah, I hear you, yeah. So a couple of years ago, it was interesting, Peter Von De Ahe, who is a Java language developer, took the statics out of Java C so that we could have multi-threaded Java C. So it's only a small part of the problem, but so it was an interesting example of trying to find those statics, which was kind of tricky to get them and get them out. I believe, I was noticing on the Open JDK site that there is a sub-project called MJVM, I think, and I wanna say that John Rose is the guy that's been blogging about that and I think he even talked about it at the recent JVM language summit. So I don't know what the status of that is. Pablo, have you had a chance to look at that? Nope, I was hoping this will get me the type of pointers. Well, this is a subject that's near and dear to my heart because I used to do performance at Sun and obviously I've been aware of the Barcelona project for a long time. So you asked for some constructive feedback so I wanted to challenge you just a little bit, Pablo, but people have brought up these points a couple of times. One of them is, well, one of the ones that I just don't know technically enough about is we have this assumption that you're showing in the picture that if application A and B have the same dependent jar, they're only loading that once. And I'm not, I assume that that's the case with an MVM but I don't know how that would work if it's actually in different class loaders or the same class loader and how the security works with that. That's the reason why they didn't address class loaders on Barcelona to begin with. They will just take the regular system classes. Okay, and then the other thing that was pointed out is that the heap space for application space is necessarily has to be separate. So you might save the perm gen and you might save some other smaller chunks of memory but probably the bulk of what your application is doing is gonna be in the heap and that will be the same whether you have MVM or individual JVM instances. Another thing that will be the same is the file system cache for all the system libraries that are loaded. That'll be the same whether you have MVM or single JVMs. There are some downsides to MVM. One is that you don't have the opportunity to do different JVM tunings for each application. You have one for the overall JVM instance. Hopefully that wouldn't be a big deal but it is a constraint. And then as you pointed out Pablo, there is the risk with, and I didn't, I don't know how far the isolates go but there is a risk that if the JVM crashes that it will impact all of the applications that are running. And as you pointed out, there is some probably some penalty to pay for doing the isolation and teasing out the statics in JDK and in the application code. That being said, the whole promise of this is to get a better startup. And so it probably would be really awesome for startup. And I think that I guess what I've learned from doing performance work is it'd be, there's nothing like measuring it if we can find a good use case and then just measure it and see, especially if there's something experimental with OpenJDK even before we try to package it just to sort of see how big a win it might be. So this is more or less it. And I, with some luck, next day I've come, I'll come and say, I was a really bad idea. But I'll take a look at the MJVM project. I'm hoping it's still going on. So do you contribute with OpenJDK? How is the process for contributors? Actually, I've been busy doing other things the past couple of years and I haven't been an active contributor to OpenJDK. I'm wondering if I can ask other people in the room to comment if they've contributed to OpenJDK or tried to if they have any comments about that. Who's actively maintaining the OpenJDK package? Oh, that's the elephant in the room, I think. Oh, sorry. They're both in the room, as I recall. Yeah, they are wearing an elephant suit. But so you don't communicate with upstream match for that. So most communication is with IST. Well, you can see that as a proxy which does collect patches and provides built infrastructure to build OpenJDK. There are only a few people, two or three, inside IST who contribute to OpenJDK or to oil upstream development in OpenJDK. So I would feel that some things like that being done within Debian would very fast bit rot and would be unmaintainable. No, the idea of doing this is really within OpenJDK. It's just trying to push it from Debian, saying, well, we want this and we will take it and blah, blah, blah. It's more on that direction while starting from Debian rather than at the... So the best thing then would be to maybe to integrate it first with IST. That's something like, what's done with the zero VM and which is now being upstreamed to OpenJDK. Does everyone use it on HPPA? No, zero is used on Spark, no, sorry, on S390, on PowerPC, on SH4, on MIPS and... Okay, go, then you... One of the things that occurred to me while you were talking earlier about where you were comparing this to Tomcat, where you have a single Tomcat instance running as the user Tomcat with all of your web apps in it, is that when you're trying to move this into the desktop application space, there are a number of other issues. Your Tomcat, typically your Tomcat thing is not trying to write files as particular users and it's not trying to connect to an X server. So the issue you're going to have here is that the user that you're system level JVM running at won't have the permissions to access your files and even running it as root won't help if, for example, your home directory's on an FS and it won't be able to access your X server. So I think for things like desktop applications, the best you're going to manage is having a single JVM either for user or more likely for each session. I think that's still going to give you quite a lot of win if what you're worried about is 10 or 15 desktop applications. But I think running something where the first time you run a Java application in your session, you get a VM and then subsequent times you just load it into the existing one is going to be a much better approach to try and take. And if you have a look at Debuss, for example, they already have all of the stuff on there for managing sessions and this sort of stuff. So a lot of that's already there for you. But I don't think that you're going to make a system level JVM fly, but it's multiple apps in a single session. Maybe that is possible. What about Mono, the common language infrastructure stuff? I mean, they are putting a lot of desktop applications built on the CLI these days. I assume Mono has a set for process for each, but I've not looked. I said what? A set for process like we have with Java at the moment. I assume. Oh, okay. Just to add her to all comment, they burn tons of memory too. Unless I'm mistaken. Thank you.