 We have a big, big review of your previous talk on the Khabar-Khabar side, and I, what are you talking about? What? Your previous talk on JVM and Shalif, we separated it by minutes and done a review like for an hour for a talk. Can you hear me? OK. All right. OK, people, listen up, quieten down, sit down, all of these things. Right. So I'm Ron Presslow. I work at Oracle and I'm the technical lead for Project Loom, which is about adding fibers and continuations to Java and JVM, and that's what I'm going to talk about today. Everything I'm going to say, forward-looking statements. So for those of you who are not familiar with Project Loom, let me first review why we need it, why we believe we need it. So the main problem domain is concurrent applications, normally servers that serve a lot of concurrent requests. And right now, developers are faced with a binary choice, either writes very simple, straightforward code, that's blocking code like ordinary servlets, that pretty much assigns a kernel thread for the duration of some request or some operation. But the problem with that is that modern machines and modern operating systems can only support a much smaller number of kernel threads than the number of open GCP connections. They have so that becomes a very limiting factor. Or other developers go the async route and use callbacks or completable futures or RxJava or any of the many asynchronous libraries for the JDK, for Java and other languages. And that's very scalable, but it's harder to write. More importantly, it's harder to debug. You lose context. If you get an exception, you don't see exactly the context of the computation you are. It's very hard to profile for the same reason. And one of the biggest issues is that it's virtually non-interoperable with legacy code or other code that you must choose either synchronous or asynchronous and it's very hard to mix the two. So we want to help people enjoy the best of both worlds. So the simplest way to do it is just to increase the number of threads that they can run. But that means that we no longer rely on the operating system for threads. We create lightweight threads in the runtime. We call them fibers, but sometimes I've just referred to them as lightweight threads. And this way you can have as many threads as you want. And the reason it helps is because you can now represent the domain unit of concurrency like a session or request or user with a software unit of concurrency. So that makes the code much simpler. So from the programmer's perspective, the programming model is just like ordinary threads. It looks blocking. It looks synchronous, but to the operating system it appears as if you were writing asynchronous code. If you do an IO operation, to you it seems like blocking, but under the covers it's going to use the asynchronous IO of the operating system. And it's not just helpful because you can create as many threads as you like, but also we're now trying to make it even easier by helping you weave all these fibers together into some coherent whole and make it easier to organize concurrency, as you'll see. So before we get to these fibers, we start with a lower level building block that's now built in a prototype into hotspot. And that's called continuation. So continuations are a low level construct as application developers are not expected to use it, but it's used by fibers and maybe other constructs we'll add later on. But if you were to use them directly, this is what they would look like. So they're basically, they're like a runnable. So they don't have any intrinsic notion of running things in parallel. They're a runnable, they run in line, but a runnable runs when you start it to completion. So until it returns, normally you'll have frozen exception. With continuations you have another option. A continuation can say yield. And what that means is that it's going to return to the caller. But next time you run it, instead of starting from the beginning, it remembers its state and continues from where it left off. So in this example, we have this loop that occasionally yields and we create this continuation. And now we loop and we can ask whether it actually completed or just yielded. And every time you run it, it's going to continue from the last yield point. This scope parameter there just tells it, helps us nest continuations inside one another and to tell which one you want to go back to. The very important thing here is that, even though this appears in the same method and the same lambda expression here, this can be deep in the stack. So you can call other methods and other methods and somewhere down the line that would call yield. So to do that, so this is currently the API and the more or less, the API and the prototype. So we've seen all these. These are the main operations. Because you can yield deep in the stack and you need to remember the state you were so that you can continue it later on, a continuation needs to capture a portion of the stack. So basically a continuation has its own stack and you can imagine that when you continue it, it's as if the stack of the continuation gets appended to the current thread stack and once you yield, it gets moved to the side again. So we also have some methods to walk the continuation stack. So far the example I've showed is a kind of a collaborative yielding that the continuation has to request to yield itself. But something very experimental we just added is that from the outside, you can basically make the continuation yield, that's going to happen at the next safe point. So this helps if you want to do a preemption. Finally, we have some technical limitations that do not allow us to yield at any point in time. There are some states that we don't allow yielding in. One example is that if you run inside the continuation and you have some Java code that then calls out to native code through JNI which then calls back into Java and then you try to yield, that would mean that there would be a native frame on the continuation stack when you yield and we do not allow yielding at that time. We say that the continuation is pinned and by default the continuation, so when that happens this method is going to get called and by default it's going to throw an exception. That's not what happens with fibers because fibers overwrite that. Some missing features, so continuation is a very low level construct. You can build some more interesting stuff on top of them like continuation that can pass values from the run method to the yield point. You can say run with some value that would be returned from the yield when the continuation is continued. We may want to allow them to be cloned. A continuation remembers its own state and you continue the computation but once you continue it, it mutates. If you clone it, you can remember where you were and go back in time and try the same computation from the same point multiple times. Another thing we want to do is serialize continuation. You can do some computation, stop it, send it maybe to another machine and continue it there. A sort of annoying technical limitation we have right now is that if inside the continuation you're currently holding on to a native monitor with a synchronized method or a synchronized block, you're also pinned. That is something that you cannot currently yield while you're holding the monitor. We have continuations and that's the only thing that the VM provides. We build fibers entirely in the JDK libraries on top of them by combining continuations with Java schedulers. Right now we're using a fork join pool and a fiber or even if you think about any kind of thread is really something that has, so each fiber has a continuation, has a scheduler. You need the ability to pause the computation and you need some ability to assign the computation to different CPU cores when it's runnable. If it's waiting for something, it's going to continue it. To show how that's done, this is a gross simplification of the code in the current prototype. He said that every fiber has a continuation and a scheduler and if you're familiar with the JaviTill concurrent constructs, for example, it has locks. Every time you grab a lock, if you try to grab a lock and it's not available, eventually what's going to happen is that this method, this is today, the JDK that's out there, it's going to call a class called lock support and it's going to get to a method called park. In the prototype, first we check to see whether we're running in an ordinary thread or a fiber. If we're running an ordinary thread, the same thing as happens today will happen. We call unsafe.park and that asks the operating system to block the thread. If it's a fiber, what we do is we just yield its continuation. For fibers, if you try to block, what happens is really that their continuation is yielded. If you're now holding onto the lock and you want to release it, what you do then is you see who's waiting for it and you wake the next guy up and you do that by calling onpark. The same thing happens here if it is a kernel thread, it's going to ask the operating system to wake it up. If it's a fiber, it's just going to ask for the fiber scheduler and just submit the fibers continuation to the scheduler and that means that it's going to get put in some scheduler queue and when the scheduler says, okay, now it's time to run, it's going to call the run method, but then it's going to continue the continuation where it left off and it's going to behave just like threads do with blocking. Fibers right now, the API and the prototype is deliberately minimal. I'll explain why later. This is how you can start a fiber. Currently fibers can also return values, so you can wait for them to terminate and that would return the result. All the classes in Javitil concurrent in the prototype have been adapted to work with fibers. All of the I.O. in the JDK has been adapted with very few exceptions for files, I think. So they're all now fiber blocking. So if you try to do any sort of I.O. on the JDK in the prototype and you do it from a fiber, you're not going to block any kernel threads even though you're still using the old blocking APIs. And fibers also can work nicely with code that uses futures. OK, so we want to have many fibers. That's the purpose. So how are we doing comparison to threads? So there are two things to consider, memory footprint and task switching performance. So ordinary Java threads have about two kilobytes of metadata, and most of the time by default they have one megabyte of stack. In contrast, fibers have only currently 200 to 300 bytes of metadata, and the stack is basically, it grows and shrinks, it can be a few bytes, it can be a few kilobytes, it can be a few megabytes. In terms of task switching costs, so we all learned that blocking is expensive and it is. For threads, it can be one microsecond to 10, sometimes even more. For fibers that we don't know yet, because what we're working on right now is performance, but it's going to be significantly less than one microsecond. OK, so you think, what are fibers? Fibers are just threads. They're not implemented by the kernel. They're implemented in the JDK, so the simplest thing to do was to just reuse the thread API, maybe add a new constructor, and instead of, you still have a Java lang thread, but it would be a fiber, rather than a kernel thread. But when the loom team presented this to the architects, Mark, Brian, and John, they said, well, this is actually a great opportunity after 20 plus years to go back to the drawing board. And if we're doing something that's going to affect how people write concurrent programs, maybe it's time to rethink threads and how we want to do it. So at first, we thought, what really can you do? We'll probably end up where we started, but shortly after we had that conversation, we happened to find a blog post by Nathaniel J. Smith. You may have heard of it. It's called Notes on Structured Concurrency, where he describes something called Structured Concurrency. It was not his invention. It was probably due to this guy, Martin Sustrich, in a library called Libdil, who invented it. But these two blog posts are terrific. And it was one of the very rare occasions, at least in my career, that I read a blog post and said, yeah, that's the way to do it. So the idea behind Structured Concurrency is that instead of spawning a thread, which is a fire and forget, you just start it and it runs for as long as it wants, you try to control the lifetime of threads in a more structured way. So in our current prototype, the way it looks like, we have something called a fibroscope. And I'm sorry, we have a lot of scopes, at least three different types of scopes right now in Lume. So inside a fibroscope, you can start new fibres, but they have the following invariance. You cannot exit this block until all the fibres launched inside it have terminated. So the program is going to block here until all of them have terminated. And again, when I say block, it does not necessarily mean that the thread is blocked. This is another fiber, so it's just the fiber blocks. You can nest them. You can create trees of these fibres. And because their lifetime is constrained, if they throw exceptions, then the exceptions can propagate to the thread that created them. You can cancel an entire scope. And that would cancel all the fibres. And if they're currently blocked, they would get unblocked and signal it somehow, maybe throw an exception. So what are some things you can do with it? So say we want to run a bunch of tasks in parallel, but we only want to wait for the first one of them to finish. It's like a race. So we start a cancelable scope. And for each of the tasks, we spawn a fiber. If we just did that and we were to close the scope there, what it would do is just wait for all of them to complete. But we just want the first one. So whenever a fiber terminates, it adds itself. And it's result to a termination queue of the scope. And you can block in that termination queue and ask for the first one that's finished. And you wait for it to finish. So now you got the first one. And you no longer care about any of the others, but you still can't exit the block until the goal terminates it. You don't want to wait for them, so you just cancel all the rest. And you can do the exact same thing, but also add a deadline. So if none of the fibres finish the task by that deadline, they would all be canceled automatically by the scope. So this is structure concurrency, one of the ways in which we are rethinking threads and concurrency in general. Another way was an idea by John Rose. And that has to do with thread locals. So in the same vein as old threads, thread locals are kind of fire and forget. You can create them, assign them a value. And that value remains set until whenever, until someone remembers to change it back or set it to another value. So we want to do the same thing to control the lifetime of these thread locals in a more structured way. And the current working title for them is scope locals. They're very similar to dynamic variables in scheme. That's the inspiration. They are not in the prototype yet. So this is a purely speculative API. But it looks like this. So you create a scope local. It has some initial value. And then you have another block. And you set the value inside the block. And everyone inside that block would see that value. And that also goes to, so it's not just in the same method, but all of your colleagues. It can be deep in the stack. You can call other methods. They would get this value 10. But once you exit this block, it would automatically revert to the old value. Now, because fibers are now also structured and constrained to a fiber scope, we can combine the two features and say that all fibers that would be launched inside a scope that can see this value would inherit that value. So this is sort of similar to inheritable thread locals. But inheritable thread locals are also fine and forget. They inherit the value of the thread that created them. But they can outlive the thread that created them. With structured concurrency, this is no longer possible. Every thread or every fiber is only alive no longer than the scope that created it. So we can ensure that they only see a value that we want them to see. We may need to make some language changes to this because we want to absolutely ensure cleanup. And right now, this tribe with resources block is voluntary, so may introduce something that says for a particular constant, it is mandatory. And that is all I had to say. Oh, you can go to the Loom Wiki page. And right now, we don't have binaries. No, Lexi Ship Live builds them. But you can download the sources and build them and play with them, and we're waiting for your feedback. Thank you. You have time for questions? Yeah. Yeah, I'm going to try that. I mean, if you have a fiber referencing something on the heap, and then it's yielded, similar, I don't know. So if it's yielded, it behaves the same way. Right, the question is what happens? How does this interact with the garbage collection? If a fiber is blocked, a continuation is yielded. But a block fiber or yielded continuation are the same as a block thread. They're still alive. But if a continuation becomes unreachable, you suspend it and then you forget about it, never terminates, then it would get collected and everything it references would get collected. No, so the question is, do we want to release a continuation without fibers? I think the main concern is once we have continuations, people are going to start writing fibers on top of them. And you're going to get 100 fiber libraries. So it's best to just do it and say, OK, if you want something else, you can do something else. You would still be able to implement your own fibers, but at least we'll show the right way to do it. Well, we can. So thank you. Oh, we have about 10 minutes. Can I start early? No, that's the starting time. I'm always fast. I'll be done early. All right, can I mirror the display? OK. Do you know how I swap monitors? It's got my view up there. Oh, I don't know. Let's see if I can do that. Let's look at it. I think it's. It's not super easy. It's a little hard. No, but that's the. I think it's going to be difficult. Yeah. Do you need to find? I don't know. I'm always just. We usually like a printer or something like that. Yeah. So you have to swap the screens. Yeah, but I can't. But this is the primary monitor. I don't. I don't think it's here. I think I have to go. I can't do it because.