 Hello everyone, welcome to my talk. Today I'm going to talk about foreign function and memory API in Java and state of FFI in general. So, my name is Deepu Keshashidran. I'm the co-lead of J-Hipster. I also created a Nifty dashboard called K-Dash for Kubernetes. I also created the JDL Studio for J-Hipster and I work on a bunch of other open source projects. I'm an open source aficionado and a polyglot developer. I work as a developer advocate at Okta with a focus on DevOps. I also publish frequently about languages and tech on my blog. You can find it on Deepu.tech. Please do follow me on Twitter if you are interested in my content. I have written a book about J-Hipster. If you like this talk, you might like the book as well. So, let's see. What is a foreign function interface? A foreign function interface is the ability to call functions or routines returned in one programming language from another. So, this is generally used to access native functions or programs on the host OS. Most languages provide this feature out of the box in some forms. It is also known by different names in different languages. Most languages use the CC++ calling conventions and natively support calling CC++ functions, especially C functions, which means any language that has C-interop can also kind of work because you can generate C headers from those programs and you can then call them. So, basically, you call routines from another program regardless of the language. So, most modern languages provide this feature in intuitive ways like, for example, Golang, Rust, they all provide very intuitive ways to work with foreign functions. The term originated from CommonList, the term FFI originated from CommonList. So, in Java, we don't call it FFI. And similarly, I think in some other languages there are some different terminologies, but the most commonly known terminology is FFI. And most languages, they default to CC++ calling conventions, as I mentioned. So, back in the days, like previously, or maybe like five, 10 years ago, most of the use cases for FFI were around interacting with legacy applications and accessing host OS features or native libraries, stuff like that. But these days, FFI is becoming more and more unnecessary, especially with the advent of machine learning, deep learning, GPU processing and stuff like that. So, these days we use FFI to interact with legacy applications, of course, to access features not available in the language that you are working on, to use native libraries, to access functions or programs on the host OS, and newer approaches like GPU and CPU offloading. Like, for example, if you want to work with programs like CUDA, OpenCL, OpenGL, Vulkan, Datatex, so that the CUDA and OpenCL are CPU offloading applications, whereas OpenGL, Vulkan, Datatex are GPU programs that you might want to have direct access to. Then there is also multi-precision arithmetic, matrix multiplications, these kind of things are much more efficient when done in a native language, and they are much more performant when accessed in a native language. Then there is deep learning with TensorFlow, CDNN, BLAS, et cetera. There are also use cases like using a specific program or library written in a native language like OpenSSL, V8, the V8 engine from Node.js or Chrome, SQLite or integrating Python interpreter in your application, et cetera. So there are many more use cases for FFI these days. So, first let us look at the brief history of foreign functions in Java. The standard for FFI in Java for a long time is JNI, which is Java native interfaces. And honestly, it's notorious for being problematic. If you're used to other languages like Roscoe or Python, you will probably know how easy and intuitive it is to use FFI in them, and that leaves something to be desired in the Java world because of how convoluted and how complex it is to use JNI. Even to do a small native call using JNI, you would have to do a considerable amount of work, and it could still go wrong and end up being a security issue for the app. The main issues with JNI is its complexity to use and the need to write a C-bridge code manually. This could lead to unsafe code and post-securities. It can also cause performance overhead in some situations because you'd have to write a native code, and we all know the performance and safety of native code is as good as the programmer who is writing it because there is no inherent fail-saves again, writing memory on safe code, and performance, of course, means you have to write performance code in the first place. The performance and memory safety of JNI code depends on the developer, as I mentioned, and hence the mileage may vary. In my opinion, it is high time for the Java world to move away from JNI, but unfortunately, it is still the most performant way to do FFI in Java, if implemented properly, not counting whatever we are gonna see later on. But for the time being in the standard Java libraries, table standard release, even as of JDK 17 or even 18, JNI is the most performant way to do native calls. So the complexity of JNI gave rise to some community-driven libraries that made it at least simpler to do FFI in Java. At least it reduced the complexity and made it easily accessible. JNI is one of them. So it is built on top of JNI, but at least it makes foreign functions easier to use, especially it removes the need to write any C bridging code manually, and hence reduces the chance of some memory safety issues, memory safety issues. It still has the same disadvantages of JNI, and it is slightly slower than JNI in many cases. It is also widely used and battle-tested, so definitely a better option than using JNI directly. So unless your use case demands extreme performance during the most, if you want to get the most performance out of your implementation, then probably consider doing JNI directly. Otherwise, there are alternatives, which uses JNI underneath, but are much more simpler and much more maintainable. So another popular option is Java native runtime, or JNI. And though it's not as widely used or mature as JNI, it's much more modern, and has a much more modern API. It has better performance than JNI, and there are also, I mean, it's much more performant than JNI. So after JNI, JNR is, it's not as performant as JNI, but it's the closest, it's a close second, I would say. And it also has the same advantages like you don't have to write any C bridging code, because it does dynamic binding. It has a modern API performance, as I mentioned, very easy to use, similar interface access for CEC plus plus assembly extra, but also has the disadvantages of JNI, because it's built on top of JNI, and it's also difficult to debug like other options. So that's like what we have currently, that's like stable stuff we have currently. So when it comes to what's next, right? What's the evolution? What's next for foreign functions in Java? That is Project Panama. So Project Panama is the latest Java project aiming to simplify and improve FFI, and as part of this, there are many proposals that's currently being incubated. So let's take a look at some of the active proposals and how it will work, and let's see if we finally get proper native FFI in Java. The first piece of the puzzle is the foreign memory access API. So the foreign memory access API has a few goals. So the main thing is safely and efficiently access foreign memory outside of the Java heap. So a foreign memory in the sense native memory or off heap memory. So that's the primary goal of foreign memory access. It has certain, all these APS have certain standard goals. So consistent API for different types of memory, that is one. So JVM memory safety should not be compromised. So that is one problem with JNI is the memory safety issues because JNI kind of exposes the entire JVM to the native library. And if you end up writing unsafe memory and safe code in the native C binding, you could compromise your entire JVM. So JVM memory safety should not be compromised. Explicit memory de-allocation, that is one goal. And this particular API is designed to interact with different kinds of memory resources, including off heap or native memory. So this was first incubated in JDK 14 as JEP 370. Then it had a second incubator in JDK 15 and third incubator in JDK 16. So currently this is, this is combined as foreign functional memory API. We'll see that a bit later. The second part of the puzzle, which actually makes foreign function calls possible is the foreign linker API. So foreign linker API provides APIs for statically typed pure Java access to native code. So it has goals similar to the previous one. It was also its own goals. So focus on ease of use, flexibility and performance. It has initial support for C interrupt. That means any language that can do C interrupt can also work. So call native code in a DLL, SO or DILIP. So that's how native calls will be made. It can create native function pointer to Java method, which can be passed to code in a native library. So it can do two way communication with a native library. So this was first incubated in JDK 16 and then this was combined with the previous foreign memory access API to become foreign memory and it was combined to become foreign function and memory API. So we'll see that. There is another lesser known part of the Panama project which is the vector API. So vector API is not that not spoken, it's not talked about much, but it's also quite important because it's an API for reliable and performant vector computations. So it is platform agnostic. It has clear and concise API, reliable running compilation performance and grace redegradation. So these are the goals for this particular project. So this was also first incubated in JDK 16. Second incubator in JDK 17. I think I have a typo there, so that should be JDK 17. And third incubator in JDK 18. And currently what we have is a evolution of the foreign linker API and foreign memory access API. They evolved together to become the foreign function and memory API. So it has the same goals and features as the original two, which is ease of use, safety, performance and generality and combining them means they will have concise APIs, shared goals and everything. So this was first incubated in JDK 17, which is the latest LTS release. And the second incubator is planned in JDK 18. And hopefully after the third incubator we'll start getting previews for this and after probably within the next two years or something we could have a stable release with Project Panama featured there. And finally we could get rid of JNI. And last but not the least, there is also an awesome tool called Jextract as part of Project Panama. So Jextract is a simple command line tool. It can generate a Java API from one or more native C headers. It's shipped with the open JDK Panama builds currently. So I would assume that once Project Panama becomes part of JDK and becomes a stable feature, this would also be shipped with the JDK. This makes, Jextract makes working with large C headers a cakewalk. We'll look into a bit more detail about that. So first let's see how foreign function and memory API differs from JNI because that differentiation is important for us to appreciate this. So let's take a very simple example of calling the getPit function from the standard C header like the uni STD C header. As you can see here, there are exactly six steps to make this simple native called using JNI. You start by writing a Java class which declares the native method. Then you use Java C to generate a header, generate a header file and a C class for this Java class that you have written. So the header file at least is auto-generated and you get a stub for the C class. Next, you will implement the C class because this is the C binding button. You have to actually manually implement what you want to do. So if you want to return getPit there, then you have to actually go in and write that code. You have to go in and write return getPit with the proper includes and everything. So you have to actually do that. So also remember like these are Java developers writing C code. So if you're a Java developer, the possibility of you being an expert in C or even you being familiar with C is kind of like 50, 50 maybe? Yeah, depends. It depends on different modules, right? But if you're using JNM, then you are expected to write C code for whatever functionality that you are gonna do. Which means you also have access to the entire JVM via the JNM variable. If you look at the generated C class here, you can see that there is JNM pointer for the entire JVM environment being passed to your native code. So you can basically do whatever you want here. You can probably go in and corrupt JVM memory. You can compromise the JVM security. Lots of possibilities, fun, right? So imagine being a Java developer without much experience in writing C code and being handed this entire memory pointer and you doing something, because even the most experienced C developers still cause enough CVEs because of the nature of C. C is inherently unsafe language. It's not a memory safe language. So there is no memory safety. So you are responsible for ensuring memory safety, which is the worst thing I would say because depending on humans to ensure memory safety of a computer program is the worst thing you could do. Because that is kind of like all the CVEs we have like every other day because of a memory safety issue talks to that. So I don't think you could have a counterpoint that trusting people to write memory safe code is a good idea because 70% of all security vulnerabilities in our industries because of memory safety issues which is caused by people writing unsafe code in a language like C, C++. So imagine having that kind of responsibility, I mean giving that kind of responsibility to a Java developer and imagine the pressure. So yeah, so fun and probably security nightmare. So once you navigate the native or C code part, you would have to compile the C code into platform specific dynamic library. So that's another step of fun, figuring out what format is required, compiling it, getting it right and everything. So then you have to figure out where to place this library to make it work on that particular platform and probably also pray that all this works without exposing your entire app to some sort of security issue. So next you load this into the Java class that you wrote in the first step and then run the class and hopefully it works. So this was just to get a simple get bit call. So this was what you had to do to get a simple get bit call to work, right? So imagine writing something like an OpenGL interface or GPU offloading or using CUDA in your Java program with JNA. So imagine that how complex this has to be. Now let's see how we can do the same using the Panama APIs. So we can do this in two ways. The first approach would be by manually looking up and loading the native function. Second approach would be using the Jextract tool. So in the first case, you just write some Java code using C linker, which is part of the foreign linker API. You look up the native method and you move it. It's as simple as that. So you can also do more complex stuff like working with native memory, two-way communication with the native code, et cetera, all this way. With this approach, you're actually using the foreign linker API and foreign memory API directly to do native calls and to manage native memory. So this is not the most efficient way though. If you use Jextract, this entire code block that you had to write on the left hand, that you see on the left hand side can be reduced to one line of code. So basically you have to run Jextract against the header that you want to use, and it will create Java API or Java access points for that entire native library or that header. It will create access points with Java interfaces and everything, methods and everything, and then you just call the method. So it's as simple as that. With Jextract, you're getting a pure Java API for the native program, and you won't have to write any native code or touch any header files to do anything. The only thing you have to do is find the file, run Jextract, call the method, that's it. So isn't that awesome? So this is the kind of FFI experience that you get in languages like Go and Rust. So with Project Panama, finally in Java, we have a similar experience. So you don't have to actually do a lot of manual binding. You don't have to worry too much about memory safety because the tool is taking care of generating the bindings and all those things. You don't have to write any native code yourself. And finally, you can be a Java developer who only writes Java code, if that's your thing. So for simple native calls, where you're just calling one method or something, then the first approach might do the trick. You could get away with that. But for any complex use cases, the second approach using Jextract is much more superior, much more efficient. Otherwise, you'll end up with a lot of boilerplate code. And there's also another reason for that. I'll come to that. So I actually ran some benchmarks to compare performance of JNI and Panami API. You can also find, I think there's a GitHub repository where there is benchmarks with JNI, JNR, JNA, Panama, and a bunch of like one or two other libraries. There's a similar benchmark there. You could also check that out, which, and the results are quite similar. So what I did was, yes, okay. So you can actually find the repository. I have given the link in this slide. You can find the GitHub repository where I have uploaded the code for this. It's a simple benchmark. It uses the GetPid example that we saw. So I use both the JNR version, Panama using Jextract version, and Panama using direct seal linker down calls. So I ran the benchmarks, and this is the result. So it seems like using Panama APIs with Jextract is as performant as JNI. And this was done on OpenJDK 17 Panama build. So the Panama, I mean, foreign linker and memory access API with Jextract is as performant as JNR. And in some cases, it was even slightly more performant. And in some cases, JNR was like, so on average, it's as performant as JNR. And this is the incubator state. So I'm expecting this to become much more performant than JNR once it has reached stable state. However, using Panama APIs without Jextract seems to be quite slow compared to JNR. That's probably due to the dynamic loading of native methods and stuff like that, because Jextract analyzes your C header, passes it and generates optimal APIs and everything. Whereas if you're writing your own code, probably it's not as performant as a generated one. But I'm not an expert in the Jextract area or the native memory access area. So if anyone knows what is the actual reason for this huge difference? Because it's not a small difference. It's a considerable difference. First, I thought it could be a one-time thing. So I ran the benchmark like 10, 15 times and with different configurations, different setups. I tried a few different variations of the code with having a static linker, with creating the linker each time using a stateful... I mean like a singleton and everything. But regardless of what I tried, the result was almost the same all the time. So I don't know what exactly causes this huge performance drop when using the APIs directly without Jextract. But regardless, in realistic use cases, I think the way to go would be to use Jextract because I don't think in a realistic use case. Say for example, in your Java program, if you're gonna do any CPU work or if you're gonna access OpenGL to do some say 3D, render some 3D model, I don't think you're gonna do that using direct APIs. I think you will be using Jextract to get the Java API for OpenGL and then you'll be working with that. And I would expect that to be the default way of using the Panama APIs going forward. Like once it's stable, I would expect that to be the default way because that is much more efficient and as you can see, it's also much more performant. So the only cases I could think of where you would actually use the APIs directly would be maybe, yeah, maybe if you're just doing a one-off call, like the get bit call, or if you're just accessing one method from some library in a one-off thing, then maybe it would make sense to use that because then you don't have to worry about using Jextract to create all the, Java API because this is also not, so because FFI is not platform-independent. It is platform-dependent. So if you want your Java program to work across, say Unix and Windows, then you'd have to compel against both the versions of the header and you'd have to provide both libraries and then you'd have to write code to detect OS and work across, et cetera, et cetera. So maybe for a one-off call, using direct APIs might be much more simpler to handle because you don't have to worry about all these natives, like all these binaries and stuff, whereas for an actual use case involving a large library, then Jextract would be the way to go. So are we there yet? So in terms of FFI, comparing to a modern language like Go or Rust. I would say we are getting there. The Panamai APIs can already work with languages that are C and R. That means you can work with C, C++, Fortran, Rust, etc. I already tried creating, I already tried out calling Rust functions from Java using FFI and works fabulously. So performance is on par with JNI. Hopefully it will be improved further. So Jextract makes things really easy in terms of using native lips. The only challenge I had when I started using this and when I started exploring this was figuring out what exact Jextract command to run because if your header file has dependencies, then you have to provide those command line parameters and everything. So figuring out that and getting the right Jextract command against the right header file was the most challenging thing. Then the next challenging thing was, of course, navigating documentation because at the incubator stage, I think most of the documentation you will find are quite outdated. I don't know why because most documentation I found even on the official Panama repository, the official Oracle pages, JDK pages, they were all quite outdated. So you'd have to read Java docs here and there, try to figure out what changed because this is an incubating API. So there'll be a lot of changes in API, a lot of deprecations, like a lot of breaking changes without warnings and stuff. So you'd have to navigate that. So documentation needs a huge improvement. Even the official examples you will find, most of them are quite outdated. If you try them with the current JDK versions, it will not work. At least I couldn't get any of them to work as cities. I had to make a lot of changes to even make some of them work. But it's an incubating feature. So this is all expected, I would say. And of course, you can already access native or off-heap memory. You can call, make JNA calls. And this is much more memory safe and less brittle than JNA. So finally, here are some links that I would recommend. If you want to learn more about Panama APIs. And with that, I would say goodbye. So thank you, folks. I hope the talk was worth your time and thank you for attending. You can reach out to me via Twitter and do check out my website for more content. Thank you, bye.