 We have Oasis Zahid. He's software development manager at Autodesk here. And he has 12 years of software development experience in web mobile development, server-side programming, like Gilesgram, and project management, and has degrees in computer science, and project management. So he'll be talking about how to create awesome polyglot applications using Graal VM. Very good. So let's welcome Oasis. OK. Hello. Yeah. OK. Hello, everyone. Thank you for joining this session. I really admire your resolve, like coming in a freezing room to listening to this talk. Yeah. Singaporeans have a very low tolerance of cold anyways. So just bear with me. All right. So today, we will talk about how to create polyglot applications using Graal VM. Quick introduction before we go further. My name is Oasis Zahid. I have over 14 years of experience working around the globe. My main focus area is on cloud platforms, creating cloud products and services with an emphasis on resiliency and scalability. JavaScript is my main forte. I love football and follow A.S. Roma. They are not doing really good this season again, but I still love them. So you can follow me on Twitter and LinkedIn as well. The handles are a bit sort of chopped, but you can ask me later. So interoperability. When we talk about polyglot applications or language interoperability in general, there are a lot of frameworks out there that can do that. So I started looking at Graal VM a few months ago when I was trying to evaluate a messaging SDK written by another team in Autodesk in Java. And my team works in Ruby. So this is not a very unique scenario that people will say, oh, geez, what are the odds? In fact, in the enterprise setting, the development work is done by various teams or agile teams. And more often than not, they choose different tools and technologies for their respective work. Sometimes based on their preference or sometimes based on the problem they are trying to solve. For example, people usually go for Python when they are working on machine learning solutions or go for R for data science. So it is common to come across scenarios like this, where you want to use something and that particular piece of code is not written in your language. So I started evaluating the options. So there are a lot of things out there for interoperability. Some of them are using a language neutral mechanism or interfaces to allow interaction like Thrift, Swig. I can throw protocol buffer into it as well, but it has a different use case. Then you have some libraries which are specific to either a platform or a language, or they focus mainly on bridging the native to manage code basis, like JNI, JNA. And if you look at cloud platforms, they are also trying a few things. Recently, AWS, they introduced layers with AWS Lambda so that you can create a common code base for use across multiple cloud functions. So you can always use their custom runtime API to create a layer in a different language. And then I came across Graal. So Graal has a very kind of a unique way of solving this problem. No interface files that support for languages, they have a good set of languages to support, and then it is not specific to any platform. So today's focus is on Graal VM. So let's look at what Graal VM is. So Graal VM is a polyglot multilingual virtual machine. It creates a common stack for applications running in Java, JavaScript, Python, R, C++, and many more. And this common stack actually allows interoperability in a shared runtime. So just to look at the holistic view of Graal VM, this is a holistic view of Graal VM. There are different layers of Graal VM. It starts with Java VM, then there is a compiler, Graal compiler, and then a Truffle framework on top of it. So there's a lot going on here. So let's take a stab at it one component at a time. OK, so Graal compiler. Graal compiler is the most important aspect of the Graal VM. The only difference between the VM, which is the hotspot and the Graal VM is the fact that Graal VM uses Graal compiler as a C2 compilation mode. So as you know that in JVM, we have two JIT compilers other than Java C, C1 and C2. So these two compilers have different techniques for JIT compilation and can generate a different machine code for the same Java method. So usually Java applications make use of both of them, and which is called tiered compilation, which is a default since Java 8. So in the tiered compilation, in the tiered compilation is start by using C1 at the start of the application for a better startup performance. And then once the application is properly warmed up, then C2 compiler kicks in for more aggressive optimization and better performance. So having like a new compiler that can take, OK, by the way, C1 C2 is also called client compiler and server compiler as well. So both are fine. So replacing a C2 compiler or server compiler with Graal compiler is not something unique. If you have seen different SDKs, like Adobe SDK or Azul SDK, they have their own customized compilers in the JVM. So what Graal is sort of giving us, what C1 and C2 are not giving us. So to start with, a Graal compiler is written from scratch in Java. Unlike C1 and C2 compilers, which are written in CC++, I know you guys will be asking, how can this be a good thing? But it is known that the C1 C2 compilers had known issues. And optimization in the context of strongly-typed managed languages can be done better. So it makes sense to have a compiler written in Java or some high-level language. There is a good link that I will share later. It was supposed to be there at the bottom right corner. It's kind of chopped in, which is actually talking about this topic specifically, like why it makes sense to have a compiler written in Java in the VM ecosystem. But I will share the references at the end as well. So I will not go into the detail of it, because that will sort of become a totally different topic. All right, so yeah, I need to catch up where I was. Yeah, so it's written in Java. And the reason it is possible for the Graal VM to be integrated in the JVM world is based on the thing called JVM CI, Java Virtual Machine Compiler Interface, which was added in JDK9 and since then backported to Java 8 as well. So that allows any compiler to be sort of act as C2 compiler and interface and then sort of get all the events and then process and give it back. So the benefits that we will get with Graal, there are three main benefits. First, it is more maintainable as compared to C1 C2 compilers. It is open source, available, good community around it. It is not tightly coupled with JVM. So you can use Graal compiler in different contexts as well. And the third thing, it has a good support for truffle-based languages. Talking about truffle, that's the second thing. So truffle is a language implementation framework that runs on top of Graal, as you can see it here. So the runtimes created by this truffle framework interprets languages as abstract syntax tree. And it currently supports our Python Ruby. And it has a good documentation for provisioning of new programming languages as well. So if you are developing your language using this framework, then the JVM will look at your programming language implementation as any other Java program. So the advantage of having a common framework to create different languages actually helps you in the interoperability between them. And you can have language agnostic instrumentation around it as well. Because if all of them are running on JVM, so you can use profilers, samplers, tracers, all the things that we do for any other Java program running in JVM. So Sulong is one of the implementations of truffle. It is a LLVM interpreter, low-level virtual machine. It supports C++, Fortron, and all the languages that can be translated into LLVM bitcode using any frontends like Clang or Mac Ruby. So if any one of you are interested in looking at how to bridge the gap between native and managed code and you want to try GraalVM, so this is the place you should be looking at. All right, ahead-of-time compilation. Ahead-of-time compilation, Graal also supports ahead-of-time compilation. It comes with a tool called native image that actually compile your code into native executable. So the native executable will not run on JVM or GraalVM. It runs on a different virtual machine, which is called SubstrateVM. So when the native image tool is used, it actually sort of bakes in that SubstrateVM into your executable as well, so that you can have all the memory management, thread management at that level as well. So here, I sort of one thing to know about Graal. That Graal has two execution modes. One is JVM, and the other one is native. If you run, if you execute your application in JVM mode, then your program will run on JVM. And if you choose the native mode, then it will run on the native images which are created. So usually, native, ahead-of-time compiled applications, they have a better startup time and lower memory footprint. But the applications running on JVM, they have a better peak performance and usually recommended for long-running processes. So this support was also added in JDK9 in GP295. So now it's available. So OK, so far, we have sort of looked at all the major components of Graal. What are the components? And what are the languages it support? And how it is actually set up? But the question is, should you be using Graal? What are the benefits you will get moving into a Graal camp? Because if you have any application running on JVM, you want to move it to Graal VM. Of course, this would be the first question you will ask yourself. And second is, should I be using truffle-based implementations of the same languages that I'm already using, like Ruby's and Node.js and the Pythons and R's? So to find this answer, I ran some benchmarks, both which are sort of stated in the Graal website and others. Based on my experience, I don't trust the benchmarks usually sort of showcase on the site of a particular framework. So I did some digging as well. So I found some interesting results that I will share with you. So the first benchmark that I ran is the embedded templating in Ruby. So the blue bar is JRuby, and the yellow bar is truffle-Ruby. The truffle implementation of Ruby, they call it truffle-Ruby. And the truffle implementation of R, they call it fastR. So you can see that truffle-Ruby is like beating JRuby hands down, like boom, which is really good. And this is the benchmark defined in the Graal VM site. So it better perform well. So the metric is IPS instructions per second. But even if you do just a time-based thing, you can see the difference. So I ran another benchmark, which is base64. I took like a ginormous string and then just do the encode and decode of it. And then I found that JRuby was actually really good. And truffle-Ruby didn't work, was not even near. So truffle was like not trading well with JRuby in this benchmark. So yeah, I will share some details later. So I have created an issue, and we are following up on the truffle-Ruby GitHub repo. There are some things that they need to implement. It's a work in progress to bump up those numbers. And then I ran another benchmark, which is take a huge file and just do a JS1 parsing of it, and then just aggregate the numbers. I will show the code later on. So Ruby was really, really fast. And the matrix I'm using is just plain execution time. It completed the whole thing in half a second. On the other hand, truffle took a lot of time, seven seconds, which is a lot. So I took a step further and just sort of try to see that if can I make this thing a polyglot? Can I, instead of using a parse functionality that truffle or JRuby is providing, can I use Node.js parse function and see how it works? So when I did that, the result I get is four seconds, which is better than seven, but nowhere near the actual JRuby implementation. So this one, the last, the gray bar that you see, the truffle node polyglot, I was running in JVM. So applications running on JVM is sort of start slow. So I want to check what would be the result if I run the same thing in native mode. So I ran the same thing in native mode, and this is the result I got. Drum rolls, boom. So it is sort of half of what the JRuby is doing. So this is, again, I was just playing around, looking at the profiling and looking at all the memory tracing. Yes, yes, yes. So the benchmarks, either the benchmark gem that you use in Ruby or the gem IPS benchmark, which is based on the benchmark gem, they actually do some warm-ups as well. So you can define some cycles of warm-ups and the iterations. So it gives you kind of a closer picture. Yes, yes. So I mean, this was encouraging. And I will show this code later on. And all of this code, all of the benchmarks that I've seen so far, it's available on GitHub. And I have created a container based on the Gral image. So all you need to do is to just clone it and then run it. So you don't need to set up Gral VM. OK, so let's look at this thing in a bit more detail. And that is my first demo for today. The interoperability between Ruby and Node.js. All right. So I need to see how I can have this thing here. OK, bear with me. Sorry. I need to probably end my, OK. How can I show this thing there? Really? Damn, OK. OK, let me see if I can do that. OK, OK. It's OK. This will make things a bit tricky for me. It's not working. All right, so let's look at this one. So this is before the polyglot code. I used the benchmark. Required.js1, a very simple. I'm reading a really large file and then just doing a parsing and then doing an aggregation of all the coordinates. A very simple benchmark. Now I want to show you guys, all right. So this one is the polyglot version of the same thing. So the first thing that you notice is this weird object called polyglot. So this polyglot is actually the interop framework that the truffle use. And all the languages, they implement this polyglot. So polyglot eval, what it is doing right now, it is taking the first parameter, which is the language. And the other one is what you want to do in that language, that foreign language. I mean, you can write a long code in it as well. But right now, all I want is just the JavaScript parser. That's all. So I'm just feeding that parser the JS1. I'm reading from the file. And then the result I'm getting is the J object, which is a foreign object in Ruby space. So to make it enumerable, I'm using as enumerable so that I can sort of loop through it. So this is the only two lines I change. And then I just ran the same thing. And the results you have seen. I think let's execute this thing. I hope I can show you guys. It's hard to sort of do something here just looking at there. Anyways, OK, not working. Sorry, guys. OK, I can just mirror. I think that will help. Cool, OK. Sorry about that. OK, close. Where are you? OK, so I'm in my truffle code base. So if I want to run this thing, let's run this thing in the JVM world first. I'll say JVM. And I will say it's a polyclot just to tell the runtime that I am intending to use foreign languages in my Ruby code. And I also need to tell the compiler that, hey, so this is important, which is only needed for Node.js if you're using Node.js as a foreign language because Node.js or JavaScript is based on single-threaded model. So you need to tell the runtime that, hey, the foreign language is single-threaded. So that it can raise and error if you are trying to access those foreign language objects in a concurrent manner. So this is kind of a safety net that Gral created around it. So right now I'm running it in JVM. Let's see the results. All right, so pretty much the same 6.5 seconds as shown in the benchmark. And so let's try the same thing with native. So the only difference between these two executions is the fact that the second one is actually running the native image of Ruby runtime. In the first one, what actually the VM do is when you say Ruby VM, then it actually sort of load the Ruby runtime in JVM as well. So that slows things down a bit. So if you are sort of fighting for the benchmark, so that you can sort of have bragging rights, so do use native for it. All right, so I have another example for you guys. This one is much more not from the space of benchmark, but more kind of mimicking the real-world use cases. So it's more about the interop between Node.js and Java. So I have created a very simplified notification server in which the notification service. And then in the notification service, I have the notification server written in Java, which is not doing a very kind of a super awesome thing. It's just sort of just putting random numbers in a queue. I mean, you can replace this thing with actual notifications or anything, whatever the use case is. So this is the Java code. It starts and do some computations, and then just put something in the queue. And on the other hand, I have a receiver, which is a Node.js receiver. It is using the worker threads. And so the idea what I'm trying to do is to create a solution in which you have a notification server putting something in the queue, and then you have n number of worker threads sort of pulling the data from the same queue. So for that, the first thing that I'm doing after defining the worker thread is using the link block queue because I want to make sure that the access to the queue is concurrent. So I'm just using the Java queues, one of the from the collection. You can use any of the queues you want. And then defining the threads. And then I am actually initializing and calling a method of a Java function. So this is new. You haven't seen this thing happening in Node.js world before, right? Trust me. So the reason you can do this thing is because both of these code is actually running in the same runtime. And so if you want to talk from one truffle implementation to another, you will use the polyglot framework. But if the truffle languages want to talk to any JVM-based languages like Scala or Java, they will use the Java type. So I will get that notification server, and I will just call the start method by giving it a queue that I created at the top. So let's see how it works. Let me check. Yeah, that's the same one. So what I will do is I will compile my Java class first. And then I will run Node. Yeah, since I am using a worker thread, I need to check that notification, receiver.js. I don't know whether it works or not. Oh, yeah, it works. So yeah, it's just sort of console out. So it is showing that the items Java server is pushing into the queue at a regular period of time. All my workers is picking it up. So another thing is, all right. So these are the two main demos I have for you guys today. But remember, at the start, when I mentioned that I started this crawl journey, I'm not an expert in crawl, by the way, and not like Oracle is not paying me to do this talk. So the whole reason I started looking into crawl is because I wanted to use the Java SDK in my Ruby code. So let me just show you the code because I cannot execute that code here because it uses all the servers in Autodesk. But I can just show you the code. So it is SDK based on Kinesis. So I just took the SDK by just running the Maven and then just create a SDK with all the dependencies. And then just, OK, let me just make it more smaller. So and this is the Ruby code that actually uses the whole SDK, and it works. And I can do the same thing with Node.js because people will argue like the people who have already worked with JRuby, like, hey, you can already talk to Java. I mean, that's not a new thing. You can have like javax.script or you have bean frameworks that can allow your JRuby code to talk to Java. But just think about the situation where you have to use more than one language. Maybe you are coding in, you are creating an Express Node.js server. And then you need to use R or Python or maybe some of the Ruby and other stuff in it. So then having a consistent framework around it makes it very simple. So is there any time check? How much time I have? OK, let's go back to our slides. So based on my journey in the last two months or so, there are some lessons learned based on my sort of adventures with Krall VM. First is this kind of solution, like a VM-based solution, is better suited for a situation where you want to use some library or some SDK written in some other language. I mean, there is a scope of this interoperability. This should not be taken as one solution that can fix all of your interoperability problems. Maybe if you are designing your overall solution more in the microservices-based architecture, then the advantages of doing something like this will be very, very minimal. So you can choose when to use it and when not to use it. It helps if you have like working in a big organization where sometimes it happens that you create something and then you create variations of it. So these type of activities will help you create a single source of truth so that everyone is actually sort of improving it and then you are not creating like a Java SDK and then Ruby SDK and then C++ SDK and all that. So this is an advantage. So you can choose when to use Krall VM and when not to use Krall VM. The second thing is there are still some performance gaps in Truffle implementations. But it's an active community. I can see a lot of improvements happening. But the problem with the Truffle implementation will always be that it will always play kind of a second fiddle to all the advancements happening on the respective languages. So let's say if Node.js come up with a new thing, they need to sort of come up and then do the implementation again. If Java come up with Java 12 or 13 with some totally different thing, they need to come back and do something about it. So this is something that we need to sort of take into consideration while working. Maybe run some benchmarks, do some profiling, do some comparisons, rigorously test it before sort of deciding what to do. The third thing which is not related to Krall at all but is like just my pain of like using JTK in Docker container doesn't fit well. It's not kind of a hidden fact anymore. So Java 8, if you're running and then so it doesn't follow the memory and the CPU restrictions that you put on your Docker container. So sometimes you say, OK, killed. It will just say it's killed. So it will not say heap out of memory. So if you are experimenting and if you can, better to use JDK 10 plus, it has a better support for Docker containers. The last thing is that Krall is not widely used in production environment as of now. So far, I know from all the material and all that that Twitter is using it. But I haven't seen any other kind of a success story coming out of it saying that, OK, they are using Krall and it's like making a difference. All right, so these are all the references that I mainly use. This is the GitHub repo of mine, the Krall benchmarks that will contain all the code and all the containerized code so you can just run it. I mean, you can run it right now and show me what are the numbers. Krall VM GitHub, they have very good documentation. If you want to know about in the JDK world what are the changes they did that actually enable a solution like Krall VM to come is you can look at these 243, which actually implemented the JVMCI. And then there are some research papers on Truffle Ruby and overall on LLVM based languages. All right, that's all I have. So if anyone has any questions, feel free. So let me ask you guys a question. I mean, what do you think about Krall VM? Because I am not an expert in it as well. But I'm sort of just trying it out. I'm finding it cool right now. But I mean, what do you guys think on all the things I've shown here so far? Is this something that you think like, it's interesting? Or you say, no, I'm good with whatever I'm doing. OK, just raise your hands. How many of you think, OK, this is interesting, OK? That's not bad. But how many of you think, OK, this is, ah, no, I'm good. Because I want to learn from you guys. Let's have this two-way communication. So what do you think? What is your reservation? Or why you think like, I mean, it's JVM. It goes wrong all the time. Even the hotspot JVMs do that, right? So but it is a JVM. I mean, you can just take that Krall VM and you can use your any JVM that you have, like maybe based on Java 10 or 11, maybe Azul or Adobe anything. And you can just sort of configure your JVM to just use it. I can comment on that. So Twitter is using Brawl because they're a scholar shop. They are using Scala very heavily. And they benefit a lot from using Brawl. I have things like 20% to 30% speed improvements in my head. But they have a dedicated team that is actually moving into the OpenJDK development. And they have some guys doing memory and garbage collecting. And they have some guys doing VM stuff. And they have their own internal OpenJDK, no JDK version, which diverts from the OpenJDK. So they're in another position than just like a normal. There's nothing like normal, but just an outsider just want to use Java or Ruby or whatever. Just want to run their programs. So whenever something goes wrong, you are pretty much on your own. Or you need to know who is actually the developer in charge and how to address the developer. And that's the basic thing about any work with open source software. Either you need to get engaged within the community and know how to address the people, work with mailing lists, whatever. And file back reports or don't and buy some commercial support. But there is no commercial support for Gral at the moment. I forgot to tell you that. So Gral VM, they have an enterprise edition as well that they say that they will support. I just use community version. But one thing that I can recommend everyone, even let's say if you are not a very technical compiler savvy techie, if you just want to see the compiler impact, especially on the GC, on the garbage collection. So since Java, since JDK 10, they introduce the option where you can run your JVM by disabling the garbage collection or just not doing it at all. So the easiest way, if I want to do it with a limited knowledge, what I will do is I will just give it a very high memory and then just run it without the garbage collector and see how it works and then use it with one compiler and then use it with another compiler. So you can at least get some rough estimate at ballpark like how things are sort of working. But again, having a dedicated team, I agree with you, Twitter, they have people dedicated working on optimizing all their JVM-based compilers and making it sort of make sure that remain there. I mean, for experimenting it, that is OK. If your plan to go in production within a year and you have a little bit of time to get in contact with the people, then it's fine. If you want to run something in production today, it's still a bit of a bleeding edge. I agree, because I haven't seen any other kind of a big success story coming out of Grawl other than Twitter. But let's see in the future. Yeah, any other questions? I'm just making questions right now. Anyone who only has experiences, positive or negative, with the JVM stack that they might be interested in sharing? We have five minutes left. Yeah. Or I mean, you can ask me, I'll be here today, tomorrow. Just another comment on the, which is another topic actually about the cloud thing, you said. There are a lot of improvements in 10 and 11, which makes it better suited for Docker containers. They switch to the new garbage collector. The G1 garbage collector is one example, which gives back more aggressively the memory back to the operating system. The problem is at the moment that Java 11 is actually the long-term service release. But it still has some serious bugs, and there is some security updates, maintenance release in the works. So people are hesitating now to actually switch to 11. So at the moment, if you have critical production workloads, you should stay with 8, wait for the next maintenance release of 11. So at the moment, there is a lot of flow in these all OpenJDK development. There is a switch that actually Oracle steps down with their leadership in the OpenJDK development. And Red Hat does the maintenance releases. So this is all stuff that is in the flow at the moment. I totally agree with you. This is the case because most of the applications which is based on Java right now, they are mainly running on Java 8. I haven't seen any production Java applications on any financial institute or anywhere running on Java 10 or 11. But I think probably sooner or later, people will make the move because recently Oracle changed their arrangement with how they will do the back porting. And you need to sort of engage with them in the licensing form. So either you go to some other, if you want the back port, let's say if they added a new feature in Java 11 and you want it in Java 8. Because usually right now it happens if you use it. There's some stuff going on there. So all of this is that good site. Yeah, so that's an interesting topic. We can discuss about it. And so are we going to any other questions? Let's thank a race. Thank you so much. Thank you. Thank you. Thank you.