 Yeah, some code and benchmarks and codes and benchmark and Hopefully that's gonna be useful for some of you So I'm a software engineers engineer for elastic in the lock stash team. So lock stash is written Jirubi entirely And I have two colleagues here guy and perry We're working with me My team So yeah, we're gonna we're gonna talk about Fast IO or how to achieve efficient Persistence using a Jirubi and Java So this is this is work. I've been I've been doing to For the for the actual implementation of a persisting queue I'm gonna and I'm gonna show you that a little bit later on So if this is this is too boring for you guys you can jump to the code right now There's a Jirubi conf 2015 repo with with all the slides and the code and the other two repose and map and MAP cues are the actual Mapping implementation and the cues implementation Okay, so first of all we're gonna talk about Using data or manipulating data in Ruby Yeah Yeah, so Soon enough you're gonna be asking yourself why there's no byte array in Ruby or you know, I wish there was a byte array In Ruby so in Ruby when you manipulate data everything is is true using you know using strings actually to hold the data and There's two important things that you have to consider when I when doing that when the you know Manipulating data is the character set and encoding with your strings and if you're calling out to Java Java classes then you have to Pay attention to the actual type conversion that occurs between Ruby a Jirubi and Java Now there's a very nice Wiki for that that explains some of the pitfalls and some of the the techniques that you can use to Get more speed when we're doing that So why I urge you to actually look at that if you ever want to play with you know crossing the world between Ruby and Java Okay, so first of all, let's talk about encoding I'm not gonna go through all the details for encoding. I'm gonna talk about a little bit the Ruby side and the Java side So these are some of the stuff that that you may have seen for encoding of course the the first line The encoding for the Ruby files So the the strings that you're gonna define in your Ruby files are gonna be encoded in that character set So UTF-8 in that example There's classes and constants that you can use a UTF-8 ASCII 8-bit and This is important to understand what ASCII 8-bit is and we're gonna see the equivalent in the in Java Which is ISO 88 59 underscore 1 This is actually a transparent 8-bit character set. So it's equivalent to saying no no encoding, right? and So main most of the these encoding Methods are in the string class. So you have like force encoding encode Encoding which which is gonna give you what the encoding that the string is in right now Is it a valid encoding and and the actual byte size of a string? so it's it's it's important to note here that The only thing that does transcoding in here is the encode method So force encoding doesn't do any transcoding. So it's only a tag for the string to tell it in what encoding it is So I'm gonna show you some code that uses that So the equivalent in the in Java So we have like the default character set in Java And then there's some some classes and constants again for for the the encoding so Like I said ISO 88 59 1 which is the 8-bit transparent encoding You can set the default encoding in your Java application using the the file that encoding You can get it With as a property or you know if you're from Jirubi you can use the end Java file dot encoding to see what the default Encoding is in Java Okay, so let's show an example here if I define a string with Erics on tigu or a cute accent E Obviously, this is a unique code UTF-8 is going to be a two-byte encoding So we see that the encoding here is UTF-8 if we ask the size It's it's a one character string if you we check the byte size then it's two bytes If we force the encoding to ask you a bit which is going to be a transparent 8-bit encoding We get back that that string and if we ask for the size it's going to be two and bite sizes too So this is to show the difference between you know a string length and a bite size for that's that same string Okay, now let's talk about object persistence so And this is why I'm obsessing with string and it's very important when we talk about persistence and IO it's important to You know get the the the big picture about encoding and make sure that that you actually Understand that because you may end up having you know wrong encoding and problems with that So the problem is that all Ruby IO use a string so you you don't have a choice you have to go through strings one of the problem that we have is that if you are a Jruby object if you want to do persistence Cannot be serial it cannot implement the serializable serializable interface in Java So you cannot benefit native native a serialization or really faster realization using I don't know Frameworks like cry or stuff like that. You actually serialize your object within Java This is because Jruby objects actually hold references to the runtime and and we cannot serialize that I know that there's been some work To actually be able to have serializable objects in Jruby There's some open issues about that, but it's not there today So basically to persist an object you need to serialize it somehow To a string because you want to write it, right? so You can use you could use Marshall dump, but that's the same thing. It's gonna serialize to a string also Jason and so on so so you're stuck with strings So so yeah, we're gonna we're gonna play with strings and We're gonna do some benchmarks with different strategies all the code and example are gonna use string object, so I'm not gonna talk about serialization. This is gonna be your problem to choose whatever method that you want to serialize You can talk to guy the other of junior Jackson here or maybe a Satoshi for a message pack go see these guys But here for the sake of these these examples We're just gonna deal with with strings objects string buffers and then persist that But of course in the real world You're probably gonna have some serialization cost to include into these these benchmarks or Okay, so what's the motivation here? this is so I work on log stash and This is basically log stash the log stash pipeline. It's just three horizontal cylinders and two vertical cylinders Connected with our rows. Basically, that's it. That's a few years of software engineering right there So we use so these these are the stages the input filter and output stages in in log stash and they are Connected through a size queues and basically we use size queues there to actually Propagate back pressure when the outputs are slowing down So these size queues gonna fill up then then the the filters won't be able to Push an event in there So the the other size queues gonna fill up and so on and then it's gonna propagate the the back pressure back to the the input The input plugins or the input stage So the idea here is that these size queues are in memory queues So if there's this a crash a system crash and application crash or whatever We're gonna you're gonna use all these in flight events and in log stash These are smallish queues, but nonetheless this can cause a problem, right? So one of the One of the way to to solve that is to see can we can we persist those events in these queues? So there's many solutions to that problem, but one of which is saying okay, we're gonna do Just a persistent queue implementation in it right there and and that can be just a drop-in replacement to the In memory queue and then be done with it It's gonna be persist and if it crashes then when the application restarts and gonna read the persisted queue and then and then continue on with the events so that leads us to trying to Find out the best way to persist because log stash processes, you know Thousand hundreds of thousands of evin per second. So we need to be really really fast in terms of persistence so RIO performance or you know storing as many objects in the last time Possible and oh this is not Satoshi here in the picture Okay, so different strategies that we're gonna explore We're gonna the base the base Benchmark that we're gonna use is you know plain Ruby file IO Obviously after that we're gonna see Can we do better with M mapping so memory mapping? I'm gonna talk about that we're gonna have a Java class implementation and We're gonna test different strategies to talk to the Java class so in plat implicit casting explicit Ruby side Java side casting play with the character set a little bit We're gonna build the Jirubi extension Java and then have a pure Ruby implementation also and check the the performance results Okay, I'll benchmarks Basically that these benchmarks are for right speed only If we have time I'll come we'll we'll see the actual queuing implementation Which you know reads and writes but these basic benchmarks are for right speed We're gonna use a one four and sixteen K buffer buffer size again. These are plain string buffers and We're gonna be writing End times two gigabit two gigabyte files So the end times depends on the on the test So I was just I just wanted to make sure that it takes a long enough to get you know Results that are in the few second range So this is this has been run on my my book my book here it has a Local SSD so of course there's a lot more IOps than than a spinning disk Latest Java and latest Jirubi 1.7. I haven't run the test on 9k yet, but I'll do that Okay, so standard Ruby file IO So if you want to go in the repo and check the the implementation I have a method there that's called bench and It passes me a buffer in there and a right count and and the only thing that I do So this this bench Method is actually benchmarked. What's happening in in that block and I'm not counting the creation of the file so it's just gonna go on and Write the the buffer that's giving to me. So it's gonna be you know one K4 case is in K and so on So so this is the first result that we get this is gonna be our base result So we see that we have you know between You know 680 and about 900 megabyte per seconds in terms of throughput With with file IO with standard file IO Okay, so now an alternative is to use a memory mapping so Conventional file IO use read and write system calls and that involves, you know copy operation between the file system Pages in kernel space and the memory area in user space. So there's always copy going on but Memory mapping IO Uses like a virtual memory mapping from the user space directly to the file system pages and with a memory map file The entire file is going to be accessed using a byte buffer class and The byte buffers that is you know, that's it. It's a byte buffer So you you actually manipulate your file the underlying file just as a byte array And you put bytes and get bytes so you don't have anything Like like read lines or end of files or anything like that. It's just a big by the rate that you manipulate Okay, so what some of the advantages of using memory mapping So you you see the file as plain memory like I said as a byte buffer There's no need to issue read or writes If you access or you get bytes from that memory space, there's going to be page faults In the OS that will bring the file data from this to these these these memory areas So if you put bytes or you modify the memory map space These pages are going to be marked as dirty and they are going to be flushed to this eventually So the OS is going to be performing the caching and managing memory According to system loads and available memory One of the important thing is that the data is always page aligned. So there's never Buffer copying that's required. So this is this is probably one of the biggest benefit And also very very large file. I think it's up to four gigabyte can be mapped Without consuming actual large amount of memory, right because the data is pulled in as as needed Okay, a few notes If the user process crash the end the memory map file is actually intact So because those bytes have been managed by the OS In the pull the plug situation. Well, just like a normal file You don't know exactly unless you've been doing, you know a flush or f-sync, right equivalent of that in In with with m-mapping is to use a force So force is the equivalent of flushing and then doing an f-sync and These tests we're not doing that So and this is this is a strategy. So just like the file. I often have not done any flushing or f-sync in the file Io test I'm not doing any any force or flushing with memory map And the performance of memory mapping is going to be relative to your file system type the free memory that you have in your system For doing the file system cache and the read write block size Normally m-mapping should be much faster than streaming Io This is what's to be expected Okay, so first We're gonna look at a simple Java implementation for memory mapping now. I don't know if I Can pull that in here. So this is Very simple in addition. So in the constructor here, we see that we create the file We we we get a channel and then we call map which established the memory mapping like the m-mapping to that channel and then the methods here are are really Just a wrapper against the the byte buffer put and get So it's very simple implementation Okay, so we get we have different put here that we're gonna use for the different benchmarks. We're gonna look at that Okay, here we go. Okay. So the first one So using the Java class we're gonna use implicit casting with the default character set that means that we're gonna pass in Our Ruby string to a Java method that accepts a Java string, right? So we can see it We we use out that put bytes buffer buffer is a Ruby string and below This is the actual method Java method which accepts a string a Java string So and it gets the bytes. So so data that get bytes And and we use the buffer Which is the memory map buffer that that's been defined and we do a put for those bytes as simple as that Okay, so in this case using a mapping we're actually slower than follow you and Buy a good margin. So this was a really a What the hell is happening here? Okay, so why is it slower? It's supposed to be much faster. What's happening? So the first question was okay. Is there is there encoding transcoding going on? So let's see if we can you know specify explicit encoding here Since our string here are defines that simple, you know ABCDF And then we do a forcing coding on as ASCII 8-bit. So our string are already, you know 8-bit transparent and And but we see that the default character set in Java is utf 8. So, okay, maybe there's transcoding going on So instead of using the get bytes the default get bytes which is going to be using the default encoding in Java utf 8 Let's use the get byte and specify our character set, which is going to be ISO 88 88 59 1 right so this is what we're going to do here instead of doing a put byte buffer We're going to put bite with the character set and this is going to be passed on to Java And there's going to be explicit character set here to avoid any transcoding and this is what we get So it's a little bit faster So this is not the transcoding problem. So what's happening? So of course here, we can look at you know type conversion. Is this is this what's happening? We see that you know, we use a buffer a Ruby buffer and we pass it to a Java method that uses that accepts a string a Java string so let's try and do explicit Ruby side casting instead of you know relying on the J Ruby implicit type come version So you can see on the upper part where we do the out the put bytes. We're going to do buffer to Java bytes So this is going to pass in a byte array a Java byte array to that method and This is going to be using below the Java implementation of put bytes, which accepts a byte array Okay, so instead of so we do an explicit Ruby side type conversion So what kind of performance do we get? It's a little bit faster, especially with bigger blocks. So at the 1k block, it's somewhat similar and then 4k and 16k it's getting it's improving So this is getting interesting right we can see very big increase in the 16k blocks and We're going to see if we can improve on that so We're going to do we're going to try some explicit Java side casting So instead of doing a two Java bytes So we're going to you know use a new method that's called put Ruby string with the buffer with the Ruby Ruby string buffer and use and create a method that accepts a Ruby string object And then on that Ruby string, which is data we're going to do a get byte list so get byte list on the the Ruby string class actually gets us a byte list Class from which we can get the bytes. So there's two way to getting the bytes You can get save bytes by doing a copy or you can use unsafe bytes Which is going to give you a pointer to you know to the actual underlying byte buffer For that string. So this is what we're going to do here So, okay, so this is getting really interesting We can see that we get up to five gig per second with the one k block and and you know seven gig eight gig per second for the 4k and 16k this is very very good So so we see that the you know the cost of the Implicit conversion in jruby when crossing the world between jruby and Java is very very expensive And and there's basically two way to avoid that from the Ruby side or From the Java side. So when you do it from the Java side, of course You have to know a little bit more about the the jruby API in terms of you know and and especially Ruby string I don't know if any of you have checked the Ruby string class implementation It's probably one of the biggest class. Yeah, it's it's just amazing So I won't go into the details, but okay, so now another implementation so instead of doing you know Talking with a plain Java implementation let's try to create a jruby extension in Java and And and so our benchmark is pretty much the same so we're going to use you know put bytes and buffer our Ruby string and then below we can see the Jruby method defined as in Java So there's there's some boiler boilerplate code in there It's basically to check the arguments, but at the arrow we can see that we do a buffer put with The actual so we know that the Ruby object there is a Ruby string so we can do it approximately the same thing is use the byte list and use unsafe bytes to avoid the copying of the bytes and And that's it so Performance for that is somewhat similar to to our explicit Java that we add but for you know a little bit faster for the 1k block So we shave a little bit more time here and If we compare that to instead of using the unsafe bytes, so for me when you do persistence I think it's it's pretty safe to use unsafe bytes because this is usually the end of the story for the string Right, you're not going to mutate the string after that. You're just saving it and you're persisting it But if for whatever reason you you actually want to copy those bytes to get a safe string to persistence Then you can do that instead of using unsafe bytes. You can use get bytes, which does a copy and The difference in performance is this so we see that it's pretty significant in terms of performance cost Okay, so the last implementation is a Is a simply a jruby calling into Java Directly so it's a it's a pure ruby implementation of that M mapping class that we've created you know in Java or as a jruby extension and Basically, so the same put buffer buffer put bytes buffer and then below we can see the implementation So so there's the you know the construction so in Ruby We're calling the the the actual Java class to create the the M mapping and then eventually we have to put bytes with data and then We we do the same, you know data data to two Java bytes and so on The performance that we get with that is you know pretty similar to the to the explicit Ruby performance So not that good okay, so Like I said the the motivation for that was to actually implement a persisting queue or persisting size queue So I'm gonna take a few minutes. So I don't know how we are with time Yeah, okay, so I'm gonna go through those slides so This is a schema of the persistent queues Implementation so there's two implementation. There's the you know a standard queue Just the same same API as the tread queue and then there's a size queue Which are you know blocking and tread safe implementation? So they rely on Page endlers and the page queue implementation. So page queue is simply a non blocking non tread safe base queuing implementation that uses the page handler to do the M mapping creating the pages and and storing the metadata and so on So those page Andrew can have different strategies and there's two strategies So there's one which called the the page cache which caches the the last use pages Because typically you have two active pages when you're doing queuing So there's gonna be the tail page and the head page the head page is where you push and And and these are Append only pages that are created so and then there's a tail one where you actually put the items right and there's the single page strategy Which is useful because in the size queue implementation Typically the number of items, you know in the queue is small So you can actually use the same page memory map and then just do a ring buffer in it So that avoids creating more and more pages, which is costly using a mapping So this is what I just said so typically when when persisting data It's just append only pages and these pages you can define, you know the size that you want I'm doing, you know, and we're gonna see some more benchmarks, but I'm using two gigabyte pages and There's metadata, which is an imab file itself that that keeps pointer to what is the tail, you know the tail Page index and the tail page offset So where do we read in that page and the same for the head for doing the push on the queue? So just a word of caution If you play with that, you know, this is work in progress and you know proof of concept code. So obviously You should be careful Again, there's no serialization involved These tests with the queuing have been done with 1k string objects only the the M map page size is 2 gigabyte we use a two-item page cache and and The for the test we push two million items per producer. So sometimes we have Multi-per-producer, so that's gonna be two million per producer Okay, first the persistent size queue It's a limit limited queue size for this implementation We use a dual queue implementation We push to both the persistent queue and the in-memory queue the in-memory queue is actually just an array So we serialize to push the persistence Persistent queue and we push the original object in memory queue and then when we pop we actually need to only pop from the in-memory So it avoids deserialization cost and we just update the metadata on the persistent queue So the persistent queue is only there if there's a crash and you need to reread The the the queuing to actually not lose data in that case Okay, so what do we get? with page cache We get so we have one consumer one producer one consumer to producer to consumer one producer and two consumer to producer treads To try different strategies. So we pop and push in those queues. So we get Approximately 100 you know ish megabyte per second in terms of throughput with a size queue or You know 100,000 100 you know between 100,000 and 150,000 transaction per seconds So with a single page we get a little bit faster Especially in the one consumer one producer Okay For the the persistent queue implementation. So this is not the size queue. This is a standard queue That's gonna grow indefinitely If if they're the consumer are not catching up And the push and the pop operation are done or persisted We need to serialize on push and we need to deserialize on pop And this is essentially just a thin tread safety wrapper around the page queue implementation Okay, so for read and write we can see that we get a little bit faster Okay, I'm almost done and So for read and write preparation So we have consumer and producer at the same time If we do a write then read then we get a little bit more performance And if we do only write then we can get up to 500 megabyte per second on only writes So just a few notes Do we really need you know dual queue implementation for the size queues it really faster? I don't know I need to test that The the caching strategy is it optimal? Can we find better page size and cache size? How does that perform on spinning this that would be interesting? I haven't tested it Is there faster alternatives to the current page and metadata algorithms that I'm using I don't know Gotta try that And The code has to be reviewed in terms of resiliency, you know doing the force do we need to force maybe at specific points And the last thing, you know the elephant in the room, of course, it's the serialization There's a huge cost associated with that. So Again, you can talk to guy or to set her sheet about that And thank you that's about it