 Hey, welcome. My name is Thijs. If you want to join, I'm doing a little demo later on, so if you want to join that and also, like, look at the code I'll be talking about, please clone this repository. So today I'm going to talk about how to do concurrency in Ruby, using just very simple stuff from the standard library. So I work on a monitoring product for Ruby called AppSignal. And we support a lot of different types of web servers and tools, which kind of forced me to learn about all the different ways you can do concurrency in Ruby. And then I realized that years before, this always sort of sounded like an intimidating subject, and I felt like I didn't really understand what was going on, but it was actually quite a bit easier than I thought. So today I'm here to share some of these insights with you. So in general, there's a few exceptions to this, but for simplicity's sake, we're going to be talking about these three main ways of doing concurrency. So you can either run multiple processes, you can run multiple threads, or you can run an event loop and have that kind of fake concurrency in a sense. So you're probably familiar with these three web servers. So they all use one of these models with their open downsides. So we're going to try and do this, discuss this subject by building a very simple chat server that's kind of like Slack. So I implemented a little chat server in Ruby in these three different ways. So my colleague Roy over there is already logged into it, hopefully. Well, of course, it's not working. Hi, Roy. Ah, it's working. So, well, here's our very minimalistic Slack. I'm afraid we won't be getting millions of VC funding for this, but at least it worked. So we'll start with discussing the chat client. So this file is called client.rb if you checked out the repository. And this uses the just basic networking stuff from the Ruby standard library to make a connection. So it starts by requiring socket, which brings in all this network logic. And then it opens a TCP connection to a certain address. And then it boots up a little thread that just asks the client for incoming data. So basically anything the server sends back to the client will get written out to the command line. And finally, it just listens on the command line. That's this scdin.get. Basically, just wait for you to type something, press enter, and then it will trigger that loop. And then it puts that on the client. So, and that means that it gets written back to the server. So the server receives this data and the server can write data back to the client. And the client is able to either get user input from you or write stuff that the server wrote back to the command line. And basically this is a full chat client. I'm just sorry to say it doesn't support any animated GIFs. So as discussed, there's three ways to do this in Ruby. And the first and most simple way in a sense is to use multiple processes. So this is how Unicorn works. What happens is that there's one master process that gets started by your system. And whenever some work needs to be done, that forks into a child process. And that child process can do some work and then might get killed again or will live for a little bit longer. And this worker process does the actual work. And the master process kind of manages these child processes. And if you would look at this on your activity monitor on your Mac or on top on the server, it would look something like this. So you've got a master and a few Unicorn workers. And you could actually kill one of these workers or just let it crash. And then the master process will make sure that a new one gets spawned. So this is pretty resilient architecture. So what does this look like in Ruby code? We'll first start with actually starting the server. So this piece of code is the same for all the examples. Basically, we started TCP server on a certain port. And that from that moment on, it just listened for new incoming connections. And since we're using multiple processes here, these processes have a completely different namespace. So anytime you, if you modify some variable in one of the processes, this won't influence the other ones at all because they're actually completely isolated by the operating system. So that's why we need a way to communicate between those. Because if we receive some chat message on a process that's handling one connection, we need to be able to write it to all these other connections that are actually different processes. So I've simplified this a little bit. You can see the full code in the examples. What it comes down to is that we use a pipe. You might have seen that on a command line. If you, you can use this to rep through a stuff, for example. What happens is that a pipe is just a stream of data from one process to the other. So if you open a pipe and you write from process one, the data will be listenable from process two. So we just set up this communication. The details are in the examples. And that will magically make sure that all the other processes also get all the chat messages. And then we get to the management part of how this works. So this is very similar to what Unicorn would do. So we start a loop. And we try to accept a new connection from the server. So server.except just waits for somebody to connect. And whenever this happens, it's the next iteration of the loop. Then we set up a pipe so we can write between the new process and the master. And we add this pipe to the list of processes in the master to make sure that we can also write back to the child process. And then there's a little magical word that does a lot of stuff which is fork. And what fork does is it just makes a complete copy of the process exactly as it is at that moment. So basically the moment you call fork, anything that happens within the do and block is the new process. And anything behind it is the old process. So the old one is still there. And there's a new one which is just a complete clone which starts doing work in like the moment do ends. Like anything inside of do and end. Which this is actually quite a hard concept to wrap your head around. I think you really have to try this out a few times on your own machine to really get it. I know I didn't really get any of the explanations. I really had to see it for myself to be able to understand it. So now what happened is we have a child process. And this child process is aware of which stream it's connected to. So next up, we can actually do some chat logic. So the first thing we do is read the first line from the socket and just assume that's the nickname. That's kind of the protocol of our chat server. The client just writes out your nickname as the first line. And then we just write back a little message back to the client. And then what happens is this is not line 31 is again simplified. This starts with a little thread that writes incoming messages back to the client. So I'm sorry to say we do actually need a thread in a multi-process example to make it all work. Otherwise, you wouldn't be able to really implement the whole thing. And then it just basically waits for you to type something. So there's a while loop at the bottom. It tries to read a line of text from the socket. And writes back this line of text back to the pipe. And the pipe writes it back to the master process. So it's quite a few moving parts. So in this case, doing it with multi-process is actually a bit more code than the other versions. Sorry, I skipped one thing. So what happens next is that the master process can write this text message back to all the children. And the children can write it back to your terminal. And we will see how this works in the demo, like how this operates in reality in the demo at the end of the talk. So there's a few good things about multi-process concurrency. So one thing is that you can basically forget that such a thing as concurrency exists because anything that happens in the process is just executed in single thread. And there's no way, like, any thread safety issues can arise. And the next good thing is that workers can crash. So, for example, GitHub really likes this model. So both Unicorn and Rescue, which were written by them, use this model because they do a lot of call-outs to the Git command on the command line. And that has a tendency to use a lot of memory in crash. So they would have issues with a threaded model because the thread could bring down the entire process. And in this case, the master will just reboot anything that crashes. And the downside is that it uses a lot of resources. So any time you want to do anything that happens at the same time, you need multiple processes which use memory all over the place. So it's actually a very poor choice for a chat server. But it does work, as we'll see in a bit, which brings us to the next model, which is multi-threading, which makes a lot more sense for a chat application, actually. What happens here is that you have a single process. And within this process, you can boot threads that do work. But they still share all the memory. So if you mutate something in memory in one thread, it will also be different in another thread. And that looks something like this. So, again, we have exactly the same TCP server that gets opened. But then things are a bit different. So what we do here is we're basically using this messages array as a database. So anytime a new message comes in, we just put it into this array so we can store it and send it to other people. But if multiple threads would be reading and writing to this array at the same time, then it might actually end up coming into an inconsistent state. Because one could be reading stuff while at the meantime another one is inserting stuff. And maybe then a message wouldn't be written to our clients, for example. And that's why we need a mutex. So a mutex is basically like a traffic light. So a thread can lock a mutex. And basically tell a mutex, I'm working with this data at the moment. And then when it releases the lock, then another thread can lock it and also work with the data. But they can't do it at the same time. So this enforces that your data stays in a consistent state. The downside, of course, is that if you have a lot of locking, then the whole thing might actually be just as slow as a single process application. Because if all the threads kind of like end up just doing work one by one instead of concurrently, then you still don't have a concurrent system. So if you hear what we're locking, also in a database context, it's kind of like how this works too. So next up, we do the same server.except call. So again, we're waiting for somebody connected to server. But instead of forking, we're actually starting a thread. So anything that happens within the do and the end block is running in a separate thread within the same process, which is running sort of independently of the other threads. So again, we're reading the nickname from the socket and we're writing something back to the socket. And here it's slightly different. So instead of having to set up all these pipes, we run a little thread again that just sends incoming messages back to the client and reads messages from the client. If you look in the examples, you will see the implementation of these two methods. They're also for simplicity sake. I didn't add that here. And when a new message comes in, then we basically just push it onto this messages array, which is kind of like our mock database. So we call mutex.synchronize, which locks the mutex and makes sure that only R thread is currently doing anything with this messages array. And then we just push a new message onto it. So in reality, you will probably store this in Redis or whatever to make sure it will survive a crash of the process. And next up, we write these messages back to every client every 200 milliseconds. So again, we have to log and then it collects all the messages that have to be sent. So we're storing a sent until timestamp. Yeah. Yeah, for sure. Yeah. So that's a good question. A mutex can deadlock. So if you, it basically means that if you lock something, but never release the lock, or you lock something and then you lock something else, and these two locks are kind of like waiting for each other, then basically your process will never continue doing any work. So this is also a risk of using a mutex. And these kind of issues are the reason why people in general say that the programming using threads is pretty hard because you have to be aware of all these risks and and and not do any stupid stuff. But it's very easy to do the stupid stuff. So yeah, that's kind of a problem. So we got the messages we want to send and we just write them out to the socket with the soccer dot pooch line. And we sleep for a little bit. So this is kind of you can already sort of see the Achilles heel of the system. Maybe because we have to lock. We have to lock the messages are a all the time. So if we have a lot of throughput and probably the percentage of time that the whole that messages are a is locked will get higher and higher and maybe at some point it will be so high that we won't actually be able to send out messages as fast as they come in. But I don't think at the moment we're probably not not going to reset limits. I don't see that many people with a laptop app. And it will look something like this in your process manager. So there's a single process with a single process ID. And it's just running a few threads. So there's there's another thing to think about when you use threads and Ruby, which is the global interpreter lock. And the global interpreter lock is this thing that's specific to Ruby. It's kind of like it's, it's kind of a relic from the past, which is still at the moment still in the MRI version is still present. So in Ruby, I rely on Ruby code cannot be executed in multiple threads at the same time. So for example, if you run a few threads, and you would, you would, you would store stuff in in hash, then, or do some calculations in different threads, then actually they're just running a one by one instead of instead of concurrently. And the only exception to that is IO. So if you, if you write to a socket or you write to disk or you read to disk from disk, then, then, then the lock actually won't be active. And this is the reason why why using threads in Ruby is often useful, because especially in like a web context, or you do networking, of course, most time is spent doing a call to debate database and getting it back. And in the meantime, Ruby can do other stuff. So in reality, it's still quite useful, even though there is this lock present. And another thing is threat safety, which we already discussed a little bit just now, there's a risk of deadlocks you, if you're not careful, then you can mutate things at the same time, or, or, and then your counters are totally off, for example. So it's, it's not really for the faint of heart to, in Ruby, at least to, to, to do this you, you do really have to know what you're doing. So the positive side of this is that it uses a lot of memory per connection than multiprocess. And you can share data easily. So you don't need to set up all this, this communication between the processes. You can basically just have an array that's stored in central location. And the whole thing will just work. Well, and you do have to make sure that your code is thread safe. And you, and it doesn't make any sense for CPU intensive operations, since these will only, will only run in a single thread at the same time anyway, which is rare in, in web plans. So it's usually not that much of an issue. And I skipped one. So if, if a thread crashes, there is a possibility that the whole process crashes. And then basically the whole thing is gone, because there's not, there's no master process making sure that it gets started again. And then we get to the, to the final way of the concurrency ruby, which is an event loop. And the funny thing about an event loop is that it's, it's not actually concurrent. It's, it's the trick about it is that it's so fine grained that it's kind of like a magic trick. It, it seems to be very concurrent, but actually it's only doing one thing at the same time. And we'll, we'll get to that in a bit. And it uses very little memory per, per connection. So it's, it's a good choice for something like a jet client in general. And an event loop looks, looks something like this. So it all starts with the operating system. So the operating system, of course, knows like what kind of network connections are open, and what's going on. And you can ask the operating system to, to basically tell you when, when something happened. So this is called registry interest. So you can, for example, you can tell the leadership colonel, please inform me when this connection is ready for reading. And then it can just ping back to your Ruby code and let you know. Yeah. So the operating system tells us like this, this thing you're interested in is ready. And that gets pushed onto an event queue. So there's this list of things that are happening. And you can just, and you can just loop through them. So basically, the event loop is doing nothing more than, and just endlessly going over this, this list of events that's in the queue, and, and just doing stuff with every single one of them. Usually it has some kind of storage. So, so, so in the context of a jet client, you want to know the nickname of the person that's, that's, that's, that's, that's connected to a certain stream. Because when a message comes in, you want to be able to like know what this person is. So you can write, write the nickname to back out there. And the event loop can often also add an event. And it can, can ask the kernel to, to, to tell you when something happens. So basically, this is a single process usually in a single thread. And it's just endlessly spinning around and waiting for stuff to happen. And it reacts to something and it writes something to a stream. And then it just goes on to the next take of the loop. So in, so you need quite tight operating system integration to make this work. And Ruby does a gem called event machine that offers this integration. So you, if you would actually do something in production, you would need something like that. At least maybe we'll end up in Ruby itself at some point too. I hear they're thinking about doing like a more, maybe like an active based concurrency model. So then, then stuff like this will be easier. But for now, I kind of cheated. And I made a, I made a kind of like a, a not so nice event loop, but it's at least it's simple. So what we're doing is we're using a fiber and an IO.select. IO.select is a, is a function that's in the Ruby standard library. And you can pass a list of, of stock, of IO descriptors into it, like a socket or something on a file system. And you can ask, ask it, please inform me when one of these, these sockets is ready for reading or writing. We'll get to that in a bit. But first fibers. So fiber is a new, is a new concurrency construct that was introduced in Ruby 1.9, I think. It's kind of like a thread, only much more lightweight. So it has a very small stack of, at the moment, 32 kilobytes per fiber. And it, it operates like, like a thread. But you can pass it and resume it at any time you like. So in this example, we have a little fiber that's, that's looping around. And it, it calls fiber.yield. And fiber.yield is, is, is basically passes until resume is called on the fiber. And whenever resume is called, then the yield call continues. And, and fiber does whatever work. So in this case, the console output would be one, two, three, four. Because we're just asking the fiber to, to put this back to the console. So fiber is kind of like, like a, like a go routine, which only, only go interpreter schedules to go routine itself. And in Ruby fibers, you have to be able, you have to do this yourself. So if you don't resume, then this, this fiber will just endlessly be passed. So again, we have these, the same example, the same TCP server opening, a few chains there. And then we have a list of clients and a list of messages. And we don't need to mute, mutex here, because it's, it's just a single thread. There's, there's no actual concurrency going on. It's, it's just faking it basically. And in the clients hash will store some, some metadata will store a fiber for every connection. So this is basically what our, our, our chat server client representation is. So anytime new connection gets opened, we, we start a fiber that's just endlessly looping and it waits for, for itself to either become readable or writeable. So the event loop will tell the fiber, this, you're not readable and you can do some work and, and then it can actually do the work. I get the sense that this is a little bit confusing. So let me think how to maybe explain it a bit better. I will, we'll just see on the next slide what happens. Hopefully it will be a bit clearer then. So when the fiber is, is in a readable state, it again can read some data from the socket and push this onto the list of messages. And when it's in a writeable state, it again, it, it gets the messages have to be written from the, from the list of messages and writes them back to the client. And then it stores the last write timestamp. So, so it knows that next time around, it doesn't have to send the same messages yet again. And then we get to the actual event loop. So, so this is a fully functioning event loop. It, it just loops endlessly and it starts by trying to see if there are any new connections and, and storing these in the list of clients. Then it tries to ask the operating system, do you have any connections that are, that are ready for reading and writing? And then it reads from the readable connections and it writes from the writeable connections. So we'll, we'll go over all four of these in a bit more detail now. In this case, it's, it's, it's again calling server.except. Only this time we used the non-block version. And the only difference is that server.except just waits for new connections and non-block immediately returns. So it just tries to get a, get a connection. If there's no such thing, it just continues. And we will get a, a, and it will raise an exception. So if there's an exception here, we can just continue with the loop. Yeah, then we get to the next step. We just ask the operating system, please tell us which one of these, these are writeable or readable. Like in a real event loop, this would, this, you would do this in a more scalable way. So this is, this is kind of, this is really a poor man's version of how to do that. And we have sort of the same code again as we've seen earlier. It, it pushes a message back onto, onto the messages list. And a writeable code is also sort of the same. So the, the upside of an event loop system is that it has a very, very low overhead per connection. And the scale's up to a huge number of parallel connections for this, this reason. Downside is that you probably already noticed if you ever used JavaScript, but if you get a more complex event loop system, you often end up with something like callbacks to be able to manage everything. And then the whole thing can get very hard to do because everything is calling each other and that's a huge stack. And the finally, very importantly, since it's a single thread in a single process, like if, if, if you, if it stops, then the whole thing is, is stops. So for example, if we go back to this, this example, if, if the reading in this case would just, would just take a very long time for some reason, like nothing would be read from any client at all. So you do need to have a workload that, that is suitable to, like to be in cut up on very small pieces. So which one to use? Well, as always, the answer is it depends. So if you, if, if, if you have stuff that can crash, then the multiprocessor approach is very good. Multi-training is a nice one because it's relatively simple and you don't need to convert your whole code to using event based model. And the event loop is nice if you need a lot of concurrency. So let's, let's try it out on my laptop and see, see how this chat server actually works. So, maybe you already checked out the example code. I know Roy has, so at least I can chat with Roy. You can connect to my laptop by, by running a command below. If you checked it out at the absolute beginning of, before the presentation, please pull because I fixed the bug. Did everybody who wants to get that? I should get a different first name. So there's already some people in, in our Slack. So I'm currently running the evented version. So if you look at, if you look at the Ruby processes running in my, in my machine, you can see this had left bottom side of, of the terminal. There's just a client which is running on the right side and the server which is running on the left side. So if we, if we inspect the server a bit more, and this is a list of all the threads that are active in this process. So you guys are a bit slow. I did the stock in, in Belarus last week and they, they like hacked this whole thing within, within five minutes. So I am a bit disappointed. Okay, so we, so this is the evented version. And as you can see, there's only, there's only one thread running in this process at the moment. So let's, let's move over to the, to the threaded version. Sorry guys, you'll have to reconnect actually once I restart. So here again, on the left side, there's a list of threads. And you can see that, that is just boots up a thread for, for every incoming connection. I've, I've tried to measure the difference between the performance of the event and the threaded version, but that's, that's kind of negligible. And I think that's probably because I'm using IO dot select instead of an actual proper event system. So we can't really see the difference in any resource usage here, unfortunately. Let me connect in. Okay. So finally, we'll, we'll start the multiprocess version. And this is the one that will break the easiest. That's why I'm doing it at doing it. That's the last one, because if my laptop crashes, I'm like, the presentation is done. So who cares? So this is a different, this is the ps3, tree command, which shows you a tree of all processes and their children. It's, you can see here that that at the top, there's the master process. And then like one step nested into that, we see a bunch of sort of child processes. And well, somebody knows how to write a loop. Yeah, let me just see if how many process we have right now. Oh, so there are about 10 people logged into the server at the moment. Yeah. And this, this, this concludes my presentation. So, so the question is, how did that, how did I apply this month, this knowledge? And so I work a lot on a, on a gem for, for Ruby and Rails, which is called app signal. And it's, it's a monitoring gem. So basically it hooks into into the web server and fetch gets a lot of information and processes that send, sends it back to us. So basically, I've been debugging everybody's weird bugs for, for, for, for more than a year and which forced, forced me to learn this. Well, thank you.