 So I'm Richard Bishop. I'm here to talk to you about concurrency Some of the thing about concurrency in Ruby, but what it is what it isn't and hopefully some things you can use in your applications today a Little bit more About me I'm a self-proclaimed Polyglot with a problem. I love programming languages. I Don't know if it's all right to say this at RubyConf But if you're not looking at another language to play with just to bend your mind in different ways You really should there's a lot of interesting stuff out there right now And a lot of them are dealing with concurrency in interesting ways and that's sort of had an impact on me as a Rubyist And then away from the keyboard I'm a huge Coffee and beer nut and it's great that this is in San Diego because I used to live here We also have the best beer in the world in my opinion So if you're out and you see some local beer try it But don't drink too much because you are at a conference and you don't want to make a fool out of yourself And since this is a conference gotta go shout out to my employer. I'm a freelance programmer We are not hiring So for me learning about concurrency was actually an accident I like to call it the other side of rack most of our time is spent on the app.call side and then Completely down the other side you go into web servers and system calls So one night I had this issue With bind 2 and at the time I'm like, what's bind 2 this looks like a method call I don't have a method called bind right not even realizing what a system call was And I kind of went from there down into these patterns that unicorn and all these web servers implement But it's a lot of fun and Ruby's actually a great language for getting your feet wet in concurrency because we have all the primitives and Obviously, of course, it leaves you wishing you had more and that's sort of the story with Ruby in concurrency But it's a good place to get started So I'm gonna talk about today. We're gonna start with just some concurrency basics Some of you are probably familiar with a lot of this or those might be new to you And we'll look at some practical uses of concurrency and Ruby And a lot of the talks actually gonna be about abstractions because that's really what concurrency is all about as I'll explain in a little bit and we're also gonna talk about Ruby because this is RubyConf and That might sound weird, but probably about 80% of this talk is about concurrency in general And then the other 20% it's gonna be about that as it sort of relates to Ruby specifically But concurrency is a huge huge topic So I have a not agenda as well a lot of talks like concurrency start off with spawning some threads and Then seeing how quickly you can put a million integers into an array That's not really exciting And never got me interested in concurrency and that's parallelism anyway, and we'll talk about those differences later I'm not gonna bash the gill because that gets done a lot It's here deal with it We're mostly I outbound anyway, and I'm not gonna look at a bunch of other Runtimes as Ruby is for pretty spoiled we get to run our code on like five different runtimes But each one of those could be a talk all by themselves So why should we care about concurrency? So we all know right we've if you've heard of the the free lunches over we're in a multi core world We need to get more out of our hardware as Ruby is we love forking. It's really expensive Maybe if you don't get the server bill every month, you don't care, but I Think we should be getting the most out of our hardware Another one is better user experience if you look at highly concurrent parallel runtimes And They're very consistent in the response times. They don't have these crazy 99th percentiles or Long-tailed latencies right that are really important Like we just you can't just look at the request per second or the average response time You have to look at the whole picture And the last one was a huge surprise to me as I learning started to learn more and more about concurrency You actually see some better design we design your programs concurrently so let's get into what concurrency is anyway Oh first, let's just just get this out of the way This is George souk here. I think he probably thinks that concurrency came from aliens But concurrency is really confusing because just to start it gets confused with something related known as parallelism But they're actually quite different But thankfully in the last couple years a good definition his surface comes from Rob Pike He's one of the creators of go and done a lot of great Unix stuff. So this is what Rob has to say about concurrency So he says concurrency is about dealing with lots of things at once parallelism is about doing lots of things at once That's a pretty good definition But if you're like me the first time I saw that now I knew what it wasn't but I didn't I still didn't really know What it was there's not really any context around it. So to look at Make this a little more concrete. I think most of us are application developers So first to look at parallelism If you're on a multi-core computer and you're using threads or lots of processes With several cores Those can actually be all CPU bone up to the number of cores that you have at the same time But the big thing is those are just points of execution. There's no sort of communication happening between those things All right. So like in the case of a web server your Requests as a boundary For concurrency, I think a good example would be batch processing even on something like MRI You could you know fire up 30 psychic processors. They're not all actually using CPU time at the same time But they are running concurrently and if you're batching there can be communication with say there's a parent job They can update with failure or success or when they're all done something else can happen There's a lot of communication And there's multiple points of execution, but they're not has nothing to do with time overlap so The reason there's a lot of focus on concurrency right now is that We want the benefits of parallelism, but we want it in a clear concise way so what concurrency is really about are these two things composition and Communication so it's at the intersection of Performance and clarity and that's why there's tons and tons of different models for how to do it So before I really get into the talk while I was working on this presentation. There is this tweet that came across I'm not really a tweet quota, but I decided to throw this one in here. So it says Programmers love two things making generalizations and dismissing other people's generalizations because they're outliers and accept So the reason I'm sharing this is concurrency is a huge huge talk and 45 minutes. I can't possibly cover everything there is about it. You might have Experiences already or experiences in the future that don't fit in with what I'm going to say But there's just a lot to know and I can't possibly cover it all so with concurrency. There's just Covering the basics. There's three things that were were given by operating systems and by hardware developers Let's processes threads and events These are the bare minimum that you'll have and if we look at these in the context of parallelism If you're on a multicore computer again, not Ruby specific With processes, you'll have CPU and IO parallelism with threads. You'll have the same So events are kernel events. Those are only used for IO. There's no CPU concurrency or parallelism with events So This last column is communication because that's really again what concurrency is about And when you have lots of these running you have to find a way to communicate between them So the processes we do that with inter process communication So I'm the same computer that might be pipes or sockets and message queues whatever it might be But that's the only way to communicate between them With threads we know sharing memory is how we communicate between threads and you know all the perils that sort of come with that And then events there. There just isn't a way to communicate between them. They're just callbacks that you run and As they finish they get completed So with with events they need to be heavily abstracted that's why there's that little no-jask Jad there by themselves. They really really get unwieldy If you look at languages like closure or laying or elixir, there's events there, but you don't even know it It's taken care of for you by the runtime So recap the basics so current concurrency is really about composition and communication So the concurrency is how you structure these programs Parallelism is how they're actually executing at the same time And then we have these three primitives that we deal with and we have these in Ruby, of course But the stock it's about concurrency, but mostly it's about threads. So Whether you like it or not threads have sort of been chosen as the fundamental unit of concurrency We haven't really embraced that so we don't have great support for it in Ruby But I think if we get interested in threads and using them correctly Which actually means not using them at all or not knowing you're using them Then it becomes a lot better So when I think of threading in Concurrency, it's kind of like a lot of other hard problems that we've dealt with So I want to talk about a couple other ones real quick so the big thing is something has to be hard to do but Important enough to want to do it and if that's the case We'll take care of it. So the first one is memory management All right, we're all familiar Especially this year with things like Heartbleed how hard it is to manually manage memory But that was abstracted a long long time ago, right? Lisp was the first garbage collected language Ruby's garbage collected, of course We work it, you know a lot of the work we do is in garbage collected languages Or there's other languages to like rust that have an interesting ownership model for dealing with memory But the important thing is we have to deal with memory It's not important to know what you know to be doing it yourself, but to know how it works at least And of course there's sequel right who doesn't love a good sequel injection every now and then But you know in Ruby, right? There's query generators. We have error all to take care of that for us And finally threading is hard Some languages have great abstractions for that Ruby doesn't all right If you're to compare it to memory management, I think thread and mutex are kind of like malloc and free All right, we should know that they're they're being used somewhere, but we shouldn't be dealing with that on a daily basis But we love we love processes in Ruby land right we love forking you know using all that RAM but it's expensive and Communications also very hard between lots of processes. I like to call it Passive aggressive communication you have one process Send something to a message queue and it doesn't know what other process is going to get that message like they're not actually communicating directly and If you're not communicating directly you can't create really robust composable things But there's actually some libraries we can use today. Thanks to our community And I want to just look at a few of those real quick and then hopefully get you interested in using them So the first one I want to look at is called celluloid So this is created by Tony or Sierra. It's a really really great library If you're familiar with the actor model That's popular in languages like Erlang and elixir. The idea is you have these this concurrency primitive They're executing concurrently and they communicate using messages So that's kind of how we view Ruby some people say that methods are messages We'll see how that's not really true in a little bit But celluloid takes that actor model and kind of puts it on to objects and combines it into one really nice library This is actually this has got to be the most downloaded gem that nobody has any clue that they're using Because it's used by some other really really great Libraries, and I think that speaks to how good of an abstraction this library actually is It has some other really nice built-in things, too You can create pools of these actor objects to do work and then you can also create supervisors Which are a really nice tool for fault tolerance and reliability So for you to look at oops an Actor it kind of looks like this and celluloid you have this actor object inside of it There's a mailbox and then in that mailbox our messages so the mailbox is really just a really fancy queue and then as Messages come off the queue the actor processes them and moves on and what's inside them the the message is just in celluloid is the method that you want to call and The arguments that you want to give to that method So here's a really Contrived example of using celluloid. It's a simple mix in After that you can instantiate object, but as you see you're not actually getting back Your class that you define you're getting back it wrapped Inside of the cell proxy And the way that you communicate is again by doing messages and you want to do a lot of this asynchronous That's why you're using an abstraction like actors So you call async and you get back this async proxy And then you can call greet on it So, but I said just before that method calls aren't messages and I think it's a distinction that needs to be made So if you look at a method call you have to wait for it. It's synchronous So there is a way to get around that right you could do some sort of a thread.new a fire and forget kind of manner But for a lot of reasons that's not something you want to be doing in your code but a big thing behind messages in The actor model is that they're asynchronous? You can make them synchronous by requiring that you get a reply back after you send a message But you start off async and it's only sync if you want to reply So how does celluloid actually do this? The way is through metaprogramming So what you do what happens when you call async and then you give it a method Async proxy implements this method missing and it wraps it in a call and forwards the method and the arguments on so that your program can move on and then That will eventually get called later So the reason I wanted to talk about this library first is the next one that I wanted to show you uses it and I hope Everyone in here is using this library by now Sidekick it's tremendous If you were previously using rescue or still are using rescue or delayed job, I'd encourage you to check it out So this is made by Mike perheim. It's a fantastic background processing library you can a typical Ruby application one of these workers can replace between 8 to 20 in my experience rescue workers and The one weird trick it has for doing this is it's multi-threaded So even an MRI threads are still useful Because you have IO parallelism or concurrency rather If you look at the design of sidekick though again, it's using cellulite behind the scenes. It looks like this So you have this launcher object when it gets started up you get a fetcher and a manager and Then the manager manages these processors, which are the things that actually Do the work of instantiating your jobs and running the code The interesting thing about this There's two things in my opinion. There's really clear visible separation of concerns when you look at these pieces and They're all running concurrently and that's because they're all actors and they're all communicating with each other using messages So the manager can send a message to the processor of a job to work on The fetcher sends that job to the manager or the manager can request from the fetcher also So this is a really great library. I'm just curious who in here is using sidekick That's that's good. That's that's pretty good It's a surprise almost So Ruby's also really popular among startups and I know startups you're all super super busy You don't even have time for stuff like concurrency But there's another really cool background job called sucker punch This is made by Brandon Hillkart also using celluloid. It's completely in memory Same API to sidekick. So if you need to upgrade later, it's very very simple to do and You don't want to use sucker punch for obviously very very important jobs, right? Because it's in memory if your process crashes Your cue is gone, but for prototyping and things like that. It's awesome The last library I want to show you a quick is called Puma. So this is a multi-threaded web server it actually does forking as well and It has a reactor inside of it. So this is like a it's a concurrency buffet. It's got it all But to look at the design of it again You kind of see these clear separation of concerns on the right You have what's known as a thread pool a very very common concurrency abstraction It's basically you'll you'll have a bunch of threads all sharing a resource and they use mutex Which is around this request queue to change that resource And then on the left we have this acceptor thread Which is just a TCP server Accepting requests and then handing them off to this this request queue that I think it's processed by these threads so again very very clear separation of concerns and That's what makes doing threaded like this possible If you don't create good abstractions for threading, you're gonna have a really bad time And another thing to keep in mind when you're doing these types of things This you have to keep an eye for for potential bottlenecks in this case. That's probably the mutex around this queue But realistically not a huge deal because this is the real bottleneck in a language like Ruby I think we can agree So to recap just this section real quick. This is all about abstractions to see actors and thread pools And I just wanted to show you some libraries that you can use right now or should be using right now to To get better use out of your hardware The next section is the one that's really really interesting to me As I've learned more and more about concurrency and have used some some other programming languages That's the design benefits that you get out of designing your programs concurrently so Right now there's or as always I guess there's a huge emphasis on design Right here so much talk about designing in the small or service-oriented architecture Service objects micro services, whatever. There's probably tons of talks here about software design So what design is really about is pain When you're working on a single threaded synchronous app and you have poorly designed software the pain that you feel is unexpected bugs it's hard to make changes and This iterating becomes a bit slower We don't actually have runtime pain in a single threaded synchronous application so As opposed to a concurrent app if you have bad design not only do you have all those other issues But you're gonna have issues with deadlock race conditions and other bad things So be knowing this when you design an application for concurrency You're all a bit more brutal about how you design it. So I want to quickly talk about another language real quick I couldn't possibly introduce you to a language right now, but just some High-level over things. It's called elixir. If you're familiar with Jose Valin, this is a language he created it runs on the Erlang virtual machine If you've ever met an Erlang programmer before they've probably told you how awesome the Erlang virtual machine is within the first five minutes of meeting them If you haven't now, you know me and the Erlang virtual machine is awesome Benjamin Tan why how is giving a talk tomorrow on elixir if you're if you're more interested in this language? But the interesting parts about elixir and Erlang for that matter is they actually treat concurrency as a paradigm all on its own So we think of object-oriented or functional programming as paradigms and they they think of concurrency as a paradigm And you really see that in the language and it's also using actors and message-passing between them When you look at an elixir program it has this sort of structure to it it's known as a supervision tree and All these individual circles are actually running concurrently and they all communicate using message-passing and the red ones are known as supervisors So there's a big distinction in your applications when you write them you have your Your library code, maybe your models your parsing or whatever it is and then you have these implementations of these concurrently running actors and These can be pools each one of those circles could be you know in the case of a web server That might be like a hundred acceptor sockets or something On a pretty normal for a core computer you can have 300,000 to a million of these actors running at the same time. It's really insane As great as the Erlang virtual machine is they're actually world-class at having poorly named things So this little structure here is called an application But they're actually it's actually just a component It's one piece of your application your whole application will look like this One part might be something that's accepting web requests you could have web sockets for messaging and then The Erlang world is nuts. We like to write our own caches It's just really really easy to do in the language so you can have these really really application specific caches. It's pretty neat The interesting thing about this though is it pertains to software design is all of these are deployed individually So something that you hear about a lot in the rails world is like monolith right everybody hates the right the monorail And because when you deploy you deploy the entire code base and you just have to keep doing that every every time right? This is different you can you can configure Create these releases and if you see a that's okay. Our cache is really really getting hit We need to scale this up you can either one you can run more actor They're called processes again Erlang is really confusing their their actors are called processes But the big thing is you have a ton of granularity you can scale all these individually up and down you can Deploy a new node just completely without some of these right it might just be the cache for example But when you look at a typical Ruby application looks like this All right, it's like here's our web server like bow down to the web server right and everything else So We think that single threaded synchronous code is very very simple, but it's a trade-off that we're make just like any other trade-off And so we have these applications in Ruby that have this like crazy distinction This is part of the web and this is worker or everything else So our codes more simple, but our deployments are made more complex because we now have different types of processes that we have to deploy Different probably different commands to run them Maybe different deployment procedures all together So it's just trade-offs. We just I think in Ruby. We've just decided Right now that we're more comfortable making these single threaded trade-offs Than we are the multi-threaded trade-offs, but if you do embrace multi-threading and you do get concurrency You also get positive trade-offs as well in terms of the granularity over scaling So to go back to this elixir design again These are all executing concurrently. They're all in one code base Can be scaled individually So it's very very modular and that's what we want in our design And that's what concurrent design really really gives you I feel and I know this right now And Ruby we're trying to fake this kind of design And it's called this So We look we want those things right we want modularity. We want decoupled things and we want to scale them individually Like 90% of the time I talked to someone about service-oriented architecture It's because they've got some Ruby application. It's deployed 40 times But it could probably only be deployed 30 times if one piece was deployed 25 times and then the others were for five So like okay, well, we got to do services. We got to start doing this So you end up with things that look like this So now you've added a message queue to the mix once you've figured out what message queue you want And then probably misuse it or get it wrong anyway And then your message is also communicate via HTTP Which is really really weird because you wanted to be decoupled But now you're communicating between your applications using a synchronous request reply model So you get this like repo oriented architecture. It's like really what you end up at the end of the day Which is either even worse because I don't know maybe it's just me But I can't even like if I have an API in one Repository and I have a client for that API and another repository like I have a hard enough time keeping those things in sync I got last thing I want is five repos that I got to try and make sure they're all on the same contract So that's the way the whole selling point of the motivation is that you get these force boundaries and you're decoupled And you can scale things individually But if you use something like celluloid or a language like elixir If you use concurrency you can do those things, right? It's just like With elixir, it's like how spicy do I want this one module like on a scale of one to ten? And that's how you you can have hundreds to thousands of actors just like that the other problem with this idea is that Going back to Processes in concurrency the only way you can communicate between processes is by using queues So now you're doing a ton of work over the network instead of doing that in process Which is a lot faster and more reliable And then to go back to the decoupling you're not really decoupled if you're using HTTP For say your user service to talk to your product service, right? If one of those goes down and the other one needs data from it You're you're still very very coupled, but now you're coupled over the network instead of over the code Which is probably worse So to recap design Concurrent design you're forced into modularity from get go Because you this is all executed concurrently if you don't design it while you're gonna have issues and Usually looking for synchronous communication and elixir app is the first place you start when you are having issues You can scale everything independently Which is why we like SOA so much And messaging is really really hard so if you can do it in your programming language in memory it's a lot easier and Then the last thing is that it's just programming when you write an application in elixir early, and you don't think oh This is web, and then this is is everything else web. It's just one little piece of your application and finally I Wanted to stick up for ruby a little bit as it relates to concurrency Because I haven't really picked on the gillet all yet, and I won't Jay ruby is actually pretty awesome. There's a talk on it again this year. It's there if the gill is that big of an issue for you But to understand ruby you have to go back to the 90s And I wasn't around and while I was around in the 90s, but I wasn't you know Using ruby in the 90s So let's go back to the 90s. I think people from Portland and Seattle really like the 90s But I want to look at the 1990s programming language design This is all speculation again. I wasn't around but if you look at the languages from this period They all have these things kind of in common So when you're designing a language, it's almost like you start off picking a paradigm the time it was probably object-oriented and We've we chose or you know maths chose object-oriented and thankfully he chose a Good version of object orientation because there's plenty of languages that have really bad implementations of it Now you decide. All right, we're gonna be static. We're gonna be dynamic. I was seemed like really important distinction back then And then somewhere along the line you slap some POSIC threads on which And that's your concurrency, right? And ruby didn't even do that from day one that we had that green threading implementation But the big point is that when ruby was designed Currency wasn't even anybody's radar yet probably So if we flash forward to 2000s 2010s we look at some newer languages. Oh Yeah, it's still the same, huh? So style hasn't changed, but I think programming language design has So if you look at languages now it goes like this step one choose a concurrency model Some languages maybe didn't do this, but they probably picked some other features that leave this door open for them Like immutability and then from that point on it's completely about concurrency You don't want to put something into the language that doesn't support your one of your key design decisions And Ruby I feel like we tend to think like concurrency and parallelism can work like this Like like maths is just holding out on us and Ruby cores holding out on us Come on flip the switch What doesn't really work like that though because you need a few other things So number one is you need a good garbage collector So all of a sudden if you become parallel and concurrent If you have a really crummy garbage collector, you're just gonna be stopping the world and you're gonna have a bad time anyway So what was the point? And you can see right now in Ruby. There's a ton of work being put into the garbage collector, right? 2.1 big change 2.2 again And secondly you need abstractions Everything I've shown you so far is completely Community-driven as far as abstractions Just actors and thread pools. We don't have Anything in the core or standard library. This is it threads, mutexes, condition variables. That's the bearer minimum I mean it's not often that you're using Ruby and you look at a language like Java and you have Envy But when you look at util.concurrent as a Rubyist you feel a little bit of Envy So the other problem or reason why I think Ruby's the way it is right now is that we love simplicity so We're all using this language because it makes our lives easier or makes programming fun and previous languages didn't do that for us The problem is a lot of what's happening in the currency right now It's pretty cutting edge. It's from papers from the 70s and 80s But a lot of those papers were missing implementation details anyway, right? If you look at Look at a language like go and you look at the core team They have probably about a century of Unix programming experience between them. If you look at their websites they have I think between five and six implementations of CSP that Didn't work so they finally landed on one that works. These are really really talented people So the cutting edge is never simple, right? We can't expect Ruby to have a fantastic concurrency model right now When a lot of other languages have gotten it wrong and we look at Node.js That they still want to use an event loop for currency So you have to you have to watch all these other languages get it wrong first and then you can finally figure out how to get it Right and I think Matt's is finally starting to see that All right, so he I don't know why I'm quoting another tweet probably look like a tweet quarter But he's saying he wants to add an abstract concurrency unit such as actors and Then stop using threads or warn you or raise an exception when you use threads directly and then remove the gill already So Matt's is giving us something to look forward to we just have to wait for it And I think whenever it comes it'll it'll be the right abstraction. It'll be simple That's all I got. Thanks