 Yeah, that's in yeah, is there power that would do it ah There we go Alright, so thanks everybody for coming out. So my talk today is async everything just real quick During the day. I'm a mild mannered programmer who works at living social writing big code at night I fight the scourge of domain bad domain and DNS providers out there under DN simple So that's what I do But I'm also a customer I use a lot of software obviously I think we all do we're all customers and I built a lot of web software so the web area is where my main focus is most of my day and As a customer one thing that's really important to me is that stuff works fairly fast I don't want to spend a lot of time waiting for a web page And the amount of time that I'm willing to wait has been going down to less and less every day So if it's taking more than a couple seconds, I get frustrated now It used to be that the the the goal was eight second response times I can't even remember a site that I could use that had eight second response times these days. It's just non-existent so latency costs money in 2006 I believe it was Amazon stated that for every 100 milliseconds of latency. They lost 1% of their revenue All right, so if you're doing a hundred million in revenue a year, that's a lot of money to lose for a hundred milliseconds There's also Google also came out and said that a 20% increase or an increase in latency of of I don't know It was a like a second or a half a second or something like reduced traffic or reduced responses by 20% people clicking on ads and things like that So clearly there's actually a financial cost with latency So what are some of the causes of latency? Well There's actually a study that basically they looked at for web applications What was the major cost of latency and it comes down to the amount of time for the infrastructure? So that's actually the hardware response time It's the amount of time that it takes to run the process on the server So this is a time where it's sitting inside your application doing stuff the amount of time that it takes to go over the Networks is the transmit time and then the amount of time spent on the client side actually rendering the thing All right, and in that study they said that only a tiny portion of the delay is actually on the server Except when it isn't that's what I added because in fact when you're building applications these days I don't know about your guys applications But I spent a lot of time integrating with other services like a lot of what I do is calling out to other web services to have them do something Whether it be billing or maybe a service that I depend on that's not core to my business But that's that I can use to to make a little bit more money whatever it might be or to have a better customer experience and So it's actually compounded because it's not just my latency I'm dealing with but it's actually the latency of the services. I have to connect to as well All right So there there's no doubt that a lot of time is spent just delivering over the network and when I say a lot It's all relative You know when you're talking hundreds of milliseconds, that's a long time if you're having it your your packets are traversing for example The Atlantic for services are here in the u.s. They're being serviced in Europe or even further about so So what are some of the ways of dealing with this latency? Well I'm gonna first of all I'm gonna tell you give you a few steps that you can use to Generically deal with well any sort of problem. That's a performance issue So dealing with latency problem. The first thing that you have to do is measure All right This is the key thing if you go ahead and think that you're gonna solve the problem without measuring where the latency actually exists first Then you're probably gonna end up with a solution that doesn't really solve the latency problem All right So the first thing to do is measure the latency and there's plenty of tools out there I mean you guys all know the tools in Ruby land you've got things like New Relic You have external tools that can they can basically ping your system from all over the world to determine what the response times are So there's plenty of tools out there. The tools are not important, but the measuring the active measuring is extremely important The second step is you actually have to think about it. Okay, so you've measured and you see that there's a oh crap There's latency here. Oh quick. Let's go fix it this way. No, the first thing is to take us to slow down Take a breath and go. I wonder what's actually happening here. Let's try to reason about it So after you spend some time reason about it Then you can actually build a solution or or find it purchase a solution or or change something to solve the problem And after that you verify it With more measuring All right, and you do it again and again and again because chances are once you solve one Latency issue. Well, then you're gonna have another one and another one They're gonna get hopefully if you're doing things. Well, they're gonna actually be small the latency times are gonna come down But just don't forget that as you build more stuff on guess what? You're probably gonna add other latency points back in as well So it's important to to think of those four steps there and to actually do them All right, so now I'm at story time here I'm gonna give you guys a few little stories and some examples in my career where it was important to actually use Things like asynchronous processing and asynchronous IO to solve issues with latency. All right So story number one This is for dn simple So this is a relatively recent problem part of what we have to do is we don't we don't handle our own internal billing system Because writing a billing system is not hard But it's time-consuming to deal with all the stuff that you have to deal with with dunning and making sure that your customers are paying Their bills and notifying them the email sending receipts so on and so forth So hooking up to an external service provider what we found is that there are variable response time So most of the time our customers would experience a nice little You know maybe there's a hundred like a hundred milliseconds to go out to the service and then our server answered And so maybe it was a second. All right, and they're fine with that Okay, but other times seemingly randomly it would just sit there and hang for 10 seconds 15 seconds To the point where sometimes it was actually timing out engine X's Maximum responsive so say look out cut 30 seconds. We're cutting off 30 seconds. I mean that's insane So we have this variable response time. That's completely random. All right. This is a problem So what is one of the solutions that we could have picked at well the solution we chose here was asynchronous processing Okay, and so what this is is basically saying I've got some work that needs to be done And it depends on something else So I'm gonna actually do that elsewhere, and I'm gonna respond back to the customer very quickly All right, and in that case we used rescue. All right, so we basically had said okay We're gonna have a rescue job that's gonna go out It's actually gonna do the process of of calling out to the gateway and saying okay Make a charge against this card come back and give us a response and so on so forth and What we presented to the customer was a holding page that essentially would refresh I mean it's a simple stupid solution, right, but it worked because the bottom line is the people when it responded really quickly They would hit that holding page, and then it would just refresh really quickly and actually go into the domains and they're okay They're set up. It's paid for people that take a little longer It would it might sit there, but we gave them an exit. We said you can continue using the app You don't have to wait for this, but your app your account won't be activated yet Until this processing is done, and it works quite well And yeah, we took a really simple way of approaching it through like I honestly I met a refresh But it worked all right you could do it through Ajax or whatever it doesn't matter But the point was is to let the customer continue on without delaying them because quite frankly They don't care if your credit card processing takes a little bit longer. They just care about continuing to do their work All right So that was the first story second story Was back when I was working for chimp I don't know if you guys know chimp CHI dot MP But it was a service that I worked on a couple years back and one of things that it did is it aggregated all of your Stuff all right supposedly into a domain that you own and it aggregated all the stuff from like RSS feeds and Twitter and Flickr And blah, blah, you know back when social aggregation was hot, you know back in 2005 or six or whatever it was And so the problem that we came out is we actually experienced a decent amount of growth and while we were trying to retrieve RSS over HDP We had a processes that were there were already asynchronous processes because they would sit out there and they would run externally and they'd update Update the database But the problem was is that the lag between when that process would start and complete was getting longer and longer and longer And sometimes it would just hang it would never complete we'd have to kill it So in this particular case asynchronous IO to the rescue all right so asynchronous IO Uses part of the OS underneath to basically say okay I don't want to sit there and block waiting for IO to come from wherever the IO is going to come from I'm gonna continue I'm gonna return back to the processing and you're gonna register some sort of a callback and when we actually get data We'll go ahead and invoke that callback. So that's how asynchronous IO essentially works pretty straightforward concept So we did that and we were able to get some really good Improvements, but the problem is is that parsing and processing still takes time So ultimately we tried to use threads and if you've ever done async IO and threads It's hard. I mean it's just really hard to get it right because you can get weird deadlocks And if you're in Ruby land trying to actually discover those deadlocks is pretty insane We were on J Ruby at the time doing this particular processing So we were actually down inside of like a Java profile or trying to figure out what the hell was going on And the truth of the matter is we just got lazy and said let somebody else handle it because ultimately that actually worked really Well, we had an external service that handled it. They called back. They became our asynchronous IO and processing service Sometimes that's the right solution. Sometimes you got to be lazy All right, so final story here Final story, and then I'm actually gonna probably show you guys some code just so we can see some examples just so I'm not just telling stories So this actually isn't about code. It's about people All right, remember the talk was async everything. I'm not telling you just to asynchronize your code and that makes sense In some place. I'm telling you that sometimes you if you do it. Well, you can actually have asynchronous processes with humans And it works it works super so that's me and That's my brother my brother and I worked together on Dan simple and we're nine time zones apart So there's not a lot of overlap all right, so Synchronous communication is great. It's it's simple. It's really simple. You call somebody up and you talk about something you find a solution Except when you're nine time zones away or 12 in the case where I was actually working with people in Hawaii as well So you have sometimes yet what we found was that coming up with a good repeatable Asynchronous solution for human-to-human communication actually made things a lot easier to deal with It's scalable and the reason it's scalable is because we can actually drop a lot of messages to each other and deal with them as We have time to deal with them and We can we can prioritize our work. So for example if a customer support issue comes in that's pretty critical But if if if I'm sitting there and I'm on a regular on a conversation with my brother And we're trying to solve something now I've got to choose between which of these things I'm gonna deal with I'm gonna deal with the customer service issue I'm gonna deal with the thing I'm working on with my brother one of them's gonna suffer But if we're working in a synchronous mode Then he just sends me we have a discussion, you know, it could be the email It could be a campfire doesn't matter We have a little discussion and we deal with it in an asynchronous fashion and I can go straight to handling the customer support issue And remember what I said in the beginning all of this stuff that I'm talking about making things asynchronous Really has one goal in mind and that's to improve the happiness of your customers All right, whoever that may be So I think the neat thing is when you're building out these asynchronous systems for businesses and I'm a big fan of this I like I'm a remote worker So I really like the idea of building companies around asynchronous processes I'm not there's still a whole bunch of people out there the debate whether this is really work with this will work I think it's a grand experiment. It's a lot of fun And I think it's important if it does work that we share about how to make it work So that's part of I think that that companies built around asynchronous processes instead of synchronous processes like meetings and phone calls Are gonna have a better chance for success in a world that moves really fast And so I'm a big proponent of that. So my message to you there is embrace asynchronous communication now Start using it and try to make it work for you even if you're sitting in the same office as Your the other developers or the other people in your company. So none of this is really mind-blowing. All right This this is this is like this great. Okay. I've told you something. It's pretty simple. What they really are is these anything asynchronous, it's a tool. It's a tool like anything else and You have to know when to use it and you have to get comfortable enough so that you can wield that tool like a professional So let's take a look at some of the tools that you might be able to use All right, so I've got a few examples here How's that everybody can see that Okay, excellent So what we're looking at here is a little event machine Code so if you've never heard about event machine event machine is a is a asynchronous IO library that is Done in Ruby so it's interface on Ruby on top of asynchronous IO from the OS it works fairly well You can use it to write servers. There's a number of web servers like thin that were written originally using it And there is I don't know if Goliath uses it on the hood. I think it does but anyway The point is is that there's there's plenty of Of tutorials out there for how to use it I'm just gonna give you a quick example so you can see how it works and and how you how it runs All right, so essentially what I'm doing here is I'm kind of Resolving that problem that I was talking about but a really basic what I'm doing is I want to download a bunch of web pages All right, so I want to download them But I if I sit there and wait for the ones that are really slow Then it's gonna take a long time to do everything in a synchronous fashion All right, I could try to do it via threads that would work as well But for this example, I want to use event machine so So I load up the URLs from a file and then I go into the event machine loop And the event machine loop is basically sitting there and waiting for IO behind the scene so that your processes don't have to I Queue up each of your requests for each of the URLs. This is built into event machine. They've got a little library I'm just not built an event machine. It's a little library called emhdp And then I register a callback for when it succeeds So this is just a block and then an error callback for if it fails And inside of that, I'm just gonna print the response You know, I bet I have a laser on here Is the red the laser? Oh Yeah, I got laser. All right. So we got the callback. We got the error back All right really simple It keeps track of how many URLs are still pending so that it knows whether or not it needs to stop in either case If it doesn't need to stop then as soon as it exits the callback, then you're basically back waiting for the next callback So let's see this thing run real quick So I got a few demos here. So that's the the code to run it All right, it's getting the that's it. It's grabbing it. So grab those in parallel Now the interesting thing to note here is if I look at that URLs dot text That's not the root the order that I registered them in they came back as the data was coming through all right and so The longest that we really have to wait is whatever the longest response time is and in this particular case It was for the ruby gems org site all right and all I did is I downloaded them into the files directory here and You could see if I open one of them. I'm gonna get everything all the HTML without any of the Without any of the images or anything like that So really simple thing but the thing is is that I was a I didn't have to sit there and write a process that looped through Each one of these in block. All right, so that's the goal of things like event machine that that's that's one example next example So there's another library out there that's kind of interesting It's been mentioned a couple times here Brian mentioned in his talk. It's called celluloid So celluloid is essentially a Ruby implementation of some of the ideas in Erlang actor an actor implementation if you will So let me just real quick show you the code here for this No, that's that's actually the Erlang code. We'll go to that in just a little bit So here's an example actor. It's not really doing much. It's just sitting there and it's saying okay I'm gonna start up the actor. I want to say whatever this line is ten times All right, and then the main thread is gonna go to sleep and then inside this I'm gonna sleep for a few seconds and then I'm gonna run this. Okay. It's just a toy example But what we can see is when we actually run it I'm gonna actually slide this over here So notice what it did there in the beginning It went into the actor and it says I'm executing. I have ten times ten lines to print Okay, and then the main thread went to sleep. So Then it came and they add the main thread sitting there sleeping and the actor is didn't five seconds later comes up And does its work. So you can actually accomplish the exact same simple thing in Threads because basically what cellulite does is provide a nice abstraction on top of threads and a simpler API to use and it actually adds a lot more with things like supervisors So if you have a lot of different actors, they're out there doing things and actors die You need something to actually capture that and then restart those actors So cellulite provides that and it really does it basically provides what Erlang has is built-in part of the language But I could do the same thing with threads here, too. So you see this little example right here I'll just run that let me can get out of this because it'll just sit there listening forever same thing Right exact same behavior because that's what cellulite does under the hood So that's cellulite cellulite also is kind of interesting because Tony Aserri the one who's working on it is Trying to do a bunch of different things with it So for example, he also has cellulite IO, which is like a non-blocking IO plus process plus threading That's actually even more interesting because that's hard remember I said that earlier back when I was doing this in J Ruby It's just hard to do well It's it's really hard to reason about it's hard to to not have it blow up and have deadlocks So while his library can't stop you from writing code that creates deadlocks It does it does help reduce the likelihood of it. So another example here So this should be I think the final example. No, actually, I'm gonna give you one more after this. So 0mq 0mq is a is a actually a C bit of C code 0mq is a what's called a brokerless messaging queue All right So the way messaging queues typically have worked in the past is there is a broker or a set of brokers that are servers that run kind of Like a web server, but in this case all they're gonna do is they're gonna handle messages and deliver them off to different queues All right, they're gonna either deliver them like there's gonna be a one-to-one I'm gonna sit there I'm a publish to a queue and then I'm gonna listen to it or Alternatively, they might say I'm gonna publish to one and a whole bunch of different systems can listen to it and act Independently all right. So message queues are actually a really important part of building a distributed system That can continue to grow and scale over time What's interesting is 0mq takes the approach of a Brokerless message queue so they say it's just like using TCP only it's Q-based all right, so that's kind of interesting what I'm gonna show you real quick I'll give you the code first so you could see this So this is gonna be a simple pub sub implementation All right So what we have here is we have a little 0mq context There's a whole there's libraries for 0mq in pretty much like all the major languages So that's one of the things that's interesting about queue systems is that you can interrupt between languages very easily Kind of like how HTTP is really beautiful in that fashion It's just HTTP is better for synchronous communication. So We're gonna get our channels. We're gonna open a socket, which is a subscription type. We're gonna connect to that And then we're gonna say hey, this is a I want to subscribe to that channel And then if we control see out we're gonna trap that And we're gonna close out the subscription Otherwise, we're gonna receive and we're gonna print a message and then the other side is the publisher and The publisher is essentially getting off the side of the screen. That's what it's doing Let's move that over here just a time So the publisher is is just going to say, okay, I'm just pumping stuff into this socket So I'm gonna bind to that particular Port on whatever host on the local host plus other whatever it'll go to all your hosts I believe on the local and then I'm going to send a message to that channel. All right, so fairly straightforward In order to see it work what we're gonna do is we're gonna start a subscriber on channel one a Subscriber on channel two and a subscriber that listens to both and then over here on the publisher We're gonna start that up and what I have to do is I have to tell it what channel I want to send to Okay, so you see what just happened there These are different processes, but this one received it this one did not and this one received it If I send a channel to You'll see that This one down here got it and this one got it as well so zero MQ is a very a simple way of Building a a brokerless queuing system where you don't have to actually stand up the broker and actually host it Which has really it's that's neat. I mean it's a neat concept So it's worth looking into to build distributed systems where you don't want to have a broker in the middle All right, so one final example, and then you all can go to lunch because I know you're getting hungry here All right, this final example A little bit of Erlang Couldn't walk away without getting you a little bit early. All right, so Erlang if you've never seen Erlang it's got this like funky syntax. It's functional language It's like, you know, it's like graveyard style stuff, right? But not really what it is is it's essentially a really great language for building Concurrent systems because it's all built into the language, right? It's either built in the language or the standard libraries and it's really good at running a bunch of lightweight processes and Then monitoring those processes so if they die off that you can handle the air is the same type of thing that we saw in celluloid but Built into the language It's got a whole bunch of other neat properties as well like a really awesome pattern matching So so let's just real quick walk that this this code was inspired by Brian's talk I was like, I wonder if you were doing a simulation Could each actor in Erlang actually be kind of like a person, right? And if you were doing that, what's the maximum number of actors you could have in Erlang before it blows up? All right, so I was thinking ads probably won't work, but it'll be fun Erlang by the way has the same problem as Ruby It's slow We're for whatever that means what it basically means is that if you're processing binary data, it's slow All right, if you're doing concurrent messaging, it's fast as hell. All right, so I think that's really important It's got a great messaging system built in it's got a really good VM under the hood with a lot of years behind it So back to this so what I'm gonna do I'm gonna in this by the way doesn't actually when I simulate something I'm not actually doing any of the calculation site I just want to see if I could stand up a million actors what would happen. All right And it turns out by the way that the maximum is like a hundred and thirty seven million actors is what it supports right now So I was like yeah, I could model something with that So anyway, so I take the population and then the iteration the number of iterations that I want to run I create the actors for it and what this these if you don't know a functional language is like an object-oriented language without a Classes, it's just like if you just take the the things that's what I mean. No, I'm I'm simplifying here I'm kind of fucking with you a little but So that what it does is it creates the actors This is that neat pattern matching So if the arguments are zero and the actors then just return the actors Otherwise if if it's n so anything other than zero then start a link to the actor So this start a link is basically saying I want to connect these two processes Internal processes never going out to the OS to do this is all internal And then I'm going to call recursively call create reactors I'm going to decrease the number by one by the way all this there's no state that actually gets modified That's one of the things that I I'm sorry I'm becoming a convert into these like into functional languages. This stuff's just badass I mean it no state Nothing that I have to worry about that like that all of a sudden there's gonna be random state That's just gonna get in my system. That's fantastic. So anyway, I've told I've drank this Kool-Aid all down So then I'm gonna add my actor onto the head of this list This is just a little notation that says I want to add this on to the list kind of like the arrows arrows And then I'm a recursively call that until finally this once we get down and hit zero then we're just gonna return the actors So I have my actors. I'm gonna say this is the number of actors. I have I'm gonna create a list That's just basically a sequence of numbers representing my iterations and then I'm gonna go to the next iteration All right in the next iteration same thing pattern matching again here if you have an empty array Then all you want to do is say I've completed all the iterations And I'm gonna say okay that I have to return something so that's when we turn This little underscore here by the way means just ignore that argument because it's really cool or lang if you give it if you Have a function that has a some sort of an argument that you pass into it And then you don't use that argument the compiler warns you doesn't like that And conversely if you try to set if you try to use something that wasn't declared the compiler warns you about that too It's really neat. It saves a lot of problems So I'm gonna do that. I'm gonna actually execute the iteration here. Tell me I'm executing the iteration And then the execute rate the iteration on says use this actor Like send them a message. That's the message sending pattern. They already said it was built in the language It's like send this actor this message, which is a tuple of myself my identifier basically and ping Which is just it's just a a symbol that I can use essentially the equivalent a symbol and Then I'm gonna recurse notice. There's a lot of recursion calling itself. It's this stuff I haven't done any tail recursion Optimization because it's just an example and then the actor on here Very simple right now This is the part that it says I want to spawn a link back and it says connect So basically these connects the two so the main one could kill off these actors if it decided to want it to and then I'm gonna receive messages With my process ID and the message I'm gonna print something out for crabs and giggles and I'm gonna recursively call listen where I'm gonna sit there and Receive again, that's an each one of these actors is its own process. So that's the amount. That's the code basically and I'm going to go over here I've already compiled it. I Have the Erlang VM running. It's got a repo so I could actually sit there a repel Whatever you want to say basically a way that I can just talk to the running processes I'm gonna tell it. I want a population of a hundred and I want to iterate ten times That's it's done Didn't do much but all it's look at everything's all out of order, which is fantastic. Let me slide that over I want you to see that Whenever stuff is completely out of order and you've know you've done something right if you're building a synchronous system That's the way it should be right. You're not supposed to depend on any this ordering So great it did exactly what I said it wanted should have do it just if you look back up here It's like yeah It did it. All right. Let's just real quick. I'm gonna close this out I'm gonna take a population of I don't think I started with the full key there. Yeah, a hundred thousand ten iterations Oh, see that's the thing where I hit the system limit. So watch this So I think it was plus P. I don't know. Let's say a million work and Then I need that last one there Hopefully I got the right. Oh, yeah, that did it. It's running It'll just run for a little bit So the point is it can go that's with a million actors going through and number of iterations and the mass the most amount It's basically spending all this time writing to IO right there and with that. I'm done. Thank you very much everybody. Enjoy your lunch Have a good day