 Can you guys hear me? Because it's weird because if I was like a rapper, I'd be like, turn me up in the monitor because I can't hear myself. But we don't have a monitor, so it doesn't work. You can turn yourself up in the monitor, but it's a display. It's a monitor. Ha ha ha. So before I start, for you to forgive me, I'm actually controlling keynote with my Android phone because that's how I roll. So today I'm going to be talking about Ruby simulators. I actually have like a whole day exposition on writing simulators in Ruby. But as I found out yesterday, I only have 30 minutes for this talk. So this is just going to be part one. So the introduction. Oh, wow. We got lag. I'm Brian L. Brian Lyles. I actually work at Thunderbolt Labs, the aforementioned company under a bus company. And we do awesome things from, I do it from Baltimore. Other guys do it from Portland. Other guys do it from San Francisco. But that's not what we're here to talk about today. Today we're here to talk about simulations. And really the kind of simulation I'm talking about is simulating a group of people doing stuff and actually figuring out what's going to happen. So we have a group of people. They do stuff over time. And what they do is they get together. Like this white chick gets with this black dude. This black dude gets with this other white chick. This white chick right here gets with this guy with the V-shaped forehead. Kind of cool guy, artistic. And this guy goes both ways. He gets with everybody. So we're going to talk about modeling this. And why are we going to talk about modeling this? Well, let me show you why. Sound guy. God. But I didn't want him to think I didn't trust him. I didn't know I could catch dog on the playground. We were in love, so I didn't think about it. All I did is trade lunchables. Every year, 2 million kids are infected with cooties. Didn't you tell me, baby? I thought cooties was something that happened to other kids. Cooties! I just wanted to play tag. I never thought I'd be it. And the numbers are growing. Save myself. And even though a vaccine is available. Circle, circle, dot, dot, now you have the cootie shop. Most children never get inoculated. Speak to your kids about cooties before cooties speaks to them first. So I'm here today to talk about the epidemic of cooties. I mean, I'm sure some of you in this audience have cooties right now. So what I'm trying to do is we're going to build a simulation to help world policymakers and public health people understand the effects of cooties on a population. So what would I do? Well, of course, what I would do is build cootie sim 2012. And because this is an arbitrary thing, and let me pick an arbitrary language to build it in. So I'm going to build my cootie simulator in Ruby. And first of all, I know what you guys are thinking. If you're going to build anything performant, why would you build it in Ruby? Well, why not? Ruby is extremely expressive. If you use things like JRuby, you have full power of the JVM. If you use MRI, it's quick in some things, not others. So let's talk more about the simulation. So there's two types of real simulations. They're stochastic and deterministic. The difference is deterministic simulations actually run the same way given the same set of inputs every single time. And stochastic actually uses random inputs. So we're actually going to use random inputs. And I just wanted to get that out of the way in case somebody was a smart ass and wanted to ask me what kind of simulation we're going to build. So to build our simulation, what we're going to do is we're going to define inputs, we're going to compute, and we're going to analyze the outputs. Very simple. And I'm what I'm trying to prove is that anyone in this crowd who actually can code in Ruby even a little bit can write this kind of stuff very simply. So to throw some code at you, because we are at a RubyConf, this is what my simulation does. It took me about 40 seconds to write this and just throw up a slide of this is what our simulation does. I'm not going to explain this. But basically, I'm going to explain it. Basically, you have people, and then you run some kind of loop with simulations, and you report on that. Now, let's go into this. So we're going to define our inputs. And actually what I'm doing here is I'm actually going to build up an app that I'll actually show you run. And this will be most of the code. So we're going to have a simulator. Because we're using Ruby, we have to use classes. You'll notice my last slide didn't have any classes on there. So I have to have a class. So you create a class named simulator. We're defining inputs. And the next thing you want to do is compute. And really, for our simulator, all we need to do is we need to create pairs. And then we need to transmit our cooties. And then we need to progress the cooties, because you know what? Cooties actually, there's a cooties disease. There's a cooties infection. And there's a cooties disease. You don't want the cooties disease. And then we're going to analyze the output. Simple things here. We're just going to actually just print it out to the screen at first. So let's go into this even more. So with our cooties simulator, our simulator works day by day. So every day, we're going to create new pairs of people. Everybody goes out to the playground, and they go find somebody to pair with. And then we're going to see what happens. So how do we create pairs? Well, this is very simple. Creating pairs is we just actually have a method for people not in pairs for the particular iteration. And then we shuffle each slice, and we map. One thing I want to point about this, I actually asked everybody at Thunderbolt to run this. And even they didn't realize that this code will not run in Ruby 1.8. It can't run in Ruby 1.8 because of the dots. Ruby 1.9 got smarter and said that, hey, we want to do it Java style and be able to have a builder type pattern and be able to put dot shuffle dot each slice. In Ruby 1.8, you can't do it. You actually have to have the dot on the other end. I'm glad they actually made that change. But really simple code here. And then we want to transmit cooties. And how do we transmit cooties? Well, very simple. All we need to do is a person has an infect and acquire. And what they do is if people are in pairs and you infect someone and then they acquire the disease. So like I'm saying, what I'm showing here is this very little code to actually move this forward. And then we want to track infection progress. And once again, this is Ruby because all the indentation, indentation, indentation, indentation. So very simply, once again, and I promise I will not throw too much more code at you guys, I just like to see it on the screen because I wrote it. So like I said, this is not deterministic. This is stochastic. And you'll notice that I have if-rand. That right there means that every single time it runs, it'll actually will have different types of outputs. So one thing to say, this is not going to be fast. And I'll show you why this is not going to be fast. When I run this for, let's say, 365 iterations, so let's say one year with 100 people, it takes 28 seconds on my Mac. My Mac is pretty fast. I actually have one of these new Fangled Macs. And it actually ranges from like 28 to 50 seconds. But that's only 365, that's only one year with 100 people. That's not modeling anything. So let's look at why Ruby is not fast. So this right here is actually output from Ruby Prof. Anyone here use Ruby Prof? Why is everybody's hand hot up? You should use Ruby Prof. I know most of us do web apps. And they don't have to be fast because that you cache. And that's how you make web apps fast. But in the real world, you've got to use Ruby Prof. And what Ruby Prof says, what we already know, going through arrays is slow, especially when you have very large arrays. So I want to talk about, anyone here know bigger notation? Can people tell me the difference between log between log square and n squared? What's the difference between n squared and 1? Anyone know? So really, Ruby Array's worst case are if you want to actually iterate through an array, worst case is order rent. Because you have to look at every single host, everything in there. One thing I don't like about Ruby that I really wish Ruby had was better data structures. I mean, you can create them. But out the box, you get a hash, you get a scalar, you get an array. And you don't even get a cool array. You don't even get a linked list array. You get just an array. So what other things can we do to speed this up? Well, another thing you can do to speed this up is you can run more than one thing at a time. So let's imagine that on a numeral, we actually had a parallel each. And what it did is, as it went through, it would actually create a thread. And it would yield. And then it would do a cleanup through a thread. And then you could actually run, whenever you iterate it through an array with each, you could actually do more than one thing at a time. And you know what? Someone thought this was a good idea. Gym install peach. You guys can't see what this says because if anyone saw this slide, it actually says Y-M-M-V. And the reason it does say that is because you don't wanna do this. Don't use, don't gym install peach. It'll actually make it slower because really what it does, it tries to create a thread pool and it actually runs each. It actually breaks up your each and tries to run it in a thread pool. But the problem with Ruby is, creating a thread is not very fast. So by the time you create your thread, if you have something that runs in one millisecond, you've already lost out. So don't do that. So anyone know what this is? Yes, this is Perftools, this is output from Perftools. So the difference between Ruby Prof and Perftools is Perftools is a sampler and Ruby Prof is a proper profiler. And what Perftools will do will actually add set iterations in your code. It'll actually see what's running. And right now we can see that we spend a crap ton of time in the garbage collector and we spend a crap ton of time in going through, once again, iterating through an innumerable. So what can we do about this? Oh, we're not gonna talk about that yet. So I was trying to illustrate the point here that we have 100 people times 3,650 iterations. 3,650 iterations. I mean that's 365,000 actions. And that's just for a very small simulation. So I have to have one slide in my talk that has cool effects and this one might be it. There it goes. So I found that I could do the Star Trek thing or this Star Wars thing and I was really excited. I spent like 30 minutes on this. So here's where we wanna be. We wanna actually find a population of a million people. So think about it, that's three billion actions. I'm a computer guy, not a math guy. So this talk is about simulations but really what I'm getting down to is how do we make Ruby faster? I'll tell you one secret. You can't make Ruby faster. It's impossible because you just can't. But there's two things we can do. We can run less code or we can do more than one thing at once. So let's look at this. Using MRI 1.9, whenever we execute code, we execute and pretend that this thing right here is your CPU because this is how your CPU looks. It has, mine has four cores plus two hyper, or plus four hyperthritic cores. We're not gonna talk about that. So I can run one task, two tasks, three tasks. If I do thread.new, what happens is I still all on one core but I can do more than one thing on one core. So what does that mean? At best I'm only using 25% of my CPU with MRI for one process. Anybody wanna refute that? Okay, just making sure. So with JRuby, JRuby same thing, one, two, three, three serial actions or three synchronous actions. Whenever I do thread.new in there, I can actually go across my cores. So this makes you think, well, why in the hell would I ever use MRI? Well, I can give you like 100,000 reasons why you would. It starts up slow. It takes a while to spin up and it uses more memory and it's not compatible and a couple other things. So, but you have to understand that whenever you wanna do these kind of simulations, what are you doing? No, and you know what? You are absolutely correct. You know, and I will not refute that because if you actually go into activity monitor, you can't see that it does go above 100%. But only a little bit. And the problem is, even with Turbo Boost, so let's say my Mac is a 2.7 gigahertz i7, it Turbo Boost is 3.7. It's not even, it's not like it's Turbo Boosting to like 11. So, here's my first thing. If you wanna do something performant in Ruby and you have Ruby code that you just written on MRI and you wanna why it's so slow, just take it out and run it in JRuby. I made no tweaks to my app and I ran it in JRuby and it actually ran in less than half the time. That's a weird case, but it did run in less than half the time. So, here's a weird section. How do you do more than one thing at once at MRI? Well, you can use queues, real popular thing that queues right now. Redis-Bacht, how many people here use rescue? People here love rescue? Who am I? How many people use sidekick? How many people love sidekick? Wow, no, one guy likes sidekick. Sidekick is awesome, but let me tell you why I can't use it in my simulation. Sidekick is like the fire hose of Redis-Bacht queues. You can, sure, I think even on the page, the guy, Mike, says that you can do that 100,000 actions per second. But what if I don't wanna do 100,000 actions per second? What if I wanna put, what if I want to create one batch that has, let's say 95,000 things in there and create a second batch that creates 95,000 things? With sidekick, I can never be guaranteed that the queue is empty and I can move on to my next iteration. So really, it's like up and down, up and down, and it actually doesn't work very well. Not saying that sidekick doesn't work very well, it doesn't work very well for me. So how many people here have used Girl Friday? Girl Friday is awesome, and actually it's even extra awesome under JRuby whenever you can use real threads. I expect everybody to check it out. Actually, the same guy that wrote sidekick wrote Girl Friday. These guys really own some queues. Other types of queuing are memory backed. Rabbit and queue, do I really write memory backed? Oh, I'm sorry here. It's not so that, it should be like AMQ backed. So people doing high concurrency and high amount of traffic using things like AMP queues or AMQ, yes, you know what I'm saying. So, and then there's also DB backed. Is anyone, oh wow, I'm going, this is Android with Apple, it's not working well. So, socket based concurrency, people are using zero MQ and D-cell. Anyone here use D-cell? I'm using D-cell, so I guess I can raise my hand. So, moving on to something more exciting, statistics. Anyone here good at statistics? I can't even say the word, huh? So, whenever you're doing things of this nature, what simulation, you're gonna have a lot of stats coming out. So, important things to know, mean, median, variance, standard deviation. Everybody knows the mean, add it up, divide it by the number of things that you added up. You get the mean, median's the number in the middle. You know what the variance is? Variance is how much the numbers vary. But no one really uses variance. People actually use standard deviation, which is the square root. Is it the square root of the standard deviation of the variance? That's what it is, see? I'm learning. Actually, I know Randall Thomas is not here. He's going to do a talk with... Holy shit dude, come on in. Oh, you're brown, I can't see you. Smile. So, standard deviation, square root, variance. So, another cool thing, and I was actually, I was just talking to somebody a little while ago. I don't do much open source. So, when I do open source, I like to talk about it. So, gym install vows. And there is no disclaimer underneath this. What vows is, is a Ruby implementation of the vows alias method. And what the vows alias method is, is a way to sample probabilities in real time. And I'll show you one cool thing you can do with it. So, let's say you had a die, and the die had six, is a six-sided die. We're not playing D&D here, it's a six-sided die. And you're playing craps or something, and you roll it. What's the probability that a two comes up? This is what I'm sampling here. So, I have an array with six items, one, two, three, four, five, six, in there. And what I'm doing is I'm actually sampling this a whole bunch of times to make sure that one through six come up evenly. And you can easily do this with RAND, same code with RAND. I did use an inject. And actually, Tamer Saleh, I showed him this code, and he's like, what are you trying to do? Brain fuck me? He does not like inject. I like inject for this kind of stuff. Ryan Davis? Yes, how you doing? Now, actually, I usually don't use inject, but I wanted to, like I was explaining to him, I wanted this to show up on one, so I did use inject. And really, all I'm trying to do is show that it's the same amount of code. And actually, right here, RAND is actually faster than my Voselius method. But I'll show you something that RAND can't do. So, what if you had a crooked dial, like I would like to roll with, and your six actually comes up, your six actually comes up more often than any, or actually, your one comes up more often than any other number. So this is what the Voselius method is for. Actually, I can actually say that my one comes up more often, and you're like, well, why would I do this? So if you're sampling mortality rates in a group of people, and you say that 10% of the people die at 70, and 10% die at 71, or no, even better yet, you have a 0.01 chance to die at 70, a 0.01 chance to die at 71, and then you have like a 0.03 chance to die at 72, you pop all those in an array, you pass that to the Voselius method, and you just choose next, and it'll tell you when you die. So, moving on. So graphing, we like graphing. I know most of you guys are probably familiar with more like high charts, shake cue plot, things like that. This is not a web app, so there's none of that. So if you wanna do trend graphing, and actually, a very important thing is I run the simulation a lot of times. Sometimes I run it, you know, maybe a couple hundred times a day. And what I'm trying to do is actually a slow iteration, a slow day for me would be one millisecond. So this is the kind of level that I'm working on. Slow iteration is one millisecond. Actually in Java, it's way less than one millisecond. It's like a quarter of a millisecond to run one day. But so what I wanna do is I graph all this stuff. So this is a graph of how long it took over time, and you'll notice that it goes up literally and then it gets up here. My guess, all this stuff up here is actually, I was getting into Ruby garbage collection, and that's actually, that's what happened. Garbage collection started happening a lot more often. So it actually got a little bit off kilter. This right here is the same code, and it's actually JRuby. So what you can see right here, anyone know what this right here is? Yeah, it's hotspot. It's actually, it is, it's hotspot. So it was actually, it was, JRuby actually without hotspot would be slow as crap. So it goes up and then hotspot kicks in. And you notice that the slope of the line is actually way less. And this is the clue that for things you need to be performing in, I wouldn't say your web apps, those are weird. But if you're actually writing performant Ruby, I would notice oxymoron. Look at JRuby. Notice I didn't do Art Rubinius or MagLove. I love those projects as ideals, because I'm idealistic, but I wouldn't really use them for this. So how did I do that? So I tried doing in Ruby and then realized that there's much better things out there. This is R, and I know a lot of people in here want to be experts at R because you can write cool code that no one can understand here. Like on this third line where it has the myGG with the arrow pointing to it, that's actually, I know it's like an equal sign or it's like a pin onto its sign, but I really don't understand how it works. This is actually just all scarfed off the internet, curve-cultured for this talk. But what it did is it does create these graphs. I do know what the next to last line does. It sets the output file name. So that's R. So the next thing you want to do is visualization. And a cool thing that you can do is, and this is actually something very interesting to me. So I created all these, so I created these graphs. And I want it to see from patient zero. Like person one always has cooties. Who is person one giving it to? So if you look all the way at person one, and then now here at person eight, this is actually, I just drew this little graph here, and this is just graph phase. And it's actually very easy code. You guys should definitely look at graph phase if you're doing this kind of visualization. I mean look, it's really simple. Actually generate this from Ruby. More cool effects from Keynote. So, back on the build in the simulator. How much time do I have? Oh my gosh. A simulator goes around and around. It's a loop. Like I said before, there's people. And every single day we have a selection of people who are available to run. And then when those people are available to run, what we do is we pair those people up. So we know that this old guy likes little girls. And actually that's just how it happened. I didn't really pick it like that. And then I guess this Asian dude with long hair, it's not a girl. And this other girl, they like each other. And this black guy is just hanging out. So when they pair, then they share hoodies. And this is my visualization of hoodies being shared. So how do you make it fast? One thing to do is be kind to MRIs GC. Does anyone here know why Rails is slow? Besides Ryan, I'm sure he knows why Rails is slow. One reason Rails is slow, if you ever dumped memory, like object space, Rails creates a shit ton of strings. So you know one thing to do? Don't run, just don't create any strings. That'll be much easier. Also run with more than one process. And look outside of MRI. And when you're looking outside of MRI, remember I said before that JRuby is not compatible. And I actually, I brought this up, I refuse to go into the JRuby bug tracker because they say that it's gonna be Ruby 193 compatible. In Ruby, you can actually pass a range to RAND. Not a lot of people do this, but I wanted to do this. I wanna be able to say between day, I wanna have a relationship that starts on day one or ends one day from now or seven days from now. The best way to do this would be RAND with a range. You can't do this in JRuby. So you have to be very careful about this. So you have to make sure that you're testing on both platforms. And notice it runs on Ruby 193, just fine. You might also wanna think about other languages. I literally spent two weeks trying to write this enclosure. And you know something? That's exactly how I feel about it now. And not saying that closure is a bad language, I'm just saying that you know what? We're very good at imperative languages and I have yet to find a definitive problem that I cannot solve faster. And this is just me. And my imperative language is that I can solve in a functional language. I mean, I'm sure there are use cases. Most of us won't run into them. So Ruby is a compromise. There's a compromise on speed. There's a compromise on just not knowing what's going on. You can hook up Visual VM to JRuby, but you will not get the same view that you would get if you hooked up to a regular old Java project. So we're actually, I was gonna bring up some code and actually I have two minutes. So I'm gonna, I have a couple minutes, right? Yes, there's no timer here. So I'm just, you know, I'm just winging it. All right, five minutes. So I'm gonna show you how easy it is to write these kind of simulators in Ruby. And don't make fun of my editor. I use a, I use sublime text. And what I'm going to try to do here is, can you guys see that? Should I make it bigger? All right, I'll make it bigger anyways. So what I have here is my simulation Charlie. And the reason I call it simulation Charlie is because I know we have a version control, but whenever you're writing code for a talk, using version control is actually very hard for me at least. So there is no version control on this. I just actually just did it old school, like cold fusion guys would, and actually just copied the file to a new name. Every, for every iteration. So what I have here is a, I'm gonna drag over a terminal as well. There we go. I have a terminal and there's my IP address. And so I have two, I have, there's the tab. Oh, that's even worse. Yeah, sad trombone-y somebody. There we go, there we go. So we'll just type in the top left corner of the screen. So I have two versions of this thing. And so if I run the simulator over, let's say we're running it over 10 years with 100 people from Ruby. And you know, it runs fairly fast. But whenever I start, whenever I go up to 100 years, you can see that, hey, you're thinking it's scrolling really quickly. You can actually control the number by changing the modulus on the print. So you think it's pretty fast. But look, this is the same code. And actually, this is like a little preview. I'm talking at Wendy City Rails about cool shell tricks. Look what I'm getting ready to do. Two terminals sharing history. That's pretty awesome. I should know how to do this. So the same file, actually running under JRuby, it actually runs way faster. So that's five seconds versus six seconds. That's way faster than whenever you're doing simulations. I mean, it's really, it's like a few thousand milliseconds. So what the kind of nanoseconds or microseconds that is, that's actually pretty fast. But you know, here's the worst thing. And I hate this because I am a Rubyist. We actually have a Java version of the simulation. And we're running it with the same constraints. And this is why you should always look outside of your comfort zone. So I can make it faster in Ruby, but it's still gonna be slower than Java or C++. So that's the lesson to be learned from this little talk. So thank you guys for not hackling me too much. It's been fun.