 presenter, calling him simply a speaker, does his pants at his service. He hardly needs an introduction. In addition to being a Rails Core team member, he's an avid beard grower and meat connoisseur. In addition to 90s fashion hoarder, he's here, I actually got the, I have the slip of paper. He's here with written permission from Gorby Puff. I had to get that, I had to make sure that it was okay. And with that, I'd like to welcome up on stage Aaron Patterson. Thank you, thank you. Thank you, thank you. I'm very excited to be here today. May I be the first person to welcome you to RailsConf? If you notice any typos in my slides, please let me know. Just tweeted at me. I want to start my talk off with a couple of quotes. The first one is, my mom has the Ferrari of sewing machines. This is from Kate Hiddleston. We were talking about her awesomely embroidered jacket last night. You should follow her. And the other quote I want to start with is Dad Gummett by Bill Dance. That's actually why I have this hat, is because I'm a fan of his. I love the animated GIFs working keynote now. Anyway, so my name is Aaron Patterson. If you don't recognize me, this might help. That is what I look like on the internet. I learned from the sketch note talk that I should put my Twitter handle there, so hopefully you can remember it. I have a cat and that's him. This is him from a different angle. So if you like this talk or if you like anything, you should give it a tender love seal of approval. So I'm on the Rails or Ruby core team and I'm also on the Rails core team. I think I'm the only one on both teams. That doesn't mean I know what I'm talking about. It just means I'm really terrible at saying no. So I work for a very tiny startup that's based in Dallas, Texas. AT&T, I actually do work for AT&T. I really have to say thanks to them for employing me to do nothing all day, basically. So I'm an open source developer for AT&T and we're going to talk about stuff that I do. Anyway, my job title is in the past. 2011, my job title is Corey Nanes. 2012, Senior Facebook Integration Engineer Elect. I lost that election, unfortunately. Then I became a senior software architect. So I got a drafting table and all that stuff. It was very cool. This year, I'm a thought leader in training. So I want to get out of development and I heard that thinking is on the rise, possibly. So somebody needs to lead those thoughts. So I figured it's a great market for me. So I noticed there are some people around here with Google Glass. That's pretty exciting. I also have a Google Glass. I did not bring it. This is my Google Glass. I saw there was a talk about, someone was giving a talk about doing 27 million hours of pair programming or something and ask them anything. I thought that was really cool. I also do pair programming, but not for as long. Like here's, I want to show you a couple of action shots of me pair programming. I'm pair programming there. This is a very close up pair programming shot. I can't do it for 27,000 hours. My pair goes bad after a couple of days. So I don't know how he did it. Anyway, the USB interface on this was terrible. It was very hard to figure out. Anyway, I have two cats. This is my not as famous cat. Her name is C-Tac YouTube Facebook Instagram. But we call her choo-choo for short. And then this is Gorbachev Puff Puff Thunder Horse the Third. And you can follow him on Twitter at Gorbipuff. That is his Twitter name for those of you sketch noting. This is me with my cat. I love them. The Christmas card in the works right here. So I, like I said, on the Internet, I am tender love. I write a lot of puns on my Twitter. And I also give people hugs and stuff because I work from home and I get kind of lonely working from home. So I hug the Internet on the weekends. That's why we did that thing, although I'm extremely shy. So I would not make you all do that. So let me give you, I want to give you a breakdown of my tweets. These are my tweets. It's 90% puns. Hugs, cat photos, and sometimes tech stuff. I also brought a bunch of stickers with me that look like this. This is stickers of my cat. And the thing is like, so it has come to my attention that people might know who I am. And this concerns me very greatly because I am very introverted. So I'm really, really shy. And I'm like, you know, you might be saying to yourself, you're talking in front of many, many people. How is it possible that you're doing that? And really shy? Well, the thing is I'm actually incredibly scared and I'm using that fear to say stuff in front of you. And also, I'm wearing a ridiculous outfit. So if I mess up, it's like, look at these. Distraction, wow. Anyway, so people have said to me like, Aaron, you're very famous. I'm afraid to talk to you. And I want you to know that this is why carry around a whole bunch of stickers with me is because I am also afraid to talk to you. I don't know what to say. So if you want to come talk to me, this is like a totally easy thing to do is just say, hey, Aaron, I heard you have stickers. Can I have one? And I will say yes, I do. I love my cat. Here's a sticker of my cat. So please, you know, don't be afraid to approach me. Now let's talk about some Chicago stuff. We're in Chicago. I noticed like, so people have told me you need to connect with your audience. Like it's important to connect with the audience and I didn't, I noticed like nobody really did this so much in their presentations, but I'm going to do it now. I'm just going to say a bunch of Chicago stuff to like connect with all of you. Like the bears, the bulls, Mike Ditka. So now we're all connected on Chicago, right? Okay, perfect. So last year, I want to talk a little bit about what I did on my keynote last year, and then we're going to go into this year. Last year, I introduced my consulting company. Here it is. This is my consulting company, Adequate Industries. I did the same joke last year, but I figured this would be good enough. Like it should work. Figured that would be adequate. Anyway, I also dropped a lot of F-bombs. It was pretty sweet. I also got into a really awkward situation. So before my keynote last year, like I always, before I give a talk, I always do one thing. That's one weird trick that I do, which is I go to the bathroom before I give a talk. And unfortunately, I was very, very nervous and I went into the women's bathroom. But there was nobody in there, so I didn't know. I was like, wow, there's a lot of stalls in here. That's okay. I need to use the stall anyway. All right. Use the stall, come out. And then I'm like, okay, there is a woman janitor. I'm okay with that. I'm a progressive person. It's fine. I don't mind if there's a woman in the bathroom. And then I see that there's another woman washing her hands. And then I'm like, oh, shit. And the janitor's like, get out of here. And I'm like, ah. And of course, the only thing running through my head is like, do I wash my hands? Do I wash my hands? Anyway, last time, so last time I was in Chicago, it was about 10 years ago. This was before I had an iPhone or anything. And this, so this story is, is rated PG-13. So like, I know we're not, we're not in prime time yet. It's only like 430. So hopefully you'll forgive me. Last time I was here, I accidentally went to, well not accidentally, I guess it was accidentally, I ended up at a party at Cynthia Plastercaster's house. Okay, I see none of you know who this is. If you Google her, only click on the Wikipedia link, please. That's the only one you want to click on. So my friends, I was staying with some friends and they're like, hey Aaron, let's go to a party, we're gonna go to a party at Cynthia Plastercaster's house and I was like, that sounds great. I'm hip, I know artists, people who like to do things with plaster, that's cool. So I didn't like, I didn't want to seem uncool so I didn't ask who this person was. And we went to the party, went to the party, and it turns out that Cynthia Plastercaster is famous for in the 70s and 80s making plaster casts of rock stars penises. And I did not know this in advance. So we go to this, we go to this party and at the time an adult video had just come out with Jimi Hendrix in it, but allegedly Jimi Hendrix and they weren't sure whether or not it was him. So she was called in as an expert witness. And so she's there and they had just come out with a DVD with her being the expert witness and she puts in the DVD and we're like watching and I'm like, what is going on? She's on the DVD, she's like, yes, you see here and here, that's him, that's definitely him. And I lean over to my friend and I'm like, who is this person, why are we here? And so then she explains it to me and I'm like, oh, I see. So it was incredibly awkward and the most interesting part of it to me was that there was a guy there dressed in a hot dog costume which seemed most appropriate for the place where we were. Anyway, so that was my last encounter in Chicago and the moral of the story is if you don't know something, you shouldn't be embarrassed to ask about it because down the line it could be potentially even more embarrassing. Anyway, so Rails is 10 years old and that's really exciting. I was thinking about like, okay, Rails is 10 years old. What did I look like when I was 10 years old? And so I decided to dress like that today and that is why I'm dressed like this. So this is about what I looked like when I was 10 years old, maybe 12 or so, I'm not sure, around in that ballpark. And I know you probably can't see the back of my t-shirt. I'll turn around here, it's probably hard to read but what it says is this, it looks like this. It says, if you can't surf with the big dogs, stay off the net. And I love it, it's like there's a dog surfing by, it's like the literal interpretation of the internet. And you know, in the early 90s, like this is how most people thought the internet worked, is like these things going around the globe with like bubbles and stuff and there's a dog flying by. And I'm just thinking to myself like, yeah, there's some guy sitting there clicking his mouse, like yeah, if you can't surf with the big dogs, you gotta get off the net. Just kills me. Anyway, so I think we've made a lot of progress since then because basically what dogs look like on the internet now is this, like, I mean this is surfing with the big dogs right here. Anyway, so 10 years ago, like Rails, the Rails community started solving incredibly hard computer science problems. So 10 years ago, we were able to solve the problem of naming. Check, it's solved. Say Rails generate new model or whatever. We got the names down, you use plural, super exciting. Last year, last year during DHH's keynote it was very exciting, incredibly exciting. We solved caching, boom, Russian doll caching, thrown in there, don't worry about it at all. Also in the previous year, like another insanely hard computer science problem solved was dependencies. We solved that. In fact, I have a quote about it here. The genius behind NPM is that it solves dependencies, a long time problem in computer science. Boom, solved. Unfortunately, that was with NPM, so all I just have one thing to say is like, get your act together, RubyGems. What is the deal? Why have we not solved dependencies yet? So this got me thinking like, what's left in computer science, right? I mean, we're doing groundbreaking work here on the Rails core team. It's amazing, there's one thing left. P equals MP. And you know what? I'm gonna solve this one for you on stage tonight. I don't even know why this is so hard. You just divide both sides by P. Done, we're done, yeah. I think we're done here, we can all go home now. Anyway, so this is my anti-keynote. DHH opened, he gave the keynote. I am closing, I will give the anti-keynote. Tonight, we are going to talk about coverage, speed, metrics, and hard effing science. So things I've learned at RailsCuff, Yehuda taught me that it is nearly 5 p.m. on a Friday at the end of a very long conference, so your cognitive batteries are probably very low, which means that I am your default. It means that you need to listen to me. Science is good, TDD is great. Baratunde taught me that I am white. I did not realize that until the other day. Farah taught me that I need to code hard and code fast, so tonight I'm gonna troll fast and troll furious. So DHH's presentation gave me a really awesome workout for my eye muscles. You can see from this diagram right here, this is the eye roll muscle. I'm not shit, not a lot of exercise back there. Super strong. So I'm like, okay, oh man, we got this person with a birth year on it, how do we test this? Well, obviously we have to pass in today. We modify our API, pass in today, and then we can test it, and I just have to say one thing, stop! This is not how you test it. This is not dependency injection. So let me show you how I would test this the way I would test this is I would use the API that we already have. Your person has a constructor. You can give it a birth year, so pass in the current year, and then test whether or not the current birth year is equal to that, and whether or not, then refute it as well. What's really cool about this, what I think is really, really awesome, is that there's no new API necessary. Wow, amazing. Constructors are pretty cool, I think. I like constructors a lot, but the lesson from this is that you need to make construction easy. If you make construction of your objects easy, then they should be easier to test. So okay, there's some trolling out. I wanna start out with my controversial opinions. These are my only, well no, I'm gonna give you some controversial opinions now, and then some at the end of the talk, so I'm gonna start out with controversial opinions. I am a software engineer. I am not a software writer. I am sorry. I imagine, when I hear a software writer, I imagine somebody sitting in their villa in France on a winery. Oh, look at how that brace lines up with this other brace. It's beautiful. So amazing. Look at how those methods are indented, just one underneath private. Oh, it's so readable. What kills me about this is that beauty is in the eye of the beholder. Like if you, for example, look at this ridiculous outfit I'm wearing. In the 90s, this would have been acceptable. Science is important. I can't believe I had to say this. So you may not have to be a computer scientist to be a web developer, but having some sort of scientific background might be important for things like making money. Like, I mean, sure, any of us can sling out some HTML, but can you make money with that? You need to be able to measure things, like, for example, AB testing. There's math. Money, more math. It just, anyway, yes, science is important to me. So I want to talk about mutable state. I think I said in the title I was going to talk about mutable state, so I'm going to do that. I promised I would. Amatsuda said in his talk, Rails has many bugs. So we're going to look at some bugs. These are my favorite bugs recently. Actually, I found two of the three bugs we're going to look at. And all of these bugs have to do with mutable state in one way or another, and we're going to see how. The first one is inconsistent return values. Typically, I assume that if I call a particular function twice, it will return the same thing. I know this is probably silly in Rails. So I think that those two methods should return the same thing. If you call that once, this literally happens today in your Rails code. If you do these two particular lines of code, the first time it'll return false, and the second time it'll return true. And when I look at this, I feel like I'm Bill Dance with a dog going. Let me just watch that gif. I love the dog. Anyway, there's another problem with this too. It also has a memory leak in it. Look at that. These are action controller parameters. This is the params thing that's in your controller, that hash that you deal with in the controller. There is a memory leak in it if you do this particular code. Like if you write this, if you fetch a value and then delete it and then assign a new one, you'll get a memory leak in it. It'll look like that. So if you run that program, it'll just go infinitely. And the reason is because we're doing a conversion on the stuff that we pull out when you do fetch, and that conversion gets shoved into an array. And unfortunately, this parameter's hash, it allows you to delete things from it. But when you delete stuff from it, this internal array thing is not updated. So it deletes from the hash, but doesn't delete from this other converted array value. And this is something that was missed because the parameter's hash inherits from a hash with a different access, and hash with a different access inherits from hash, and hash implements a whole bunch of methods that maybe you didn't think about. For example, delete. So when you inherit from something like hash, you have to think about all these different ways that people can interface with your code. And it's similar to this. You're like, oh, they might call delete, they might update, they might replace. What do we do about that? So this is one thing that I do not like. It's not very fun, and I mean, you can tell this is not done with TDD, possibly. Oh, I'm just kidding, I'm just kidding. It probably totally was. So another example here is an inconsistent return value. So the first time we call size on this one, on this relation, it'll return a hash. And the second time, if you call 2a on it, the second time it'll return a one. So I would expect that both of these return a hash or return a one, but they don't. And the cause is because we have a mutated state inside of the relation object, which is when we load the records from the database, we shove that into an instance variable so that if you try to reload them again, we don't actually go hit the database. So if it's loaded, then we'll return the length of the records that are in there. And if it's not loaded, then we'll actually do it, we'll do a count on that. I'm not totally sure how to fix that. These first two problems I'm not sure how to fix. I feel like this one, another build dance. I should probably tell you whenever we find issues like this or mistakes, we take one of these gifts and put them into campfire, which is why I own this hat. I feel like I am screwing up all the time because it's probably my fault. The next one we're gonna look at, I like to call WTF SQL. This one is courtesy of that Eileen codes. She did not call it WTF SQL. That's my name. You can see her speak at Ruby Nation and I highly encourage you to do so. Actually, I saw her present this bug at Mountain West Ruby Conference. She was giving a talk on active record stuff, like in turtles and she's going through all this stuff and she's like, if you call it in a certain way, then it's incredibly inefficient. It does this weird stuff and it's incredibly inefficient and of course, during Q and A, I was like, why don't they fix that? And then I was like, oh, they is me. Okay. Anyway, so let's look at the bug. So we have an object here. This is just using fixtures. We have an author, the author is David. We create a whole bunch of posts for it and then call delete all. If we think about what the SQL is for that, we would expect that it's like, okay, delete all from posts where the ID is whatever and if you execute this, yes, that's what happens. Delete all from posts where author ID equals one. Okay, now let's say you're messing around with this thing in IRB. You say, okay, David up posts and you inspect the return value of it. So this test is simulating that. We're calling inspect on that because that's what IRB does to any return value. It calls inspect and then shows you the text below it. So what do you think the SQL would be for this? If you thought it was the previous one, you are wrong. Otherwise I wouldn't be showing this to you. This is what the SQL looks like. A giant in statement. So yes, this is definitely inefficient compared to the other one. Anyway, the cause of this is exactly the same as the cause of the previous one. We had state saved. We said, okay, if this is loaded, then we do a different thing versus if it's not loaded. So we're like, okay, if it's loaded, then we're gonna do a delete with an in. Otherwise we're just gonna delete all the records when actually we just wanted to do a delete without the in. So this is my next bill dance gift. Oh, I love this one. So the lessons that I hope you all to take from this are avoid subclassing built-in classes. They probably implement stuff that you're not expecting. They're probably implementing an API that you may not need or may not want. You should be conservative about the API that you implement. It's much easier to add a method than it is to take one away. If you take one away and you need to deprecate it, there's a lot more work. It's a lot easier to introduce something new. The other thing is that you should be extremely careful with mutations. I'm not saying like, don't mutate stuff. I'm just saying be very careful about it. If you keep a small API, then it's much easier to be careful with your mutations. So you might be asking yourself, Aaron, you find all these bugs in the rails and there's all these bugs. Why would you want to work on this piece of code? And I'll tell you, the reason I like working on rails, the reason I love being on the Rails core team is actually because I love Ruby. Is that weird? I just Googled for heart images. I don't know if that's like. Is that? Anyway, the reason I love working on rails is because Rails is what gave me a job as a Ruby programmer. So I want Rails to give everybody jobs. I want everybody to be able to work in Ruby on a day to day basis. So it's really important for me to make Rails a better place so that companies will continue to adopt it and that people will be able to have jobs and get paid using Ruby. So the next portion of the presentation I want to talk about is adequate record. The thing that I've been working on is from adequate industries. So what is adequate record? Adequate record is a fork of active record with some performance improvements put on top of it. And I want to talk about those performance improvements. But first off, I gathered a whole bunch of quotes about adequate record, like what people are saying about it, like using it on their applications and how they find it. So I want to share those quotes with you just so you know the quality of the software. It's like active record, but more adequate. I'm quoting myself. That's how you know it's legit. It's all right, but I've seen better. Thanks Jeremy, I appreciate that, that was really good. Yehuda says, oh it seems to work. Josh Susser says, sometimes good enough is good enough. I actually, I tried to get a quote from Matt and he just said, what is that? So why did I name it? Like why did I actually name this library adequate? I actually haven't had a motivation behind naming it adequate record. And the reason is humility. Like I'm actually super tired of libraries that are named like active something and terrible this and all that. Like why do we have to have things that are named like super awesome whatever software? Like I just want to know what your software does. I don't care how awesome it is. I just want something that works. So that's why I named it adequate record. It's also another thing, like I think it's an anti-marketing term. I feel like a lot of software is trying to be sold to me and it really bugs me. I don't like it when people try to sell stuff to me. It's kind of like, I don't know. The new version of people coming to your door and trying to sell you stuff. I want people to buy this software based on its merits not based on like how cool the name is or whatever. So why did I fork? The reason I forked is that I don't actually want new features. So this is a dirty little secret of mine. I don't actually want new features in Rails yet. I think that Rails accomplishes like 90% of the stuff that we need to have make great web apps. And I'm afraid that if we try to keep going that extra 10% we're just gonna be adding cruft that nobody wants to use or it's just even worse increasing the complexity of the code that's there. It actually makes me incredibly paranoid when we add new features to Rails because a lot of people think like, hey there's a new feature there. This is really awesome. I can do these super magical things and I look at it and I'm like, oh man, how many years of refactoring is that gonna cost me? So let's talk about adequate history so I can give you a little bit of background about why I have opinions like that. I've actually been working on this project for about three years. So I've been refactoring the Rails code base with this particular optimization in mind and it's taken me three years of refactoring this legacy code just to get to the point where I could actually add this stuff. If you look at my talks for the past three years I've been talking about this stuff in theory like talking about doing prepared statement caching, talking about doing yak shaving. All these things are actually just leading up to this. So I'm actually very glad to have all of you here today because this is probably to me the most important talk that I've given because I've spent so much time on this. I've had this idea in my head for three years now and I really wanna talk about it, adequate. So in order to talk about this we need to talk about how ActiveRecord works. So I'm gonna show like a brief overview of how ActiveRecord works and then we'll talk about the performance improvements of adequate record. This is what it looks like when you say post.find10 we go through a process that looks similar to this. When you say post.find we create new ActiveRecord relation objects. Those ActiveRecord relation objects eventually get translated into arrow SQL nodes into a SQL AST. That AST is then transformed completely into a string, literally a string. That string is sent to the database. The database returns a bunch of records. Those records are turned into ActiveRecord base objects and then sent back to you wherever you called that. So this is basically the high-level overview of what happens with ActiveRecord. So the change, the first change that we had to go through was bind parameter introduction and I'm gonna show you why we need to do that. If you looked at the logs in Rails many years ago you'd see something that looks like this. If you did post.find1 or find3 you'd see these SQL statements in your logs and you may notice about them the only thing that changes is that number. The rest of the string is constant. It does not change. So what we did now, if you look in later versions of Rails you'll start to see something in your logs that looks like this. Select star from posts where id equals question mark and then some square brackets that are saying id comma one. And what the idea here is that we're trying to separate static and dynamic content. Those id's are the only dynamic thing in that string. We wanna be able to generate something that's always the same. The adequate theory is that in this diagram if everything that generates that string always generates the same thing we should be able to cache it. If you say post.find and it goes through all this stuff and it generates a SQL statement that's always the same we should be able to cache that computation. It means that we can say okay let's just cache that. Then the next time you call post.find we don't have to generate the SQL statement it's already there because we don't have any dynamic values embedded in it. So this was done quite a while ago. The next thing that we needed to do is decouple our code. Now separating the static and dynamic content sounds like easy enough but it actually is not because it is embedded deep within the Rails code. I mean I'm gonna show you where it is. This is the worst part. Right here, you don't need to read that, don't worry. Those three arrows are pointing at dynamic values that are being inserted into your code. This is deep within the call stack. This is actually inside the associations code. And that is one method. It is one method. It is one method to rule them all in legacy code.find. So this method defines how all the joins are done inside a relation. Basically the problem with this method is if you have, specifically it was STI. So if you have single table inheritance and you need to insert that value for the model name in there, that's what that was handling. So in order to simplify that, so that code was, I think I said this, was only to do with associations, like has many, has many through, et cetera, et cetera. And I wanted to simplify that code so in Rails 401 I deleted has and belongs to many. It is gone. Thank you, I'm extremely proud about this. And for those of you that haven't upgraded to 4.1 yet, it still works, okay? Has and belongs to many, the keyword is still there. You will be able to upgrade your app and it'll actually work. And the way that I got it to work is has and belongs to many is actually a subset of has many through, if you think about it. So if you think about has and belongs to many, it's got three tables. Has many through also has three tables, but has many through forces you to define a middle model. So the refactoring that I did is just on has and belongs to many, it translates that call into a has many through with an anonymous middle model. So it should be completely transparent to you. And because I was able to do that, I could remove a bunch of code from that very complex method. And I wanna show you that, I don't expect you to read this diff. I'm gonna show you a diff. I'm just showing it to you because I'm very, very proud of it. It's one of my favorite commits in the Rails code base. Like all you need to see is that there are red parts and no green parts. And there's a bunch of commits that look like this. And the reason I love these commits so much is because they're removing stuff, not replacing it. And they're specifically removing conditionals. There are if statements in there that are just getting pulled out and thrown away. It makes me so happy because when you look at this code, you don't have to think about those conditionals. Those conditionals are what tire you out. You look at that and you're like, oh, if it's this thing, then it does this. And you think, well, how did it get that way? And you have to go figure out how did it get that way? So I felt very happy about this. I was eating code out of the Rails code base. It made me super happy. So the next part, I'm actually making it on time, I'm happy, okay. The next part, part three was to introduce a cache. This is finally where we get to see our speed improvements. I'm gonna show you what the caching API looks like, but this is totally internal, it's like, don't use this, the API is unstable, but I'm just gonna show you what it looks like and what, I'm gonna show you what it looks like so that you know what the cache is or what we're caching. This is the cache code example. So we can say, okay, person.wear, name, parameters.bind, and then limit one. And what that does is it caches the relation that you created in there. And we can execute that cache statement multiple times with dynamic values. So wherever that params.bind is, that's a dynamic value. So there's our normal active record relation object and that is where we execute it with different values. Now the thing about this is the reason I haven't exposed this to anybody or one reason I haven't exposed this to anybody is because the return value of that statement inside the block must be an active record relation object. And probably I'm figuring most people outside of the Rails core team don't know which methods return active record relation objects. So I know this, but probably not many other people do. So I don't think it's a very good interface for public consumption. So I wanna show the cache object internals. This is what the actual statement cache looks like. We have, from that previous example, we have the bind values. These are the bind values. This keeps the offset. It indicates the offset into the SQL statement where that dynamic value is and it also contains the information for how to cast that dynamic value. So for example, when you get an ID that comes in from the request, that ID is a string and we need to be able to cast that to an integer. This contains that information. And then down below is where the actual SQL statement is cached and you can see it's just a string. So the next step after defining these cache, this cache was to update the internals and cache any relation objects. And this is very interesting because we went through and looked at all the places in ActiveRecord where relation objects are actually used. So they're used, these are the examples of things that are used in, I know it's a cardinal sin dots with stuff to read, but I don't care. I've been working on this forever, so screw you guys. So all these things use ActiveRecord relations internally. Basically everything that you do with ActiveRecord uses relations internally. All of these are cacheable. So we can add cache to all these things and I'm gonna show you an example of the cache that we added internally. This is an example implementation. So this is the implementation for find with an ID. You can see in there we have a key and we say, okay, create that relation. So in your Rails code, you may have typed post.wear primary key arrow, that thing, we're basically just doing that internally. So it's really not much different than what you would do inside your app code. So that is our static values for the SQL statement and then here are our dynamic values. That's the ID that you actually passed into find, so you just always call with that ID. So I wanna share a few, after looking at that, it's really exciting, I wanna share a few fun facts. These are the different ways that you can call post.find. I don't know if you knew this. You can call with an array. You can call with an array of an arrays. You can call with a hash. You can call with an ActiveRecord ID. You can call it with a block. You can call it with a scope and you can call it with a block and a scope with any of those combinations above it. So only that one is cacheable. The rest of these are like, I have no idea why we even support that. I don't even know what they mean. So when I was trying to figure out what we could cache and what we couldn't cache, I felt like this. So if you go look at the method, if you go look at the method, I'm like, okay, this is all the stuff that we can't cache. I don't know this, this, any of these. So I'm gonna show you some performance statistics for this and if we could remove all of that API, that would actually be super nice. I would like that because we could have ActiveRecord even faster. So let's talk about measuring performance because we wanna see the performance of these things. I'm gonna talk about some benchmarking tools, the way that I benchmark this stuff and the way that you should be benchmarking things. The first tool I wanna talk about is benchmark IPS. This is a library that actually Evan wrote and I'm gonna compare this to the normal benchmark library that comes shipped with Ruby. This is the normal benchmark library so you say like, okay, benchmark something and then you need to loop over something and see how long it takes. That'll report to you how long it takes to call that thing. The problem with this benchmark is we loop n times but how big should n be? You ever written a benchmark and you run it and then it's like, oh, zero time. Good job, dude. So you have to sit there and think like, okay, I gotta increase the n, maybe decrease the n and maybe you send the benchmark off to somebody who has a faster machine than you or a slower machine than you, then they have to change the n. What benchmark IPS does, it actually measures the iterations per second. The number of times it can call that method per second. So you write a benchmark that looks like this and it will run that block as many times as it can in five seconds. So it'll just defaults to five seconds. And the output looks like this. It says, okay, we can call it that many times in five seconds. And the other thing I really, really like about it is it gives you a standard deviation report, that plus or minus 3% that's shown there. Problem with the previous benchmarking tools, it doesn't tell you standard deviation. So let's say you write some code, let's say you do some sort of performance improvement. You think you've done a performance improvement and you compare your benchmark, you have this benchmark, you compare your code to the old code and it looks like it's slightly faster. You're like, great, it's slightly faster. But you didn't know what the standard deviation was. If you don't know what the standard deviation was, it could just be noise. Could be other programs impacting your benchmark. You don't really know that information. It's better if you have the standard deviation so you know, ah, I'm one standard deviation outside or I'm two standard deviations outside. So yes, this is definitely a performance improvement. Getting standard deviation with your benchmark is important. The next tool that I use is GC.stat. I'm not sure what version of Ruby this is in, I think 2.1. GC, this is in order to count object allocations when we're doing queries. So this call will do a total object allocation count. So this actually counts the number of objects that have been allocated in your system ever. Not like now or not inside a loop, but ever. Every time you allocate an object, this value increases. So what we do is I say, okay, I'm gonna call, I'm gonna warm up this benchmark, I'm gonna call person.find, then I'm gonna count the total number of objects that were allocated in the system, then I'm gonna actually execute my benchmark and say, okay, find this person so many times, count the objects again, subtract and divide, and then we know the number of objects that have been allocated per call. Science. I should have put a slide of Jesse Pinkman in here. Anyway, so let's actually look at some numbers. Let's look at the performance of adequate record. So we're gonna look at post.find and post.find by name. So I wanted to benchmark these because they're essentially the same thing. One is just finding by ID, the other one's finding by name so they should be approximately the same. And this is a graph of their performance essentially over time. The x-axis along the bottom is the branch, so on the left side is two, three stable, to three, oh, stable, proceeding to the right, the very right is adequate record. The bar along the y-axis is the calls per second to post.find, so the higher that is, the better. You can actually see there around in three, one stable, it was tanking like super hard. I mean, look at that. What's also interesting here is that yellow line represents mySQL and I should say these, along the bottom there, I say SQLite and mySQL2 and mySQL and Postgres, that actually represents the particular adapter that we were using. So not exactly mapping to the database or testing mySQL twice. And I think this graph is interesting because you can see in two, three stable, there was actually a bias towards mySQL. MySQL is significantly faster in two, three stable than Postgres was, but you'll see that those actually even out along here. Now this is the same benchmark but with post.find by name and you can see basically the look of the graph is about the same as the previous one. We started out faster in two, three. It slowed down, then over on adequate record we're going much faster. And I just wanted to call out this one here. This is mySQL, this is that same graph but I specifically pulled out mySQL to show you exactly the performance, the performance changes of mySQL over time. So you can see it just tanked super hard at three, one and then slowly came back up. This is the percent faster the adequate record is than four, one stable. So let's see, what is that? Some of these are over 100% faster so it's two times faster. This is the percent faster it is in three, two, three stable. We can see mySQL, it is faster on adequate record but not by much. So we are faster. This is a graph of the object allocation so this is the number of objects that are allocated per call. And you can see there around three, one stable and three, two we're actually increasing the number of objects that we allocate per call to this. So it doesn't necessarily directly impact the calls per second. It does have an impact on it. GC pressure does cause issues but they're not exactly aligned together. Same issues with find by name. And you can see there again on two, three stable a strong bias towards mySQL. So, adequate record allocates 70% fewer objects than four, one stable does per call. It allocates 55% fewer objects per call than two, three stable. Now I wanna look at belongs to, like I wanna look at the, I'm gonna look at each of the relations. So belongs to graph looks very similar to those previous ones except that SQLite 3 is insanely faster. So again, along the bottom is time, along the y-axis is calls per second so higher is better. Percent faster than two, three stable and four, one stable. The MySQL 2 bar doesn't have a two, three stable one because MySQL 2 that adapter didn't exist for two, three stable so I didn't test it. But we are all adequate record is faster in all cases. Has many through. So I wanna show has many as well as has many through. This is the performance graph for that as well you see exactly the same thing. Sorry about fast two, three stable went down and now we're faster and adequate record. Also has many through call speed over time. So again, we're also faster than two, three stable in every case, much faster than four, one stable in every case. And the last thing I wanna look at is has many through growth. So caching these SQL statements should give us a, should give us constant time performance with regard to generating the actual SQL statement string. And that's what I wanna demonstrate here is show you that constant time performance compared to actually any of the other branches. So what I'm talking about with has many through growth is let's say we have a model that looks like this. We say has many through one thing. What I wanna study here in this case is what if we add another has many through. So what if there's two of them, we go through two. Then what happens if we go through three? What happens if we go through four? Et cetera, et cetera, et cetera. So we keep increasing the number of tables that we're joining against on has many through. And this is what the graph looks like. So you can see on master, well, four one stable, actually any of the branches will look similar to this. Along the bottom is the number of tables that we're actually joining against. So that's the number of has many throughs that we're joining against. So at the very right hand side, we're joining against 16 tables. And you can see on master, we actually have linear growth. And on adequate record, we have constant time growth. And it does grow just a little bit there. And the reason it grows just a little bit is because we're actually sending those queries to the database. So database time is impacting this, but the algorithm is constant time. So I am extremely, this is my favorite graph. Oh yeah, constant time. So this is one of the things that I'm talking about with why I think studying computer science is important. Okay. If you study this stuff, you can say, you can look at these, this performance, you can increase your code and you can talk about it. You can quantify it. DHH gave the example of like, oh, I take a lot of pictures and now I've taken a lot of pictures. So I'm just good at pictures, which is very nice, except that there is a, I'm happy that I take pictures, that's cool. I like to make, I like to cure meats. I mean, we all have things. But even with, even with studying picture taking, people are like, okay, we have the rule of thirds. We have the golden rule. There are these things that we do that eat, that we can all talk about. We have a common language and we also all discovered that hey, if we do these particular things, our photographs turn out better, generally speaking. We have these things we can talk about and I think that's exactly what studying science brings to the table. We can talk about this. I can say to you, here's exactly why the performance is much better. Generally, constant time is better than linear growth. Anyway, TLDR 100% faster. 9,001% better. It's over 9,000. So what I wanna talk about now is I wanna talk about the adequate pricing model. How much are you willing to pay for adequate record? $200, $300? Well, today I'm offering a very special price. Today I'm offering it for $299 plus in-app purchases. I'm just kidding. Let's talk about some of the challenges, like some of the issues with this thing because we're not like, it's not all unicorns and fairies and all that stuff. Like there are definitely trade-offs for this code and I wanna talk about that. We're trading memory for speed, obviously. Like we can't just get this speed from nowhere. We have to cash this stuff. So we need to understand what the cash size is and I'm gonna give you a little formula for the cash size. I'm sorry, it's math. So we need to count. The cash size is equal to the number of relations that you have, all those has many's, and also the number of find buys that you have in your system total. But it's bound by that and it's multiplied by the size of that cash object. So I can't tell you exactly what the size of the cash objects are because they are variable depending on the size of the SQL queries that you have in there, but they're not very big. So we are definitely paying for memory. So this will actually increase the amount of memory that your system is consuming. So we need to study this. We need to study this because maybe we should introduce an LRU cash to this and expire things as they go out. But since this particular cash is bound, I don't think it's gonna be a problem. The other issue is raw relations. So you might be saying to me, Aaron, I'm really excited for you to increase the speed of all this stuff, but I don't use find by name. I use this. Person that wear a name, that thing. I use that, I don't use find by name. Why don't you just make that faster? And I say to you, why don't you make it faster? Jerk. And the question is, so the question is can we cash this? Like can we cash that thing? Can we make that thing faster? And we can, it's true. We can do it, it's possible. I have an experimental branch to do this. This isn't in adequate record right now where we take those values and then generate a key from the values that you pass to each of those relation objects and then cash the SQL query that's generated from that. And we actually see a 30% speed improvement. This is person.wear, name to a. So blue is master and green is the experimental branch with the caching built in. So it's about 30% faster. So that's good. The problem though is if you take a look at the implementation, like this is approximately the implementation, that blue part is calculating the key for the cache and then the red part is actually executing the query. Now the problem with this code is that over 11 different variables impact the cache key. So I have to actually inspect 11 different variables to say like, are we sure that we've seen this query before? Are we sure we've seen this query before on 11 different variables? In fact, that implementation that I showed you only handles two forms of person.wear. I only cashed like person.wear and person.limit, those two. Because I wanted to show you one more experiment. One more experiment is doing a finder comparison. So let's say we have person.wear, name, errand like what you might write in your controller and then person.findbyname, errand. These are both on the experimental branch of adequate records. So the person.wear does have the speed improvements that I just showed you. And if we benchmark these two, this is what the results look like. Wear compared to findbyname. It's not even a contest. Findbyname is three times faster in this case and that's even with performance improvements. So the wear case and the reason is because every time we call that person.wear, we're actually allocating a new active record relation object and we can't get around that. We're always gonna be allocating those objects. So my actual question is like, should we cache these relations? Should we do this? I'm not sure. I'm really not sure if we should do this. It introduces some very complex code and if you're doing a bunch of like dot wears and all those things, I'm not sure exactly what you'll be caching. In adequate record today, the cache is bound but if you're doing a bunch of crazy stuff like all that wears and stuff, maybe it will be unbound and you'll see unbounded memory growth and I would not like that. B2W unmounded memory growth is bad. Science. So I want a new API. I want something that looks like this. So like cached query, find all by name, do name, and you pass in a name. So you can define a method that does this and we pass in the dynamic values to that. So we build up a cached relation in here and then we can execute that cached relation multiple times and the reason I want that is because the cache key is easy. The cache key is just that thing you pass to me and we're bound in size. The relation API is maintained as well. That's another advantage of this thing and I know you might be thinking, Aaron, this looks like scope. We can't use scope because people in their applications today expect that the scope block will be executed multiple times. I expect that there are many of you in the audience who have a scope that are like today's posts, right? Where it generates a date, it says give me all the posts that are after that happened today and you'd expect that that date is calculated every time that block is executed so we can't just go off of the scope block. All right. So I've got one other more thing. I think Steve Jobs said that, right? So let's merge Attica record. Should we do that? Yes? Okay. So I know this is a totally cardinal sin of presentations but I'm gonna do it live. Is that okay? Would you forgive me? All right. All right, so let's do this. All right. Hold on, I'm not very good at typing, sorry. Okay. Okay. Okay, this will work, this will work. Okay. So we're getting pulling now. It's downloading from the internet. The internet might be a little bit slow. Hopefully it'll work. Please work to do this live. Oh, okay. Okay, I think it's there. Ah, oh. How do you merge? I don't know what I'm doing here. I mean, don't you think that Git has really good documentation? Isn't that true? Like we can read through all the help pages there? That looks, ah, ah, I didn't know about that one. Okay. Ah, oh, back to the merge, back to the merge. Okay, we're merging it in, we merged it in. Ah, all right, let's commit to merge. Commit it, commit. Okay, doing good, good job, Aaron. Oh, that was, that was the wrong way. Okay. Now let's merge it in, now let's merge it in. Okay. Merged, ah, nice. Okay, that's master, right? Are we doing it right? Is it coming up? Ah, oh. Okay. Add a good record, it's been merged. Three years of horrible stuff done. Ah, okay, so one, this is my last very, very strong opinion. My last strong opinion of the evening. I think that Rails 4.2 will be the fastest Rails ever. This is what I am working for. I also think that this is just the beginning. This is just the beginning of the performance improvements that we can do to Rails. So, thank you.