 I just wanted to show off with contributing to your open source product and be like, this is the meme with him? So it looks like Tenu loved any of these. He wants you to merge this. And then add some stuff so while you know, onto the pulse that I want you. And then, oh he updated it. This board request is updated. And then merge this or else. And then he applied it. And then he sees the board request and he drops it. So, anyway. Open source can be fun too. Not really boring. I'm not talking about open source. I am going to do this instead. Today I'm talking about Venice. Going behind the keys. I don't know what that means. I was trying to make a lean joke. If you want to get a hold of me, I am Qlush on Twitter. Or just go on and bug me afterwards. So I work at Thoughtbot. We make web apps for everyone. We have everything from big companies to start us going off the ground in Boston, Massachusetts. Yes, I know I came to the wrong city for St. Patrick's Day. And we use Redis a lot for stuff like, we use Rescueton. We use it in a lot of our clients and a lot of our internal products which you may have heard of. Hot Toad, which is our app error app and plus errors. And you may have heard of this thing but that uses Redis a ton. So Redis can be used to make your site faster but we're going to go a little bit off on the tangent there. So what is Redis? If you haven't heard of it, it's one of those NoSQL data stores. It's actually termed as an advanced key value store. So other key value stores when our memcache, the good cabinet, is one that I'm missing. Well, the keys are basically strings. The values are strings, usually. So there's no tables, there's no relations, there's no graphs, there's no documents. All you have is a string that maps to another string and your database figures it out. Except for Redis, there's more than just strings at the value portion. There's many data structures. So there's actually six. Technically there's five, I really think there's six. There's actually strings, counters, lists, there's kind of arrays, sets, which are like our Ruby sets, which are named terribly, and hashes, just like our Ruby hashes. So Redis knows about all of those and can speak about those at the database level. So you don't have to bang your data into a table. You just store it in that same data structure that we're all used to messing around with. Right now, Redis stores everything in memory. So it's seen as, it's used very much like a cache, like memcache. So if it goes down, well, you're mostly screwed. But you can kind of get more or less parallel with Redis, depending on how dependent your app is on it. So you can basically save the disk every so often. It's not full asset compliancy, but you know how it goes. By everything in memory for now, that means there's a branch being worked on by Redis guys that will allow Redis to write the disk by default instead of the memory all the time, which has a serious performance loss, but also makes Redis a lot more useful in other cases. So what does Redis use for? It's a whole huge array of what Redis has used for a while. I think the three big ones are smart caching. So for example, instead of just storing stuff by strings, let's say you've got a user's timeline on the site, that's a list. You model it as a list inside our Redis, and it's a hack-a-lot history. So we use that for a ton on the little pop code and RubyGems.org. It's also used a lot for job use, so that same list data type can also be used to basically push and pop jobs onto a queue, and then you can have tons of burgers, hundreds of them, and they will all contain over the same resources. Redis is inherently single-freddit, so each command is going to be executed one after another, so that kind of atomic action is really necessary for a job use. If you've ever looked at the internals of a delayed job, you'll see half of it is like just trying to make sure that workers aren't fighting over the same job. Redis kind of just blows that all the way at the protocol. It's also used for a lot of high-speed analytics, so if you need to count a lot of things really, really fast, Redis can do that. A good example of this is on RubyGems.org, every time that you ever download a jam, so when you type jam install, we're bumping around. It's four different counters now. Pretty much everything you would ever want to know about who's downloading a jam, or... So that's really fast because Redis had to deal with role-locking as you try to increment a column and my SQL table will cross-press it. This is a lot faster. So however, Redis is a lot more than get and set, and that's kind of what I'm talking about today. I'm kind of skipping over the basics of Redis. I'm glad to talk to you about that, but I'm going to talk about some of the newer features that are in Redis and some of the oddball things that are really, really useful in your apps. That's a binary operation which we turned to as of Redis 2.2, just a few weeks ago. StatAlgebra, and what you can do with the Redis set data types sorted and then transactions which are also very convenient, really, really important. So a binary operation. So Redis has strings as one of the data types. In those strings, they're encoded as a binary as a UTF-8 string, and you can now twiddle with the individual bits in those strings. I want an example to show this off, and I think one thing that we're all used to is the file permission bits in Unix. So if I make a file and then I run stack with this wonderful command, I don't even know if that does any more. I had to read through the whole man page to figure that out. It pulls out the permission bits. So WRRR, so I can read and write to it, the group can read, and the world can read. That's all for this. And those are basically implemented as bits somewhere deep within the bottles of your system. So I'll only get that out and store that in Redis. Now, mind you, this is terrible for production systems. Probably don't blame me when your system goes down because you use this. This is just an example. I think in practice, using Redis for permission bits could be really useful. For example, if you want to make sure you have millions of users that, let's say, you want to make sure that a user can make a blog post or not. And then you store that in Redis. Instead of a string that might be many bytes, you store it, which is going to save you a lot of memory over those two things. But for right now, we're just doing this example. So here's some binary operation. So I'm going to pull this out using Ruby. Since I love Ruby, I wanted to do that. I'm not a computer engineer. I think I'm a programmer. So I need to write out all the bits I need. I couldn't do binary math in my head, so I'm still at the point where I need to write out the 2, 8, 2, 7, and so on. So for kind of models, it's a binary number, right? So we've got WR. Those bits are flipped on. So we've got a 1. Each little column of 3 is a couple that can be boiled down to an optimal number. So we've got a 7, 7, 7, 7, 7, all these bits are on. Everyone can read and write and execute that file. But for right now, all I care about is that row, the third row. So the binary number. So I've got that 1, 10, 100, and 100. I know that's a base 2. I'm just going to say that. So I need that number in Ruby. Getting that out wasn't immediately obvious. But you can get it out with file.stat. File.stat It basically returns a file.stat object. And you can call it on a file. And then if you choose a mode. That mode, however, is not the binary number you were looking for. It was wrong. It's actually a fixed number. Which is base 10, not base 2. So in order to kind of force it back, I used printf. I could have done and with some crazy hex number. But I'm not that good at it. So I just used printf. So printf with %b binary number. So I get this guy. Which has a 1 in the front of it. I still haven't learned what that does. But at the end there, there it is. This is Unix. I know this. And the wrapping didn't show up. So there it is. I've got those 9 bits at the end. And it worked. So I want in the script as I was running it, I wanted to get those bits out. But a little bit of a trick that I learned that if you haven't seen it, I think it's really going to help you. So I need to then store that binary string and that binary printout to standard out to a string. So I use sprintf. So I now store that in the mode string. And then I'm using this little method on the string called scan. Scan is kind of like the really awesome cousin of spring split. Because it cannot own split strings, so you can just lock and do all sorts of crazy stuff with it. So I'm going to use that more than split, then split when I'm doing the strings. So if you scan and then with a regular expression, I get a lapd. And then that gives me an array of all the bits I need. So those last 9 are it. And then those last 9 I can shove into Redis. I'm going to spare you the really tedious free code of looping over all those numbers, which ended up so visually this is what I want inside of Redis. I want at my Redis key which is at perms, calling normal. This is kind of a common pattern with Redis we'll see a lot is that there's a delimiter. So kind of at my perms name space I've got this file name. I could have used a underscore I could have used a dash or slash, whatever your heart desires you can use. Redis doesn't care. At that I have this binary thing. And it's going to represent my file functions. So basically at a certain offset I'm going to set a value and Redis basically works in this way. So I'm going to bring in the Redis fiber which is you bring in the Redis Ruby gem I'm going to create a new instance of it that connects to the Redis server that's running at your local post and then you set it. So I'm going to set it for the key at perms, normal, sxt my offset's 2 and it's value is 1 simple and done. And if I want to retrieve that back out it's get. So if I want to get the bit that's at offset 8 I'm going to it's going to seep there and then yank that out. So that's pretty much all there is to binary operations within Redis. Like I said if you're doing a ton of user permissions or really flipping you might be actually storing Booleans inside Redis. You might want to check this out but it will save you a ton of memory instead of just straight up storing strings or even storing an entire object, serial objects inside Redis. Because believe me there's it's really hard with Redis to save memory and you probably want to do that as your app's lifetime goes on. Set Algebra. So there's two set data structures in Redis. There's one that's a normal set. In Ruby and our standard library we have a set which works like we would expect to set this. It's got set math in it. It can only hold unique things if you want to look up. If something's in a set that's in order that's a big old book of one. So it's got basically a little hash table to figure out hey, this thing's in the set or not. And Redis's set works pretty much just like that. There's also this other weird child of Redis called a sorted set. Which is the worst thing that is structured I think in history. Really I think it should be called a high score list. Because that's what it is. It's a high score list and that means it's a hash of strings to floats. So I really consider both of these in the set math part of Redis. So we're going to see how we can use both of those. Just like this last example I need an interesting use case for it. I figured I would watch people's tweets. My emotional intentions were a little more nefarious than I ended up being. I wanted to basically watch or stalk all the speakers' tweets and see what they said over the past few weeks. I was hoping maybe we would reveal some secrets but it didn't work out that well. However, I ended up doing this anyway. I used a really cool tweet stream by Michael Blank sorry for butchering your name that basically influenced the new Twitter streaming API. So whenever someone tweets with a given user ID it will send me a bad message. So once again I'm bringing my friendly Redis gem and I can set a bunch of stuff up. I found all those speakers' user IDs with the Twitter API. It's really simple and scary how easy that is. And then I started a new tweet stream Damian. This all this code by the way will be up on my GitHub after I had a chance to read it because I didn't read it prior to this presentation. So you basically started Damian with your user password. My user password is not in the script I'm sorry. And then you follow a bunch of user IDs. So I'm going to splat that big array into the follow method and then that basically makes a vent machine start up and says hey every time I get something from Twitter I'm going to yield a status object to this blog. And that status object has all sorts of junk about that tweet as the last time anything you want to know about that tweet. The only thing I care about right now is that hanging. That Twitter user's name was. So now that we've got my handle from all my speakers oh by the way it's not only just the speakers which is one of the reasons this experiment failed it ended up being everyone who ever mentioned the speakers so I ended up tracking a lot more users than I wanted to so my statistics were a little off. So the redest part finally comes in when I'm going to do a Z anchor file. So first of all we've got this Z thing. Like I said the sorted set is the worst name based structure in history. Every Z command in red deals with sorted sets. I don't know why. But this means that the count sorted set I'm going to bump up handle by one. So every time someone tweets I'm bumping them up in the high score list. So it ends up looking like something like this and this is the results from I think yesterday which I had more sets as well if you're curious. So for example to the life year I had 353 tweets Mr. Timberman 340 and so on and so on. So you basically get this big list. If you think of a set you think of a set kind of early you think of oh there's all these different elements but sorted sets are really kind of vertical. You kind of see the string of their score which is a float internally but it can be an integer for our purposes right now. So red is basically stacks it up and you can then query it based on either the score or the rank in the list. So E light at the bottom is rank number 3 I think. But you can then query it from the bottom or the top no matter how you want to do it and we're going to see how it's done. So we also want to actually use sets in this set. I'll show you an example. So that status text, the status from a tweet is in the status text I'm going to split each word and then add each word everyone has ever said in the last few weeks to a set for their user. So I've got this words and then so I have words and then a handle I add in the set and I'm using the most depressing command ever set. So that will basically add that word into a set. The set just like our Ruby set is it will reject duplicates. So that's the first time it's ever seen that word. It will return true otherwise false. So with that we basically get these big blobs of things. These are randomly chosen. This is really randomly chosen from these two offerings and I did ask them for permission. So you basically get these as arrays of unique elements as a set. So then you can do all sorts of math. For example, we can see the words that they both said by doing an S-inter. This is actually I kind of looked over this S-inter. So there's a command line up there that's using Redis CLI and not the Redis Ruby driver. Redis itself comes with a little command line to help her that you don't even need Ruby to start with. As soon as you compile Redis you get this little almost IRB tool to start messing around with it. So if I did S-inter and then with Wacats and Zed then I get the intersection of both of the words they ever said. So not as similar as I thought. So another cool thing is that you can also, so there's other set operations, but you can also store them into a completely new key. And this is a really common pattern I assume with Redis. Where you basically take the result of a calculation dump it into a different key and then those original keys there's actions still going on the whole time. But you never have to worry about stopping the whole system to do that analysis. You just kind of bring it over to the new key do the analysis you need to do while your system is still running into other keys. So here's an example of that. So I'm going to use S-union store which will do an union of all the sets I get into this new key. So into all words I'm going to store the union of these three sets. It can take as many sets as you get it but I just want to free so I've got a bunch of a bunch of words that they didn't say this time and then I'm going to store them into the all words set. The only repeating word is bar so bar only shows up once in a new set because it's a union. So in Ruby that's actually really easy. So this example was actually I was trying to figure out how many unique words everyone was saying. So start on my reds driver and then I'm going to get this array of all the word sets that everyone has. So I have this big useless array that had everyone's actual Twitter in it. And then I'm going to slap that big array into S union store. I really like to slap by the way if you've got a place. And then in order to get the amount of things in a set it's not as straightforward as one would think. So in real math land we wouldn't get the length or size of the set we would get the card amount. So the equivalent of the length of a set is S card. So if I get the S card of that all word set right it was around 9000 I'm sure by now it's more. So that's doing some analysis on a different key and you'll see that all time if you're dealing with reds a lot. So back to that high score list thing. So here's an example of what that high score list might look like. So this is our sword set or a C set and it's basically like I said it's a hash of strings so reds allows you to do queries on ranges on basically either the keys or the scores in the sword set here. So to do that we would basically do Z range. I want the top three scores here I want to see who tweeted the most so I won't use Z rev range because for some reason it puts the highest thing on the bottom of the high score list I don't know why. So it's kind of a low score list so Z rev range will give me the last it will go basically from the bottom up. So from 0 to 2 that will give me the last three things. However that only gets me the keys it doesn't give me the scores. So if you want to actually see how many times they tweeted it won't tell you that it will just give you the straight up keys. In order to have it tell you that you have to give it with scores. I don't know why I need to beg it to give it the data I want but that's what happens. So hopefully you're using 1.9 right? Anybody? There we go. So that will work otherwise bring out our front of your hash draw game. So with scores we'll pull out both the key and the sword. So using this is pretty easy. I'm going to do z rev range from the bottom to 2 up so I'm going to top three and then I'm going to splat that array once again into hash brackets. The hash brackets method is really, really neat if you haven't messed around with that yet. So if you ever have an array that has an even amount of things to it which this does it's basically key score, key score, key score so it's an even thing. That hash bracket method will basically create a hash out of that array. So if I splat that array into it it knows what to do it gives you a hash tab. So that's a really neat method that I don't think enough folks use because it's just not, no one's that documented about number two. It's kind of like intention hiding because it's a good way. But it's still kind of neat. So there's a lot more stuff that's available. There's intersect and difference on the normal sets. There's also the store version I think with both of those. The z sets also have z union and z inter store. Or z inter there's just the store equivalent mostly because the reds commands are kind of waiting. There's a lot more of those reds commands along with the really neat interactive tutorial on reds.io slash command. So you can actually go in there on each different command it gives you a little tutorial and examples and you can type around and mess with it. We'll buy reds instance if you're acting right there. And believe me you can try to delete it So as an example of things that aren't kind of normal with reds is sorting keys. A lot of the questions I get when folks are trying to figure out okay how do I take this accurate record thing and model in reds the first thing I say is no and then I say well maybe you should look at this to kind of understand how to model your data. So reds has a store function that is pretty obvious in the way it works and then it kind of throws a curve like you. So let's say I have a set of user entities so I've got a set of five things it just has a bunch of numbers that I've tossed in it that represent users in my set and yes it's an order that I don't know why it just is. So if I want to sort that I just do reds.sort So you can sort that you can do it descending or ascending you can also do it with alpha so you can sort it by another index so you can kind of treat it as a separate index to pull out. So if I want to basically sort things by you can call it as the index I really don't think it is but you can basically say okay for user ID 133 I've got 300 tweets so he's the first one to come up and user ID 42 has 200 tweets so he's the second one to come up so you can basically treat these other keys as this kind of separate thing so you can sort by and that's really neat however it also gets even crazier because you can toss in a get option. So instead of just returning that original user ID list you can also retrieve an entirely different key all together it's going to just like it does with a buy option is going to replace that wildcard with the values so user ID 42 has 300 tweets he's first in the list and chat is the first result to get back since I'm doing a get with handle star and so on and so forth for Joe and Matt. So it looks like he's got a few minutes left right? So I'm going to I have another example here but I think I'm going to skip it and get interested in questions if you want to learn more about various transactions which are not at all I think at all like SQL transactions which you guys have any questions for me? No those are a bit fast if you want to learn about how this works and how to deal with the single friend of nature of us Thanks guys