 Godric Gryffindor's enchanted sorting hat, and I don't usually do intros for my talks because the idea of a bio is to get you into the room and all of you are already here. But this time I'd like to. Hi, my name is Caleb. As you can see, this morning I was at 970 followers. I'd love to hit 1,000, and that means that I need to get 30 people to follow me. And there's at least 30 people in here, so I think we can make this happen. So, if you like what you hear, take out your phones and follow me at Caleb Thompson. Also on this slide, we have Joanne Chang giving a truism, a little bit of information behind the scenes on what it takes to get up on the stage, it's mostly not preparing for your talk. All right, that's out of the way. This talk is not for people who studied computer science. If you know what Big O notation is, great, sit back, hopefully you'll still have a good time, but I might not be teaching you anything. This talk is for people who call sort and have no idea what demons are moving strings around in the array, but you get it back sorted and that's pretty cool. You would like to learn a little bit more, so let's dive in. To understand what Ruby or Postgres do, when you call sort, we need to talk about algorithms, and I know algorithms are a bit of a scary word, but what it comes down to is algorithms are just a set of step-by-step instructions. A recipe for chocolate chip cookies is an algorithm. I didn't have to understand chemistry or gastronomy when I followed the instructions. When I realized that this was just a good example, I just found a recipe for chocolate chip cookies and made a batch immediately, procrastinating on preparing for my talk, and I ate them. Before we get too far, I also want to talk about Big O. This is another scary concept, especially when you see it written out next to some algorithm or something. I usually assume that it's for math people, and I make websites, like DHH said, we're not computer scientists, so it's probably not me, but what it comes down to is Big O is not really that scary. It's how we answer the question, how fast is an algorithm, and how we can take that answer and compare it to other algorithms. It's based on the assumption that the basic operations involved in an algorithm will take the same amount of time between different algorithms doing the same sort of thing. So what it does is allow us to compare using the number of operations as a function of the inputs instead of actual time taken in running the operations. So let's take a look at a couple of examples of algorithms for cutting out squares from paper. When I previously gave this talk, it was pointed out to me that it was a little bit small and hard to see. So naturally the first thing I did was buy a giant pair of scissors and a big-ass easel and a giant pad of post-it notes so that you can see. So hopefully you can see this time. There's a fly up here. Goodbye fly. All right, so each time I end up with another piece of paper, that's going to be one operation. These scissors are terrible. Let's see if I can do this two hands. I had to draw these lines because I kept on getting like really weirdly shaped pieces of paper. This hat, the scissors, who came up with this idea? Fun fact, this is actually the first time I've cut on the board. It's pretty exciting. No. Yay. All right, so we ended up with 16 squares of paper. That makes this algorithm an order of n squared time or quadratic time algorithm, or I'm sorry, linear time algorithm. It's right there on the slide. And so as you can see, the growth is linear, goes right across the screen. If I wanted to cut out 32 squares, it would take me 31 cuts. Now, that's a different number. That's not n. So n is the number of squares that I need. And so it's 15 and not 16. It's 31, not 32. But it doesn't matter because big O is a little bit lossy and the growth is linear. And that's what we're really looking at in big O notation. All right, so I bet you didn't expect this, but we're going to look at another algorithm for cutting out pieces of paper. I'm going to take off this hat. All right, this one's a little bit better. All right, and we still have 16 pieces of paper. If I wanted 32 pieces of paper, it would take me only one more cut, five cuts, which makes this on a logarithmic time or order of log n algorithm. So much more performant than the original piece by piece cutting. This is an example of a lot of different representations that are fairly common for big O. Down at the bottom, in orange and yellow, yellow is actually behind green, we have order of log n and order of n time algorithm. So you can see that there's some very big ones here. And the two that we show, we actually looked at the graphs already for order of n and order of log n. And you can see that they're a little bit smaller. That's because I've had to scale down the y-axis to 10 to 1, just to show how big some of these bigger ones are. So n squared, 2 to the n, and n factorial, very large. If you draw an imaginary line diagonally through this chart, then what you really want to shoot for is the big O notations that are below for a performant algorithm, rather than the ones that are above. Zooming back into 1 to 1, we can see that there's still a big difference even between order of n and order of log n. So we've gotten through all that. When do you update a user record on log n? If you've understood everything so far, I'm really on it today. How do you debug an app? We all know this, guys. You start log n. What happens when you buy something? I promise this is the last one. Well, obviously you own it. This is a dumb question. So we're also going to talk about big O mega. This is going to be a little bit easier to understand because it's exactly the same as big O. Except, rather than worst case performance, we're talking about best case performance. Which is why it has the little tails down at the bottom because it's the lower bound for how fast an algorithm can be. Bubble sort is a nice and simple algorithm. So it's a great place to start jumping in and explaining sorting algorithms. The algorithm is this. For each pair of elements, compare them. If they're in the wrong order, swap them. Then move one over and keep going. Keep doing that until you hit the end of the set and then start over. You don't have to look at the last one because the last one has already bubbled up to be in the correct position. So if you can go through an entire iteration and no changes are made, then your set is sorted so we can short-circuit that way. So my volunteers volunteered folks. Please come over here where the papers are. Your pie. You weren't here so you were unvolunteered, I'm sorry. So if you can go and stand in the line over there with the light in your eyes. All right. I have a cheat sheet. These are our numbers. Can everybody read the little text at the bottom? 5.1, is it out yet? Pie, 3.14159265359. E, 2.18281828. 42, life, et cetera. 9001, it's over 9000. All right. So 42, you're over here. Then 9001, 5.1, pie, and E. So you just shuffle down. Great. We're going to go through the bubble sort algorithm. We're going to start here. So the algorithm is we compare each number to the number to its left. So 42 is less than 9001, so it's in the right place. No changes. The hat moves. 9001 is definitely bigger than 5.1. We're going to swap positions. The hat already moved, so we don't need to move it again. 9001 is bigger than pie, so we're going to swap places, pie or E. 9001 greater than E, so we're going to swap places again. That's the first iteration. Pretty easy. We know that 9001 is in the right place now, because it's been bubbled up all the way. Back to 42. 42 is greater than 5.1, so they're going to swap places. 42 greater than pie, swap places. 42 greater than E, please swap places. All right. This is pretty easy, forgetting it. OK. 42 and 9001, both in the right place. All right. Rails the version we don't have yet. It's pretty big, so it's going to be bigger than pie, and we're going to swap places. And definitely bigger than E, so we're going to swap places all the way to the end. And again, we're not needing to go past what we've already bubbled up. So pie is bigger than E, not better, just bigger. So we're going to swap places. Awesome. That's the second to last iteration. Now we're going to go through this entire set. E doesn't have anything to compare to it. We've made no swaps, and we're sorted. All right. Thank you, friends. You may take your seats again. Go ahead and keep the lunchboards. Bubble sort works in n squared time. For the purposes of sorting algorithms, n is going to be the length of the set. So since bubble sort of operates in quadratic time, that means that we compare each element to every other element, in the worst case. Even though we have some optimization so that you're not actually comparing every element to every other element, like skipping the ones that have already been bubbled up, when we're stating an algorithm's complexity, we still say n squared, because again, that's the growth. It's quadratic growth. For bubble sort, order of n squared is also the average case complexity, such as on a randomized array. It has a big omega or lower bound best case time of n, which means that for its best case, bubble sort returns a sorted set in linear time. So if we had instead of 42, 9001, 5.1 pi e, if we had e pi 5.1, 42, 9001, then we would have just had to go through each person once, check that they're in the right order, and then we would have been done. As we saw before, n squared is not that great. It's this red line here. And again, that's above the imaginary line that we wanted to avoid. It's bad because it means that the number of operations taken is much higher, which generally means more time spent in the algorithm. We want to be in one of these four smaller lines. They go of n log n, they go of n, or ideally, order of log n or order of 1. Now, order of 1 would only work in a perfect world where every set is already sorted, and so the sorting algorithm is, return this sorted set to me, please. It's useful to see a visualization of how these algorithms work on different types of inputs. Topple has this great set of visualizations that show bubble sort operating over different types, different sorted sets. These can help you sort of build an intuition about how bubble sort will work on these different types of inputs. The red arrows here indicate our subjects. Black arrows are sorted, and gray arrows have yet to be sorted. We have random, nearly sorted, reversed, and few unique sets of input data. This takes quite a bit, nearly sorted finished, but we're not going to watch the rest of these takes over a minute, and I'm on a timer. Insertion sort is another simple sorting algorithm. It's very efficient at sorting nearly sorted sets and very small sets. Insertion sort is so-called because it inserts the subject element into the set of previously sorted elements in the correct order by comparing it until it's greater than one of the previous elements and inserting right after that. So the algorithm is this. For each element, consider all elements to its left. Continue until you reach one that's smaller than the element or you run out of elements to check against. If you found anything smaller, move the element just after that place and shift everything else up. Let's take a look. We'll start first with this one, and we're going to compare it to everything on its left. This is a pretty easy step. There's nothing to its left, so we're done. Now we're going to consider the next element. The red element we're comparing to is not bigger than the green subject, so we're not going to move anything here as this subset is already sorted. Onto the next green subject. We'll compare each of these and see that they're both smaller. Next. This one's bigger, so is this one and this one. So we're going to move this to the beginning. Now the first four elements are in order. We'll repeat this until the whole set is sorted. Let's speed through that. It's supposed to be playing Vinnie in. Okay, well you get it. Presentations never work. That's another insider tip. Great. So now we have a sorted set. Insertion sort is a fun one because it's one of the simplest sorting algorithms to write. Like bubble sort, insertion sort operates in quadratic time. And for the worst case, and in quadratic time for the average case. Also like bubble sort, insertion sort has a best case of in squared time or in time. Because of the inherent lossiness of big O notation, this ignores that insertion sort is almost always faster than bubble sort. But it's definitely not a fantastic way to sort a large set of data. It does work very well on nearly sorted arrays. It's already finished here. Because of this, insertion sort is sometimes used as a finishing step for quick sort, which we'll look at next. It shares the benefit bubble sort has of returning an already sorted set in linear time. You can see that it's working much better on the reversed set and is generally faster despite having the same complexity representations than bubble sort was. Again though, it takes a while so we're not gonna watch all of it. Quick sort's often considered the state of the art in comparison sorting algorithms. This is because the average case complexity for sorting algorithm is very low compared to others in the same class of algorithms. It's a little bit different than insertion or bubble sort. Rather than picking something and using it as a subject and moving it based on how it compares to other elements, we're going to pick a pivot, which is the same thing as the subject, but it's a different word for quick sort. And we're gonna move everything else around it based on whether it's larger or smaller. So if it's larger, then it'll go to the right. If it's smaller, it'll go to the left. If it's equal, we won't do anything. They'll stay in the same order. So quick sort doesn't specify what you should use for the pivot, but in general, it makes the most sense to pick either a random or the middle element rather than the first or last elements, and we'll get into why that is later. But let's take a look at the actual algorithm. So we're gonna start with eight. Eight, and we're gonna compare it to everything on its left and right, and then we're gonna move cards around so that they're in the right order. So eight is greater than two, so that's in the right order. Eight is greater than Jack, so Jack needs to move over beyond where eight was. We'll just shift everything down, and I'm gonna go ahead and flip the Jack over and the two over because we've already looked at them, so we don't need to look anymore. All right, so now I need to compare eight to the Joker, but I don't know what Jokers are. Jokers are wild, so what should we use for Joker? Pick a number, preferably one of these numbers. Six, six it is. All right, so six is less than eight, so the Joker's gonna stay where it is. Six is also less than eight, so the six is gonna stay where it is. Now, it's important to note that because the Joker and the six started Joker here and six here, they're not gonna change. That's because a quicksort is a stable sorting algorithm, which means that for values that are equal, they will stay in the initial ordered state. So we have an eight here. Five is less than eight, so it's gonna move over, and we're gonna flip it. King is greater than eight, so it's in the right place. Aces, depending on the game, aces can be lower high, what should we use? High, all right, well then it's in the right place. Queen, greater than eight, so it's in the right place. So we're done with all of these, we'll flip them over. Nothing ever works. All right, eight, I'm gonna just set eight down. That's funny, but it's also because eight is in the right place now, it's not gonna move again, so we're not gonna move it. It's just gonna be out of the way. So now we're gonna start with the lower set, the things that were lower than eight. And again, we're gonna repeat this process, we're just gonna pick a pivot. We're gonna pick the midpoint rounded down, so since there's four elements here, we're gonna pick the second element, which is the Joker. Joker again is six, so two is less than six, so it's in the right place. Six is equal to six, and so it's not going to move. And five is less than six, so it'll move before the Joker. And the Joker goes down, because the Joker's in the right place as well. So again, we're gonna continue this process. There's only two elements here, so we're gonna pick the first one, the midpoint rounded down. Five is greater than two, so it's in the right place. And we'd be done with two. Now we're gonna look at five, there's nothing to compare to, so five is in the right place. Now we go the things that were greater than the Joker, which is just the six. Six is in the right place, so we're gonna leave it there. All right, so now finally we're gonna look at things that were greater than the eight originally. This didn't work well last time, but let's see how it does this time. Okay, so we're gonna pick that king, and that king is gonna be our pivot. Joker, or Jack is less than king, so it's correct. Ace, we chose as high, so it's greater than king, it's always greater than everything else. And queen is gonna be less than the king, so we'll move it over. So the king is done. Now we're gonna look at the Jack and the Queen. Queen is greater than Jack, so it stays where it is, and Jack is done. Now we look at the queen, nothing to compare to, it's in the right place. Same for the ace, nothing to compare to, so it's in the right place. So now we have an ordered set of cards using QuickSort, it's ordered, right? Awesome. So it took forever to do that demo, and so you might not be surprised that big O of QuickSort is also in squared. In the case of QuickSort, however, it's very unlikely that you would actually hit in squared operations. To be in squared, you'd have to always select a pivot that was the smallest or largest element in the entire array, such that the remaining subsets to be QuickSorted are of size N minus one and of size zero. To illustrate that, let's take a look here. So we picked the midpoint as the pivot, which is going to be the tallest bar. So it moves over, we've got a set that's everything except for that pivot, so N minus one, and a set of size zero to QuickSort. If this happened for every single operation, every single time through the algorithm, then it would take N squared, N squared operations to complete. And this is why you don't want to use the first or last element necessarily in QuickSort, because if you handed it a sorted set and it picked the first or last element, then it would always end up QuickSorting in this case where you've got a set of size zero and a set of size N minus one. The magic of QuickSort though is that the average case complexity is linear rhythmic, which is the fancy word for N log N time. So linear time times logarithmic time, linear rhythmic. The average case here is not just intuition, like we said for bubble sort, like it's probably the random thing is probably going to be taken in squared time. There's actually several mathematical proofs that show that QuickSort runs in linear rhythmic time, but they're over my head and out of scope for this talk. Linear rhythmic or N log N performance is also the best case for QuickSort. So while it's not always the fastest algorithm and can be just as slow as insertion or bubble sort with specific types of input data, its average case is the best case and the best case is faster than just about anything else in any other comparison sorting algorithm. For general purposes, meaning integers, strings, whatever, this is the cream of the crop. That said, it does have its weaknesses. You can see here that it does great with the reversed set and the nearly sorted set and the random set, but it's having trouble with the few unique set. Here you can see those same visualizations that we looked at for each type of algorithm all compared together. As the different algorithms work and finish, you can get an idea that some of them are better suited for specific tasks. The random set in the first column does give a fair approximation of the average case and you can see that QuickSort works very quickly there. It also makes short work of the reversed set and which trips up a lot of sorting algorithms. Insertion sort works very quickly on the nearly sorted set. Bubble sort's not really good at anything. DHH was right. Don't use bubble sort. On the other hand, it's very difficult to beat the average case performance for QuickSort, which is why that's what Ruby uses in its sort. I'd like to thank my employer, Heroku, for supporting me in coming to RailsConf this year. You may have heard about the new automated certificate management offering, which gives you free let's encrypt certificates for paid dinos at any tier. This video shows how easy it is. Let's say order of one complexity. You should check it out if you haven't yet. You've seen me speak now, but if you're interested in hearing from other Heroka, you're welcome to check out the speaker schedule we included in the swag bags, which has the listing of talks from Jonan who's next, Richard, Nate, Amy, and Stella. If you have any questions about sorting, ACM, or really anything, you're welcome to come to the office hours at the Heroku booth in the exposition hall on Wednesday and Thursday, where all of our speakers will be hanging out for a bit, usually right after their talks, but the expo hall isn't open today, so Jonan and I will be later. And you can also find me in the hall or ask questions on Twitter because you're gonna follow me, remember? Thank you.