 Hi everyone, so this video we will talk about a third sort which is called insertion sort and insertion sort is a very popular sort in reality. It's also a big O of n squared sort, just like bubble sort and selection sort, but it has a kind of an interesting property that makes it more efficient in practice than in theory, believe it or not. So the way insertion sort works is it's kind of the way that you probably sort a hand of playing cards, right? So that's why it's sometimes called the card player sort. So the idea behind insertion sort is that you kind of like pick an element out and you keep everything below it in your hand or in your list sorted, right? And you shuffle things or slide things along and inserting items or moving them to where they need to be, okay? So let me just demonstrate it for you. It's probably way easier to explain it this way, all right? So here's some cards and I want to sort them according to their value from least to greatest, right? Now imagine these are in your hand. Go get them. If you've got cards, go get them and put them in your hand. It'll make a lot more sense. The algorithm basically works this way. We're going to keep track of like the next element that we're going to sort into its right spot, okay? Or at least shuffle down, okay? So we start and you imagine these in your hand. I actually have giant playing cards but you know that's not going to work in this environment so I'll just do it here in PowerPoint for you. So you've got these cards here, right? And what we'll do is we start, we've got a variable here called I and we start it in the second position, right? And what you say is, all right, let me make sure, let me kind of rearrange this two to get it where it needs to be relative to everything in the list before the two, okay? So I pick out the two and if I'm holding this in my hand I like just slide it down like I'm going to look at the six next to it and I say, oh, this two is less than, you know, this two here is less than the six. So let me put it in order, right? That's what we're going to do. So this two is less than the six so I compare it to the item to the left. And when I see that the item to the left is bigger than this one I've pulled out I slide it into its spot, okay? I copy it over, I shift it. And now I'm at the beginning of the list so I'm going to wind up putting this guy here, okay? So now these two guys are sorted relative to one another, okay? So now I move up and I come to the nine and I pull the nine out, all right? Well, and I look at this card and I see, oh, this card is already less than nine. So you know what? I don't need to do anything. It just goes right back where it was, right? I don't need to compare the nine to the two because I know that these guys are already in order, okay? So I do a comparison but then I stop. I don't have to move this, all right? Let me move up here to the queen, all right? I move up to the next card and I pull it up and I say, all right, do I need to slide this down? And I see, no, I don't need to slide it down because it's bigger than the nine. And since the nine, I know also that these things are sorted relative to one another. So I don't even need to look at these guys, right? I pull the king out, same thing, right? King has a higher value than the queen and the deck of cards. So I'm not going to slide him down anywhere. He just goes right back where he is, okay? Finally, the seven, ah, now I need to rearrange something, right? So the seven is less than the king. So I should have moved this marker. The seven is less than the king, so I slide the king up. The seven is less than the queen in value, so I slide the queen up. The seven is less than nine, so I slide the nine up, okay? And the seven is not less than the six, so this must be where the seven goes, right? That's the card player sort, that's insertion sort. You're doing some comparisons and you're sliding down as you need to, right? So do it with the deck of cards in your hand if you have one or money or something like that, and it kind of has a little more weight to it. Let me pull up the handout for this. It's a little less detailed and a little less tedious than the other handouts that we went through, right? So here's our algorithm up here and this is what the code would look like. So let's start through this thing. Let me make it so that these will pop out red, okay? So the next element, we start out in the first or slot one, right? So we start with the second item here, this guy here. That's what we're looking at first, okay? So the next element that we're looking for is number six. That's who we've just pulled out of the deck and we ask ourselves, is six less than 12? Yes, six is less than 12. So we slide the 12 from this position to this position. What does this look like in the code, right? We start out with at i equal to one, which is here, right? Next element gets a sub i and position gets a sub i. While position is greater than zero and a sub position minus one, so this slot is greater than the next element, move it, slide it up. That's what this little line of code does, slide it from here to here and then walk down here and do your comparison, okay? So when we end, it's gonna look like this. We've done one comparison here. We've just compared six to 12, right? And we've done one move. We slid the 12 from its initial position here up a slot, right? And then we also moved the six itself, and the six got moved down here. So we did two moves, all right? These things we haven't touched yet. We haven't gotten there. This outer for loop is counting up. It's moving the next element up. So we're going from left to right in your hand of cards, right? So the next element is gonna be at position two, so 22, okay? So we start here, position equal i. And we ask the question, right? Is a sub position minus one greater than the next element, okay? So this element is 12 greater than 22, well, no, of course not, okay? So what do we do? This is the important point. This is the optimization. If the item in the, just to the left of the one we're looking at, is less than or equal to, we don't do anything. We don't move, we don't swap, okay? So we stop. We stop. There is technically, I guess you would say there is technically a swap here, because we're basically reassigning this value to itself. So we can, we'll count that as one move. That's okay, but we only did one comparison, too. Maybe we should say, all right, if you're only, if you don't actually move the element, if it doesn't change the slot, let's just, no, we'll continue to count it as a move, all right? But these guys never even got looked at, right? Because they are sorted, they're already where they're supposed to go. All right, next element we're looking at is 14. Is 14 less than 22? Yes, it is. So 22 moves here, all right? Is, now we're in this slot, is 14 less than 12? No, we're here, okay? We're done. We did two comparisons. We compared 14 to 22 and 14 to 12. And we moved 22 up a slot, and then we put 14 in its place. So we did two moves. These guys don't move. This is the optimization. We're not messing around over here unless we really need to. All right, next slot, the value is eight, okay? Is eight less than 22? Yes, we slide the 22 up, okay? This is the state of the list. Is eight less than 14? Yes, we slide the 14 up, all right? Is eight less than 12? Yes, we slide the 12 up. Is eight less than six? No, so the eight must go here, right? And the six stays here. How many comparisons we do? We compared eight to 22, 14, 12, six. So we did four comparisons, and then we also did four moves. We slid 22 up, 14, 12, and then we put eight in its spot, okay? All right, final element. We're gonna go all the way to the last slot here, right? So the last thing we look at is 17. 17 less than 22? Yes, it is. So we slide 22 up. Is 17 less than 14? No, so we stop, right? 17 must go here. And all these guys, we already know. They must be in the right position, because we've already sorted that, basically that half of the thing. So we did just two comparisons here and two moves, right? Okay, so all said and done, we've drastically reduced the number of comparisons we've had to do compared to selection sort, right? Both selection sort and bubble sort, we had to do a lot of comparisons. We were guaranteed that we had to do, we absolutely had to do n squared comparisons or n times n minus one over two comparisons, right? We had to do it, not n squared, I apologize. Here, that's not necessarily the case. We've got this optimization in here, this guy right here. So, but we do potentially need to move more often, right? We do potentially need to shuffle more things forward. All right, so in the worst case, how many comparisons must be made to place the item at position i? Well, in the worst case, right? You have to make i comparisons. In the worst case, right? If we need to position this guy, in the worst case, he's got to go one, two, three, four, five, we've got to do five comparisons. That's the worst case for any particular slot. If we're here in slot two, in the worst case, we have to do one, two comparisons, okay? So, we have to do i comparisons. Given that, and given that we start at position i minus one and go to i equals n minus one, what is the worst case number of comparisons we need to perform? Well, it's this, this is just the sum, right? Because we're starting here, worst case one comparison, worst case, excuse me, worst case one comparison, we start here. Two comparisons is the worst case, three comparisons worst case to move this item, four, five comparisons is the worst case to move this item. You get the sum here and the series sums to this value, okay? Looks familiar. So the worst case though, the worst case number of comparisons is this value. Not the guaranteed case like it was with selection sort and bubble sort. But given that, of course, the worst case, what is the big O of insertion sort? It's n squared, right? This is an n squared algorithm in the worst case. All right, what additional elements do we need? We need something for i, next element, the position variables. These are all parts of the algorithm, right? Scroll up here and see. So not a lot of extra space, just a couple of little extra variables. Can you perform insertion sort on a Python list? Yes, definitely. But it looks very different on a linked list. You need a lot of helper methods and helper pointers. It's a little tricky, but you can do it. What initial state of A would require the most comparisons? So I want you to think about this for a minute. Think about how insertion sort works. What would be the worst case? Right, what would trigger the most comparisons? Well, go back, kind of think about it. Think about how this algorithm works. The answer is if the list is in reverse order from its final sorted order, right? Because in this case, you're always finding, yeah, I need to keep going left, right? So if you give a reverse ordered list to insertion sort, it's gonna be a big O of N squared algorithm. On the other hand, if you give a list that is already sorted to insertion sort, it will be a big O of N, or it will have N time complexity, right? Because there, thanks to the optimization right here, it just loops through the items in the list, but it never does this. It never, this inner loop never triggers, right? So effectively, it just loops over the items in the outer loop and it never moves anything. So it becomes a big O of N algorithm, which is pretty neat, right? All right, so this final question, suppose you have a list containing blah, blah, blah. This, what is the state of the list after three complete passes of insertion sort? So three passes of the outer loop. I encourage you do this trace. This is the kind of thing that I would very much likely to ask on an exam, right? If you know the algorithm, you should be able to trace it, or if I give you the algorithm, you should be able to trace it. So I'm not gonna do that here. We already did some tracing. I will post the solution to this, all right? So that's insertion sort. Have a good idea in your head of the, you know, kind of the general feels for bubble sort, selection sort, insertion sort, how they work. Know if they are stable sorts or not. Insertion sort is stable. Remember, selection sort was not stable, all right? So that's it for now and I'll see you next time.