 Alrighty, welcome back to 105. So the second last topic of the course is sorting. So why are we doing sorting? Well, one of the most common problems you'll probably encounter throughout programming, you don't really have to solve it yourself, but it's good to know how it's done because it will come up and you'll need to use it. So you should probably understand what is going on. So computers are really, really fast, but they don't really care about the order of the data, whatever it needs to present it to a human, such as yourself. Probably it should be in some type of order, so it's easier for you to read. And so today we will discuss several sorting algorithms throughout the lecture. And if I haven't used the word algorithm before, algorithm, just a set of steps to solve a problem. So we just break it down one step at a time, typically in more or less a programming language like C. So here is our problem for today. So we may need to sort an array. So we are given an array such as this int array that has four elements in it. So our array has the elements and they'll be in order of their indexes. So start at index zero and go on. The actual values are two, five, three, one. And we want to write a function that will sort the array in ascending order. So we can go in either order. So we'll go in ascending order, which means the lowest number should come first, then the next lowest, then the next lowest, and the next lowest, and so on and so forth. And the largest number should be at the end. So after sorting, the elements of the array should be in this case one, two, three, and five. So they should be in order. So given that this is our problem today, let's go ahead and take a minute to think about how to sort it using C, trying to explain it to a dumb computer. So essentially everyone probably knows how to sort if you're given a deck of cards or something like that. But how do you really break it down? So computers, they are essentially like telling a two-year-old what to do. They can't do many things at once, and what they can do is pretty simple. So computers can only compare two elements at once and tell you whether or not one is greater than the other one or less than whatever you want. And then the only other operation we should need for sorting is just swapping two elements in an array and we wrote that before just to swap two elements. So let us think about this for a minute. So given an array such as that, how would you explain it essentially to a two-year-old called C how to actually sort the numbers in the array? So give you a minute to think about that and then we will dive into some ideas other people have had and explore them, evaluate them and see how they would work. All right, that's about a minute. So anyone want to give me any ideas how to sort? How you explain sorting to a dumb two-year-old? Yeah, yeah. So I compare the first element of the array with the second one, see if they're in order or not. So in this case I compare two to five and in this case they're in the right order if I want the smallest to come before. And then while I can do that to the next pair like three and five and compare them in this case, okay, well they're not in the right order so maybe I swap them so that we have three, five and then I could go ahead, compare the next two and then they're not in the right order so I should swap them again and make sure it goes one then five and then I do that whole process again so I scan the whole array again, make sure that see if they're in the right order, if they're not, I swap them and I keep on doing this until eventually maybe I might have the idea where if I scan the whole array and I don't have to make any swaps that means it's sorted, so I'm done. So if I don't have to make it, if I'm just checking if everything's in the right order, if every single element is in the right order then that means I am done. So that idea is actually given a name and it's the first sorting algorithm people typically learn, it is called bubble sort. So that is that idea. So it compares the first two elements of the array index zero and index one. If the first element, since we're doing ascending order if the first element is larger than the second I swap them and now I compare the next two elements so index one and two, swapping them if I need to until eventually I compare all the elements of the array and then while the whole array might not be in order but I'm kind of guaranteed that the largest value is now at the end. So now I can just repeat these steps until the array is actually sorted. I would just repeat these steps until the end of the array and then the larger values go towards the end in order. This is also given other names, it's also sometimes called a syncing sort because numbers kind of very slowly go to their right direction. So if after I scan the whole array once things are a bit closer to being sorted. So now I just repeat the entire process again until the array sorted and the array sorted whenever I scan the entire thing and I haven't made any swaps. So how would that look like with that little array? So in the blue boxes are the values and below them is their index in the array. So this is what we were given initially and how bubble sort would work is, okay, we compare index zero and index one. In this case, well, five is greater than two so they're in the right order so we can leave it alone. Then we look at index one, index two, oh, three is less than five so they're in the wrong order. I should swap these two around. So we swap them around. Then I look at index two and index three. Okay, these are in the wrong order so one is less than five so I need to swap them around. And now after my first pass this is what my array looks like. So not quite sorted yet but getting closer. So now we just do it again. So we swapped something so we just need to do it again. So now we look at these two. These two are in the right order so we leave them alone. Now we look at the next two so index one and index two. They're in the wrong order so I need to swap these around and then finally I look at the last two and they're in the right order so I don't have to do anything. So I've made at least one swap so I know my array is not in order so I have to do it again. So now I look, do the exact same thing. Look at the first two again. They're not in the right order so I swap them and then the next two are in the right order. Don't have to do anything. Next two are in the right order. Don't have to do anything. And then if I re-scan the whole thing again I wouldn't have to do any swaps because now it is in order. So any questions about that? Cool, let's implement it. So how I would implement so remember C you just have to explain it step by step so if I wanted to translate that into code I'm essentially doing all of these steps while I have no swapped elements so maybe I make a variable called swap and then I put it in a do while loop because I need to check everything at least once. So I should do this while I have swapped something and then initially maybe I'd say swapped is equal to false because I'm keeping track of whether I swapped something in this iteration of the loop. Now my for loop, I have to inspect every single, well not every single pair but I'm comparing like element zero to one then one to two, then two to three and so forth until the end so that tells me well I probably need a for loop so my loop should go from zero and not to all of the array length because I'm going to compare two elements so you have a choice you can either start at i equals one and go to the end or start at i equals zero and go one less than the end because I want to compare the last or the two elements beside each other. This case I'll start off with i equals zero and then go to array length minus one plus plus i and I'm doing that because essentially I want to compare array of i to array i plus one. So if I just left that at array length well then I would go one past the bounds of the array if I added one to it and that would be invalid. So now this is the check where I'm comparing the two elements with each other so the first one so I can check I want the condition where they are out of order so if they are out of order that means the element i is greater than the element i plus one so that means the element to the left is greater than the one to the right which means I need to swap them. So if I need to swap them I wrote the swap function which takes a pointer so we actually swap the values so I can swap array i and swap the address of array i plus one or you could use pointer arithmetic or do whatever you want and then I would just set swapped equal to true because I've swapped at least one pair of elements in this iteration of the loop. So does that look good? Looks like does that look bubbly enough to be a bubble sort? I don't know why I described it like that. So if we compile it that's pretty much all the steps we need to, right? So just keep on going over and over again until eventually we have not swapped anything. So if I run that I see that hey, here's my array before, here's my array after, it is sorted, joy. So any questions about that? All right, let's try and think of some different things because well we can ask our good friend Obama if this is a good idea. What is the most efficient way to sort a million 32-bit integers? Well, I'm sorry, maybe you should do it then. No, no, no, no, no, I think the bubble sort would be the wrong way to go. All right, bubble sort's wrong way to go. All right, thanks Obama. All right, so we'll try something better. So here is our implementation. So let's have another idea. So that was kind of silly. It was fairly simple to implement. I just checked if it was in the right order or not, but we know how to do some other things with loops, right? Oh, and then question, what happens if we loop at the number of times there are inputs? So yeah, if I go back here to this implementation, there's different ways to do it so I could optimize this a bit because every time through the loop, the largest element is the one at the back so I don't really need to check the back to and I know I have to do this a maximum number of times that are elements in the loop. So you can change this code a little bit if you want. This just has an optimization where it will stop if it's already sorted but it will check like the largest numbers that are already in the right place that won't get swapped. So yeah, there are some slight tweaks you can do to this. But if we just get rid of that idea and try something else, well, we've written loops before so why don't we just find the smallest number? So if you can find the smallest number, you can put it at the beginning of the array and then we can think, I don't know, it's like a little bit of recursive thinking so now if the smallest number is at the beginning of the array, well then to sort the rest of the array I can find the smallest number in the rest of the array then that should be the second number in the array and then if I look to find the smallest number in the rest of the array, that's the third and then so on, so forth. I just repeat this process until I've gone through all the elements so every time through I'm finding the smallest element and then I'm putting it in its right place. So this algorithm is also given a name and it is given a fairly literal name it is called selection sort because I'm selecting the element I want to put into its place. So just so we can go through that we can see how that would work. So say I have a larger array of six elements they are out of order so it goes 9, 5, 18, 8, 5, 2 and if I do this using selection sort well I can just scan this entire array so if I scan this entire array what's the smallest number? Hopefully two. So I can scan the whole array it is two and then I can compare the smallest element and see if it's already in index zero so index zero is the first one so I can just say okay the smallest number are you at index zero? Oh no you're not well I should probably swap you there. So now two is the smallest index I don't have to worry about where two is I don't have to look at any more. Now if I look through this and try and find the second smallest well the second smallest is five so we compare five maybe we can pair five against five so in that case well it's already five it doesn't really matter it's already the smallest number and then for the rest of the array from two onwards the smallest number is five so if we compare that to where it should go well it should go in index two so we should swap 18 and five so we swap them and now we know for sure that the first three elements are in the right place and then we do that for the next one so excluding this the next smallest would be nine so we don't need to swap them and then we look at the next two and then while that's it we need to swap them and then the array is sorted so yeah every time through the loop like we're just looking for the correct number to place at index zero and that correct index would be just or the correct number to place there would be the smallest number in the array so if we were to implement that I'll just show you the code for that although I may have missed time this lecture this may go quick so how we would write that is well we need to figure out the slot for index i all the way up to essentially everything except the last element because if there's only one element left while we don't need to find the minimum it's already in the right place all the other elements are placed correctly so inside this loop we're trying to find the minimum index and we can just say initially it is equal to i so we can just say that we're trying to place i so we'll set min index to i and then scan the rest of the array so we can write another for loop that goes from j of i plus one all the way up to the array length so we're scanning every other element and then we check if the element is less than our current minimum then we can change the minimum index equal to here because this is the smallest index we have found so far and then at the end of the loop we can compare the minimum index we saw to i so the first time through this this would be i would be equal to zero so smallest index compared to i and if it's not equal to i so the smallest number isn't at index zero then we go ahead and swap it so we swap the right element to the right place otherwise it's already present there and it's already sorted so we just keep on doing this over and over again until we have sorted every element just like that example so any questions about that code and let us make sure it works so if i run the selection sort i'm just printing it off here let's make sure i print it off before and after just to make sure i didn't cheat so before my array is nine five eighteen eight five two and then after it's actually sorted we got two five five eight nine eighteen and then yeah so we have a question here so why do i start this loop at j or start j at i plus one well if i go back to my example here oops so if i go back to the example here i start my min index at zero and this is the smallest one this is the index i want to replace i'm trying to find the index of the smallest number in the rest of the array and initially i say that the smallest index is equal to zero so i start off comparing everything to nine so that's why i started at i plus one so the first time through that inner loop here let's go back so the first time through this inner loop j would be equal to one so i'm first comparing it against the element at index one and then in this case the value there five is less than nine so the minimum index is now equal to one and then if i look and i scan okay eighteen isn't less than five eight isn't less than five five isn't less than five two is less than five so min index would be five so min index would be five and then i is zero so we're comparing these two things against each other and then determining whether or not we need to swap them so if i found my minimum index was at zero that means the smallest number is the first one and i don't have to swap anything because it's already there alright, any other questions with that? oops alright, let's do a that was bubble sort oops, not that okay, so throughout all of that we kind of had an idea where we had like a sorted part of the array and an unsorted part of the array so the first time through we knew the smallest number was at the start so we could ignore it from then on then the second number was at the start so the second lowest number was the second element so we can ignore the rest of it and then we went so on and so forth so that brings us to our last idea so we kind of divided like i said we kind of divided the array into two parts like an sorted part an unsorted part so give you another few minutes to think of another method of sorting that uses the same idea so you might think if you've done this or if you've played what the hell is that game, Uno or like any playing card game where you have a lot of cards in your hand you probably had them sorted in some type of manner unless you were just insane like one of my friends that just didn't sort them and just looked at all of them because i don't know i assume that's known in this class since we're all in engineering i think they had like i don't know, they were probably like an art student or something like that i don't know but think of it how would you sort playing cards in your hand if you really had to break it down and especially if you already had your card sorted and i gave you another card so give you a minute to think about that and then you can tell me what your thoughts are with that because yeah we should probably listen to Obama and not do the bubble sort do you want to have any ideas there's one thing in discord well let's see if anyone in class has any ideas so yeah, Uno or card games or whatever they're already sorted, i give you another card what do you do? do you do you bubble sort it? do you start comparing against any other well i guess bubble sort wouldn't really work in that case unless you just placed it randomly in your hand and then did a bubble sort that would be weird so what would you do? how would you keep your hand your card sorted yeah yeah so you have your pile of sorted cards you get a new card you just look do you go there there there there there like you figure out where it goes and it's already sorted right so you just figure out where it goes so same idea and yeah so someone else said with Uno cards they're already in sorted order you just go like i have a specific order it always goes what was it blue yellow green red and then the numbers so that everything's always sorted and you just pick a card put it in its right spot so that is the idea for the last algorithm of the day so you can if you just have all random cards in your hand well you keep all the sorted cards to the left unsorted cards to the right initially everything's to the right you take a card from the right and then you just put it in its correct position right so everything's sorted on your left if you have all of your cards already sorted you get one unsorted card you just figure out where it goes you put it in sorted order right so this algorithm well you insert a card in the right place or you insert the element in the right place so this algorithm is curiously called the insertion sort so it uses that idea so we try and keep an unsorted part or a sorted part of the array and an unsorted part of the array so let's go over how that this would work with insertion sort so here are another six elements so we got nine, two, six, five, one, seven they are not in order and we have to go ahead and place them in order so initially I can say that well if I just have one card on my left hand side so just with a nine one card by itself is sorted because you don't have to compare it to anything so I can consider nine as my sorted side and the rest of it as my unsorted side so I might take element two and then figure out where it goes so I'm taking element two and I'm figuring out where it goes and it will either go in index one or index zero so what I would do is I would take that element two look at the element right before and then decide whether or not I need to shift this one over to the right so since nine is greater than two that means my number should come before it right so two should come before it so I need to essentially just shift nine over one space so I can make room for this two if I need it so I should yeah so really subtle so I take that two I don't like I'm keeping it in a variable called element because I know what I'm placing I'm trying to find a place for two so I can go ahead I know nine is larger so I need to shift nine over one so I copy nine over two element zero or element so I index one and now I'm at the beginning of my array so this must be where two goes right so now my sorted side is here are these two numbers they are sorted at index zero and index one so the next thing I am going to examine is element six so I can compare element six with whatever so it came from index two so I start by comparing it to one so in this case the value at index one is nine so that is greater than my number so I need to shift it over one to make room right so I shift my nine which I'm currently looking at over two index two and then well now I have to look at the next lowest number I would look at two and be like oh two is actually smaller than six so six should go exactly here right so six should go at index one so I would put six at index one so now my first three elements are in place now I have to figure out where element five goes okay well I compare five to index two which is currently nine nine is greater than five so I need to shift it over to make room for my five so I would copy it over to index three then I look at the next index index one its element is currently six has a value six which is greater than five so I need to shift it over one to make room for five so at index two I make it a six and then I could look ahead and look to see oh okay well two is smaller than five so that means five needs to go right here right so then I place five there so everyone with me so far so we got our sorted side on the left of that red dot so our first four elements are my sorted side the ones to the right are my unsorted side kind of sucks once you have to break it down and see right so if this doesn't illustrate computers are dumb computers are dumb right you could have probably sorted this a long time ago because you are smart try doing this a million times yeah that kind of sucks now we got two more elements to go so now we're looking at element one oh this one's going to be fun because it needs to go all the way to the end right so whole process is going to go again so we look at index three okay the value at there is greater than one so I need to shift it over then I look at the next index so index two six is greater than one I got shifted over look at the next index the values five it's greater than one I need to shift it over then I look at the next index two well that's greater than one so I need to shift the two over and now I'm at the beginning I have nowhere else to go so I know the one needs to go at index zero right so now it looks like this so good so now our sorted part is now to the left of the red dot unsorted to the right of the red dot we're almost done so now finally we have element seven so we have to look back at the last element from that so it's nine it's greater than seven so we need to shift the nine to the end and now seven well that's uh greater than sorry yeah seven is larger than six so it's in the right spot I should just put the seven here and then mercifully we are sorted all right we're good ish let's see how good we are because we need to implement it do do do insertion sort so now we get to implement insertion sort so we can use that idea so we need to we can start at i equals to one because we're initially going to assume that well index zero is already sorted right we don't have to do anything with it so we'll start at index one and then let's think about how we actually go ahead and implement this thing so I'll give you a second to think about it and then we can go through it quickly or not so quickly and I'll give you a hint so the first thing we probably want is to create a variable called element like I had on the slide that just remembers the current value at that index that we need to save because we might shift things into this index right if it's if it's greater than this element do you want to have any thoughts of what I should write yeah okay yeah that's a good start so I need to essentially what I was doing before I was starting at index i and then looking backwards right so if we think about our loops I want to go backwards I'm not sure how much I want to go backwards because there's some conditions that might stop me if I found the right place I need to stop so I probably need a while loop of some type so I could just put a while here while I'm thinking about it and then I know each time through I probably want to decrement j so how I might think about it is well another thing I probably want to make sure at least that j is greater than zero so I don't break things because if I need to shift things what I want to do that I might think about is to shift things I kind of want to copy make array at j equal to array at j minus one so when I do that that's when I kind of shifted that nine over one element right so I just copied it and since I'm starting at i that's why I need to save this element because well I just overwrote where it was in the array I have to remember it so that's why I remembered the element so we have a j greater than zero because well if j is not greater than zero if j is equal to zero then zero minus one is minus one so if you take an index of minus one of the array bad things are going to happen so I probably need something like that so I'm kind of trying to find only go as far as I need to to actually to actually place this element so the other condition is just if array at j minus one so whatever I'm currently looking at if I look back one index and that one is greater than my element then that means I need to shift things right because if it is less than or equal to then I can go ahead and it's already in the right order so this would be my condition which might hurt a bit to think about you might have to try this on your own and play with it and there's other ways to write this but this is one thing I could do because now if I break out of the loop then j is the index where the number should go so I could just simply write I could just simply put at index j I could just put the element there and that should be it unless I'm off by one so greatest thing the programming is we can just run it so if I run it my array is now sorted so might hurt a bit to look at there's other ways to do this so yeah there's some other conditions here so at the end of this loop well j and i could be equal to the same thing so if the element is already in the right spot so I look backwards one and it is actually less than then I can go ahead I can skip it and then I would just be reassigning element equal to j so if I want won't change anything if I just do if j does not equal i doesn't really hurt anything I'm just kind of optimizing things so that I just don't reassign the same value but if I do that same idea still works all right so any questions while that is up I'll leave it for a minute because it might hurt for a little bit but basically doing those same steps we did when we did it by hand so we remember the number we look we remember like it's index so it starts at index i which we put in j so we look at the number before see if we need to shift that number into the current index so if we do that means while the number is greater than our element so I need to shift it over to the right by one and that's what this does so it takes that element which is greater than this and then shifts it to the right because well j is greater than j minus one that's just shifting it to the right one and then I check the next index and I go down and then I also have to check that j is greater than zero because if j is zero that just means well I'm at the beginning of the array and I need to shift it there yeah and then there's a question so is this algorithm computationally the same as selection sort since for selection sort sample size is reduced with time and insertion sort it's enlarged with time so computationally they're about the same in general on the average case but insertion sort due to how computers work which you won't know until you take like an operating system course or something like that insertion sort is generally makes computers happier for reasons I cannot explain to you yet that will not that will take me much much longer than this lesson so I will leave you with this might hurt so try it yourself it's not that much code or if you want you can try writing it yourself I think on the exams they probably if they do have you to write a sort you don't have to write the most efficient thing so bubble sort was probably the quickest and easiest to write so you could just write a bubble sort but for your own programs if you want to make things go faster probably want to do something like insertion sort so yeah funny we got that question about computationally the same so we can take a slight detour to computer science land so computer scientists they evaluate algorithms and runtimes using something called a big o notation and for performance they just care about how many operations an algorithm has to do they don't care about how long each operation takes and big O is just the relationship between like the number of elements so in this case the number of elements in the array and how many operations you need to do so you can think of it is it is a function that takes the number of elements as input and tells you essentially how many operations you have to do and that's the formula of it so for large problems they don't really care about constant values because as you'll see the kind of scaling factor will just dominate everything so the sorting algorithms we implemented today they're all something called big O of n squared so it's n times n because we had four well we had essentially two loops right we had one outer loop one inner loop and they both ran proportional to the size of the input so maybe sometimes we didn't scan the whole list but if we had double the amount of elements we would have to do double the work so like I said constant factors don't matter so for every time we ran the loop which is n we had to run it n times so it's n times n or n squared and why is that an issue well first of all problems doesn't look that bad so I plotted a bunch of common run times for algorithms so on the x axis is the number of elements which typically everyone represents using n and then the number of operations which is just the result so today we got n squared so that says red line here which seems worse than the other ones if number of operations is how long it takes then there's a few others so there's like n log n there's on so that's just like a single for loop and then there's log n which we'll get into later right now that doesn't seem that bad right it's like it's not eyeballing it doesn't seem that much slower especially if there's not that many elements who cares which that's why in this course all your problems are small so you're like in this land so that's why we don't care how efficient your algorithms are because you don't really do that much so and not that much for computers is like thousands it's not that much but if I took those functions and I plotted it over a larger time it looks something like this so this is as you become facebook or google or whatever you have a very very large problem so if you have something that is n squared you are in this land which is much much much much much much much much much bigger than this and this isn't even that big of an n so if I were to increase it even more just you'd see the red line just grow up and up and up and you probably would not see the other lines anymore so just gets worse and worse and worse the larger it gets but today we're in this land we're in the red line so in the next lecture we'll do better so we're going to explore some better sorting algorithms that are not n squared but today we implemented three sorting algorithms that at least got the job done so there's bubble sort which is easy to implement obama didn't like it very much it's not that efficient but you know what can you do easy to implement then we saw selection sort and then insertion sort so with that I will leave it and then be around for any questions so just remember pulling for you we're all in this together