 Now, before we get to convolutional neural networks, I always think it's a good experience just to explain what a convolution or what the convolution operation is. What does it mean to convolve, for instance, a matrix? And a good way to cheat in the showing of it is just to use a very simple program like Microsoft Excel. So we don't need R. Just going to use Excel, Microsoft Excel spreadsheet. You can use any spreadsheet software, actually, just to demonstrate what it means to convolve a matrix. I have here just two sheets here that I just want to show you. In case you are not familiar with Excel, I just wanted to show you two things. We've got these columns here, and I've got values here, 1 through 10. And I've got my multiplier 1 through 10. And all I want to do is to multiply this value by this value. The way that we go about that is 1 times 10, but I put the equal sign in the front just to tell Excel that this is a formula, and it should go out and it should do that multiplication. Now, that's going to give us 10. What we want to do instead is say equal. I want this value, so I click on it, times, which is shift 8, and that value. And you see the blue and the red, and you see those cells are color blue and red. And it says A2 and B2 because this is column A, row two, column B, row two, where those columns, column values and row values intersect. That is the cell address. And what Excel is really doing behind the scenes, it's not saying A2 times B2. What it is saying is I'm in this particular cell. For this A2, go two cells to the left, multiply, go one cell to the left. So even though it says A2 and B2, that's really not what Excel is saving behind the scenes. It is saving what is called an address and it's just referencing those cells. So it's just saying go two cells to the left and go one cell to the left. And if I hit Enter, return, we see it's a 10. One times 10 is 10. But now I can go to the bottom here when the cell is selected and the mouse changes and I can drag all the way down and you can see what it's going to do there. It's going to see two times 10 and three times 10. That is because we have this reference in this addressing. It's just saying go two cells to the left and then one cell to the left of this particular one. And this one go two cells to the left, multiply by whatever is one cell to the left. So it's just this reference to those values. Now we can do something else if we just have this constant multiply here. So I'm going to say equals again this one and it's going to count one, two, three, four to the left times and I'm going to hit this 10 which is just one to the left but I'm going to do something different here. On Windows I'm going to hit F4. Other operating systems it might be something else. And instead of saying D2 there you see dollar D, dollar two. That fixes this cell. So if I enter return I'm still going to get 10 but as I move down now it shouldn't work. You might think it won't work because it wants to move down one cell to the left of this one but it sticks to that 10. That 10 doesn't move because of these dollar signs that we have there. So if I drag that all the way down I'm going to get exactly the same values as we did there. So that's this constant referencing by just fixing that address. Fixing that address let me hit F2 there. You can see A2 times dollar D, dollar two. Let me hit escape. I'm going to go to this one. I hit F2 and you can see the blue and the red. It is still saying goes one, two, three, four cells to the left of where I am multiplied by this constant thing that doesn't move. And the dollar in front of the D means the column doesn't move and the dollar in front of the two means the row, the row two here doesn't move. So that is how we do this multiplication. That's how we do this multiplication. Let's just go there again. Let's hit F2, there we go. A3 times, you see there the A3 times there. And just for argument sake let's go down there and hit F2 and again you can see this one came down the one, two, three, four to the left of where I am but this F10 stays stuck. So in case you knew this, that was a waste of five minutes of your time. And let's go to some product now in case you don't know that. Let's do this. Some product means it's actually the other way around which should be product sum but it's actually some product. We're gonna do product means multiplication so we're gonna multiply all the values and then we're gonna add them up. So let's do the exact same thing. I'm gonna say two to the left times one to the left and that equals 50, two times 40 is 80, three times 30 is 90 and then 80 again and then 50 again, that's what I should get. And that's indeed what happens. We can see there if I hit F2, it is those two that are multiplied by each other. So it's the product of those and then I want to sum all of these and in Excel it's very easy if they're all highlighted I can come across to there and just say auto sum and it'll sum all those values 50 plus 80 plus 90 plus 80 plus 50 gives 350. But I can do it all in one go by this function called sum product. So I'm gonna say equals sum product. It's gonna double click it and accept it and I'm gonna say take all of those so I highlight all of those, comma and I'm gonna highlight all of those. So it's gonna say one by one in this selection multiply one by one in this one. So element wise multiplication. This element times this element, this element times this element, this element times this element, this element times this and in the end add all of them up. Let's close the parentheses, hit return and learn behold 350. No magic there. So just wanted you to know these two things before we go to this convolution. So this is what it's all about. What does it mean to convolve a matrix because we've got a rank two tensor here, six by six. Another rank two tensor here, which is a matrix. Six by six, six by six. One, two, three, four, five, six and one, two, three, four, five, six, six by six. And we're going to convolve that with this three by three kernel or filter or whatever textbook you're reading or lecture you go to or videos that you watch is this three by three matrix that we're going to multiply. And what we do is we're going to multiply it by this first three by three. And if the stride is one, we move this little window one up. We're going to move it one up and do that element wise multiplication and then adding all those values. So some product as we saw before and then it moves slides on and slides on and now you can see why there can only be four. That's one, two, three, four. It can't go anywhere else. And then one down, one, two, three, four and then one down, two, three, four and one down, which means we four down now and two, three, four and we end up with a four by four matrix. That's how it works. So let's just do that. And I'm going to say equals some product and we're going to take these first three, comma, and then I'm going to take these three but I'm going to hit F4, shift F4, whatever the case might be because I'm constraining that to those. This is not going to move and I hit close to parenthesis, hit return and then I'm going to slide it one, two, three across so that I have four and then four down. Let's leave it there, four down and we leave it there. So what it's done is it's taken this little window and it's slid it across here as you know that is the convolution operation, shifts it across when it gets there for instance. It means, well let's start with the first one. It's going to be, you see it's three by three and three by three so it's going to be product of zero and minus one, zero and zero, zero and one and then zero and minus one and the next zero and zero and then that one and that one, multiply each of those and just sum all of them up and then we move across. So that's why I wanted to show you the addressing, the relative addressing and then that fixed constant addressing and the sum product. So it's element wise product and then the summation. So what we have here is a very special filter or kernel in that I've got all negative ones on this side, all zeros here and all ones there. It is something that will detect a vertical edge. It'll detect a vertical edge. You kind of see there's negative ones on this side plus ones on this side and there's zero on here. So what's going on here? Imagine this thing that we're seeing here, the six by six matrix. It's just a tiny, tiny little spot on a bigger picture, maybe of the sun shining and reflecting off of a window and the window sill is vertically down and somewhere in this picture there's this yellowy color, maybe that's the window sill reflecting a bit of the sun and then the very reflective glass of this sliding door reflecting the orange setting sun. So somewhere along the line, there is this vertical edge that you can clearly see. So use your imagination. It's a tiny, tiny bit of a picture and there's some vertical line where two colors are split very distinctly here. And the way that I've done this is I've just highlighted all of them and went to conditional formatting colors and morals and just accepted this value. So it'll take the lowest value and color it this way and the highest value color at this yellow and then everything in between shades like that. That's all that happened here. So that's just for demonstration purposes. But if we go here now and I highlight all of these, let's do a bit of conditional formatting here. So color scales, morals and let's just make this that color up to that color just to give it a bit of separation. Now, remember again, this is part of a much bigger picture. So this is just gonna be a tiny bit in this bigger picture but you can clearly see this line down the middle. So it's gone from this part of the picture and we've slid over it by this little very remarkable special little filter or kernel and what it has done is it has demonstrated that there is this clear vertical edge. That's how the convolutional neural network learns initially at least that there's a vertical edge here. Now, remember we can set how many of these three by three or two by two or five by five filters we're gonna send across this and this is going to give us eventually the depth. So we've come from six by six we're convolving with a three by three that's gonna give us a four by four but I can convolve with another three by three which is going to give us a four by four by two then I'm gonna stack the next one behind this and then a four by four I do another one and it's gonna be four by four by three so it depends how many filters I go over here and now you might imagine wonder well, where did this special these special values come from? Well, they're not special at all well, they're special in the way that we created it to show this demonstration but remember these are parameters that are going to be learned so that irrespective of where on the picture we are you know, if it detects something up here if that same thing exists down here it's going to detect it there as well that is the beauty of a convolutional network. Now, those parameters, these six and nine values here I should say they are going to be learned they are parameters that as we go through epochs mini batches and epochs it's going to update these weights and we're going to learn what those are so that these numbers are going to fluctuate and eventually settle on something that is being learned that is the learning process, finding these values now we cheated here because we constructed this because we know that this is the kind of filter that will detect an edge so let's do it this way around this definitely is going to detect some way on your image it's going to detect a horizontal edge so let's do this again I'm going to say some product I like that, double click I should say let's do the first three, comma these I'm going to hit F4, shift F4 to constrain those and then I'm going to pass I'm going to move this along one, two, three places so I have a four, one, two, three places another four and again let's just do this conditional formatting and go down to rules let's click this, choose the same two colors we had before just to show that there we go and you can clearly see that a horizontal edge I'm just highlighting it so that we don't have to stay at the actual values we can stay at the colors and it irks me that we can't see the values there actually so let's just do that we can clearly see that it has learned that there is this horizontal edge there and that's how it learns again this is very special the way that we put it here but it is those are nine parameters and it has to be learned and that is how it learns it eventually is going to settle on something like this which learns that there is this horizontal edge and there's proof of the fact that there is this horizontal edge there and that is the process of a convolution and it is these filters and we stack them up we can say to the neural network in this layer give me a convolutional neural network make it three by three make the stride one make it valid or same so we're going to do padding or no padding don't worry we'll get to all of those but it's going to move across the image it's going to slide across as I showed you it is nothing but a sum product that gets done every time nothing but a sum product as it moves across element by element multiplication then addition moves across moves across moves across moves across moves across and we end up with this four by four on this other side because it is constrained within that position and there you have it the convolution operation very easy to demonstrate here in Excel or in any other spreadsheet software