 In this video, the Gram-Schmidt orthogonalization of polynomials. I'm Dr. Jean Clapper. I'm with the School for Data Science and Computational Thinking at Stalingbosch University. So I'm going to talk about how to orthogonalize a set of basis vectors, but those vectors are not vectors in the usual sense. They are polynomials. We all know famous polynomials, x squared, and we're going to look at how to work with that. This video costs in the Wolfram language. So if you have access to a Wolfram language product, feel free to use that. If you don't, of course, you can always go and search for the Wolfram programming lab on the internet, sign up for free account and you can code long right in your browser. Of course, I think when it comes to computer language software, although I would always punt the open source products, I think value for money is also a very important factor in life and I think it doesn't come much more, a much better example of value for money than the Wolfram language. Really for the money that will buy you a couple of cups of coffee a month, you can get access to a very powerful computer programming language. So I've chosen for this video cast, at least the Wolfram language, let's have a look. So introduction, we're going to look at a set of polynomials of degree n. Remember n is then a positive integer. If it's 2, it'll go up to x squared, any polynomial up to the power x squared, something like x squared plus 2x plus 3, that will be within that set of polynomials and from that we have this idea of a vector space. Now we're going to denote the vector space p, uppercase p of n and it's a vector space under the binary operations of polynomial addition and then scalar polynomial multiplication. So I'm going to assume that you have some idea of what a vector space is and also what an inner product is, although I'm going to show you exactly what the inner product is defined on a set of polynomials. Now we're going to consider this basis that we see in equation number 1 and that is the set of polynomials to the nth degree. So 1xx squared x cubed x to the power 4 all the way to x to the power n, that's my set of polynomials and that clearly is a basis for a polynomial vector space to the nth degree. In other words, a linear combination of that set of vectors that we can see here and one that will give me any polynomial up to degree n. So we have to look at our inner product that we're going to define and remember we're going to take two examples, two vectors in this vector space p and q, they're both polynomials then of at most degree n and we denote the inner product as you can see right there with these nice little brackets p and q and that is how we define this inner space, inner product space between these two polynomials, it is the definite integral on this interval negative 1 to 1 of p times q. So just a product of the two polynomials with respect to the variable. Now these variables, these equations, these polynomials of course only have a single variable and we're going to use x such as x squared, x cubed, etc. So you just multiply the two polynomials with each other and then you integrate that over the interval negative 1 to 1. That is how we define the inner product in this instance that we're talking about now. What we also have to talk about is the norm or the length or the magnitude of a polynomial. Now they are correct terms to use but let's just say norm of a polynomial and such that if we take any polynomial p in this nth degree polynomials of ours, the norm squared is just the inner product of the polynomial with itself. So if we look at 2 again, that's just going to be p times p, the product of the polynomial with itself and if we take that definite integral on that interval of negative 1 to 1, that's going to define the norm squared. Of course we take the square root of that, then we'll get the norm. And then very importantly when we talk about Gram-Schmidt, the Gram-Schmidt process, if you don't exactly know what it is, if you want to refresh it, there's also some links down below where I use Python and where I use Julia just to go through the steps of the Gram-Schmidt process to orthogonalize a set of column vectors in a matrix. And this is what it's all about, equation 4, you've got to know this one. Now I'm putting use and a's in there, you read different textbooks, you're going to see different notation. So if we scroll down, let's just a little bit to 5 here. I've got my set of vectors, basis vectors for this fourth degree polynomial vector space and I've just labeled them vector a sub 1, vector a sub 2, vector a sub 3, all the way to vector a sub 5. So they're both there, they're vectors, but as I said, we use the term vectors because this is a vector space but they are polynomials. And u, all the u sub 1, u sub 2, all the way to u sub 5, that is going to be the 5, my 5 polynomials that will now be an orthogonal basis. So I'm going to change the basis from the one that we have here, 1x, x squared, x to the power 3 and x to the power 4, we're going to change that basis such that all 5 of our polynomials are going to be orthogonal to each other and each one of those is going to be u sub n. So you'll have u sub 1, u sub 2, all the way to u sub 5. And how you work that out, of course, u sub 1 is very easy, we're just going to take one of these to be our first vector and then we'll get polynomials that are orthogonal to that one and then what next one orthogonal to both of the previous ones and that's how we build it up with the Gram-Schmidt process. But equation 4 is a very nice, succinct way of looking at it. So if I want u sub 4, for instance, that's u, the n is 4, that'll be a sub 4, so you'll just go look what a sub 4 is and then minus and then the summation and the summation really goes from 1, so the very first one you're working with, till the one just before the one you're working on. So it only goes to n minus 1, remember that? And then it is the inner product of the two vectors as we define there with our definite integral of a n, the n that you're working on and this instance if we're looking at u4, that would be a4 and then we're going to go from i sub 1, i sub 2, i sub 3, all the way to i sub n minus 1 and then the norm, we've seen how to do the norm, squared, so it's just very easy and then multiply by that specific one you're working with ui, then u1, u2, u3, okay that's not the best explanation, the best explanation comes from the example and that's exactly what we're going to run here. So first in the Wolfram language I'm creating two functions and this is how you create a function, I'm giving mine a name p and q, it's going to take two parameters, the languages might call them arguments, remember with the Wolfram language it would always use square brackets and not parentheses and I put a little underscore after each just to show that this is a parameter on argument, something that I'm going to pass to the function, actual values and this function will now return something for me, so I've got 2 there, that's separated by comma and then colon equals, the colon there means don't execute this now, I'm just defining a function here and what I'm doing is let's set that equal to x to the power n, so I'm assigning some, returning some value to this function I'm creating, so if I put in x and 3 it's going to return x cubed for me, if I put in x comma 4 it's going to return x to the power 4, I'm just doing two of them, you needn't do two of them but just to keep the clarity of there being two polynomials in my inner product, so below if we just run this that's going to create those two functions, so if I want a 3 and remember a sub 3 here was x squared as you can see there, so just pass in p and then x is the variable that I want and 2 is the power that I want and it's going to return for me x squared, so it's just an easy way to do this and then I'm going to define a function called inner product, as per usual with the Wolfram language we always start the first word with a lower case and each subsequent word we just concatenate and the first letter will be in uppercase, so it's easy to read, if someone else reads this code they know I try to generate a function that calculates the inner product, so it's nice just to use a descriptive name and that's of your choice, it's going to take two things p and q, again the delayed execution there which this means this is an assignment, this is what's going to be returned to this function and I'm using the definite integral and going from negative 1 to 1, now you see this beautiful notation that we have here that is part of the Wolfram desktop that I'm running here, it has a palette that has all these lovely buttons that you can click on and it generates this nice looking values for you and of course remember if you look online there is definitely still the integrate function, you can just type in integrate and you'll see the integrate function there so you can still do it by hand, so you look up that function in the help section but if you have access to a desktop version you can just use this kind of notation, so I'm just saying take p and take q and multiply that and there's a space between the p and the q and that is short-hand in the Wolfram language for multiplication, so then you have to put a star there, you can just put a space and then with respect to x on that interval negative 1 to 1, so there we go, so I'm just going to show you here I'm going to take now the inner product of x squared and x cubed so I'm passing my p and my q, remember p takes two arguments or two parameters, the x and the power and q takes two the x and the power, so what I'm just doing here is this definite integral of x squared times x to the power 3, in other words x to the power 5 with respect to x on that interval now that's going to be an odd function and this is important to remember because it's going to save you a lot of time and effort if you do this by hand if you have an odd function like this and your interval is symmetric around zero, so we're going from negative 1 to 1 it's always going to evaluate to zero, isn't it? and there we go so I'm using inner product passing x squared and x cubed there and we see the inner product indeed is zero, so if we just look at a plot of that so it's a plot of x squared and again I can put that little square up there, just some keyboard shortcuts or clicking on just an easy little button in the math help palette that you have access to here in the desktop version and then there's a little space in between just to denote that that's multiplication and then I'm saying go for an x from negative 1 to 1 and the filling is towards the axis and you can clearly see why this results in a zero integral because I have positive area at the top, negative area at the bottom, it's symmetric because this is an odd function it's going to yield zero, so let's get started as I mentioned with Gram-Schmidt process we're always going to assign one of our vectors to be the first vector that we deal with and it's easy in this instance we're going to make u1, u sub 1, we're going to set that equal to a sub 1 and remember a sub 1 is just one, so my first new basis vector is this going to be the first one in the original list, next I have to create one, I have to generate one that is orthogonal to this first polynomial and there I'm going to use my equation and that is the summation notation we had in equation 4, I'm just expanding that, so for u sub 2 that is going to be a sub 2 minus and then remember we're going from i equals 1 to i equals n minus 1 and n is 2 at this instance, I can only go to 1, so there's only going to be one expression here in the expansion of my summation so that's going to be a sub 2, u sub 1, that's the inner product between those two polynomials over the norm squared of u sub 1 and we already know what u sub 1 is, it's there, it's 1, we've already decided that and multiplied by u sub 1, so if I just substitute a sub 2 was x and then a sub 2 is x again it's there, u sub 1 was 1, so I'm taking the inner product of these two polynomials x and 1, I'm dividing it by the norm squared of the polynomial 1 and I'm multiplying it by 1 so let's have a quick look at that, remember if we just look at 1 times x in the product of the two polynomials p and q I'm going to get y equals x and again that's symmetric above and below the x axis, so of course it's going to yield 0, so to get 1 remember it's x to the power 0, so that's going to be my q there my p is x to the power 1 and if we take the inner product of that it's going to be 0, so all of this is just x minus 0 there, so u2 is just going to be x, well so far it's exactly the same as our original basis but what we've learned from that is the polynomial 1 and the polynomial x are orthogonal to each other, let's go for u3, now u3 has got to be orthogonal to u1 and u2, remember that, so we're going to do our summation expansion, so this time we're going to have 2 of these, we add them but remember there's a negative sign in the front of that summation, so it's minus and minus and so we're starting at i equals 1, so u1, u1 but we're busy with u3 so it'll be a3 there and the a3 there and the a3 there but now we're just cycling over our summation counter, so it's u equals i equals 1, i equals 2, we're only going to go to i equals 2 because we're working with n equals 3 and we only go to n minus 1 in the summation, so I think you can clearly see where 9 comes from and when I substitute these remember a3, the inner product of a3 and u1 would be a3 is x squared and u1 we know is 1 and there's u2 we know that it's x, so I'm taking the norm of x, so let's just do these, if I look at the inner product of x squared and 1, that's what we have there, the inner product of x squared and 1, there we go, x squared and 1 if we do that we see it's 2 thirds and what about the norm of the polynomial 1 so that's x to the power 0, but remember the norm squared is just going to be p times p and I'm doing that integral over my interval, so I'm using px0, qx0, I could just have said p and p there, should be more proper because this is p times p, that's how we define the norm and I'm taking the inner product of that and we see the inner product of that is 2 and next we need this inner product here, x squared and x, there we go, x squared and x, if we get that of course it's going to be 0 because x squared times x is x cubed which is an odd function and its interval is symmetric, negative 1 to 1, of course it's going to be 0, and that's why I mean if you do this by hand just look at it before you evaluate an integral because you can clearly see just by looking at it that it's going to be 0, so if we substitute what we've done up till now, we get u3 here in equation 10, u3 is x squared minus a third let's ramp it up a bit, u sub 4 which is now going to be orthogonal to u sub 3 and u sub 2 and u sub 1, they're all mutually orthogonal, so again I've expanded my summation notation there and you can clearly see that it follows that pattern and I've substituted it there in the second line of equation 11 here, so it's going to be x cubed is a4, a sub 4 is x cubed and then I have the x cubed in the front there and then I cycle over u sub 1, u sub 2 and u sub 3, and u sub 3 remember we multiply by that vector as well, so I've got to put that in for u sub 3 that one in for u sub 2, that one in for u sub 1 so if we just do this, remember what do you think is going to happen to the last term, I have an x cubed times an x squared that's x to the power 5, anyway let's just do this one x cubed, I'm going to show you x cubed and then x squared minus a third if I do that inner product I get 0 and once again just to show you very quickly the plot why this would evaluate to 0, you can clearly see that it does not function so here what I've done is I've just done all my inner products and we can see the inner product of x cubed and x1 is 2 fifths on my interval the inner product of 1 and 1, that's just 2 we've seen that one before of x cubed and x, that's going to be 2 fifths and of x squared, x to the power 1 sorry, x to the power 1, that's 2 thirds and if I substitute this is what I end up with, so do that substitution, we'll see we get 2x to the power 3 minus 3 fifths x so that's my fourth polynomial and it's now orthogonal to all the others and then eventually here we go to 5, the fifth orthogonal polynomial and again it's just that simple expansion have a look, pause the video and have a look at it and if we expand it all the ones that we're interested in, of course we're going to get the 0 there remember x to the power 4 times x to the power 1, that's x to the power 5 odd function symmetric interval, definite integral is 0 and there is the one of x to the power 4 and then x squared minus a third and also x squared minus a third and x squared minus a third so that's going to be the normal of that one and then our last inner product there of course again that's going to be an odd function that's 0 so in 14 here I show you, we just simplify this and we end up with our last orthogonal basis vector for this vector space up to polynomials with degree 4, so let's just visualize what happened here because what I've done here is put all my new bases there, my new 5 bases vectors and they are mutually orthogonal so if we plot them just to see what that would look like, this is what they look like and they just plot that original basis as well and you can see the difference and if you think about just normal vectors, say in 2 space, why are orthogonal vectors better, you can clearly look at them as a basis versus 2 vectors that are very close to each other and that all comes down to the application of what we're dealing with here if you think about vectors in 2 space, if you want to span that space you need 2 vectors as long as they're not linearly dependent but they can be very close to each other but what we're doing here is in applications we're trying to add some values to these things and if we take any kind of measurement, any kind of value, there's some uncertainty in that measurement and if you have 2 vectors very close to each other, that becomes unstable because just with a little bit of measurement in the wrong direction you're going to land exactly on one of the basis vectors where if they are mutually orthogonal, there's no ways that in that close measurement area you're going to land up with problems in the same here so we can see when x equals 1 of course they land up in the same spot but they're all very close to each other if we look at them here on the right right hand side on the positive x axis and if you look at all of these on the new orthogonal basis it's not a problem at all which just brings me to a neat little subject that I just want to make your way of and that's the genre polynomials and they are orthogonal just as the ones we've had before through the Gram-Schmidt process but they just each multiplied by a constant, remember that does not change the fact that they are still all mutually orthogonal to each other I'm just multiplying each one by a scalar and the scalar that they multiplied with the 5 that we've got up here is 5 I'm just multiplying each of them by a scalar and the way that I'm doing it is such that p of 1 equals 1 so for all of them p of 1 equals 1 and you can see all of them there very close to the ones that we have up here just change by a exactly by a constant scalar so that p of 1 equals 1 and they usually denote it up against l sub 0 sub 1 and to whatever degree you need to go and there's a neat function in the Wolfram language called the genre p and I give it the order that I'm looking for and the variable that I'm interested in and it'll just generate that 4 you'll see it's exactly what we have there 35x to the power 4 minus 3x squared plus 3 all of that times an eighth and it's very useful these Legendre polynomials so you can start reading up about those so I hope that clears it up for you how to take and how to change the basis of the basis of a polynomial vector space I have to change that basis so that my new basis vectors are all mutually orthogonal