 this is my third lecture on polynomial regression and here is the content of this topic. So, polynomial models in one variable, orthogonal polynomials, piecewise polynomial fitting and polynomial models in two or more variables. So, we will be talking about this polynomials in two or more variable today. So, in the previous classes we have studied polynomial in one variable and we know that polynomials are used in situation when the response variable is non-linear and we have studied how to fit a kth order polynomial using orthogonal polynomials and also we have studied piecewise polynomial fitting. So, piecewise polynomial fitting is used in situation when a lower order polynomial does not fit the given data properly, but increasing the order of the polynomial does not improve the situation substantially. So, this indicates that the response function it behaves you know differently in different part of the range of x. So, what we do in a common approach to deal with such situation is that we divide the range of x into several segments and we fit an appropriate curve in each segment. So, we talked about all these things you know in the previous classes today we will be talking about polynomial models in two variables. So, polynomial models in two or more variables, the second order polynomial model in two variables is y equal to beta naught plus beta 1 x 1 plus beta 2 x 2 plus beta 1 1 x 1 square plus beta 2 2 x 2 square plus beta 1 2 x 1 x 2 plus epsilon. So, this is a second order polynomial model in two variables. So, here the linear effect parameters are beta 1 and beta 2 and then quadratic effect parameters are beta 1 1 and beta 2 2 beta 1 1 beta 2 2 and then the interaction effect parameter is beta 1 2. So, here we usually call the regression function expectation of y is equal to beta naught plus beta 1 x 1 plus beta 2 x 2 plus beta 1 1 x 1 square plus beta 2 2 x 2 square plus beta 1 2 x 2 x 1 x 2. So, this is called a response surface and so this response surface is used in industry to for modeling the response variable in terms of controlled variable like in the regressors variable. So, they have huge application in industry. So, we will be talking now how to fit a second order polynomial model in two variables. So, for that I will give an example. I will talk about the fitting of second order polynomial in two variables by using an example here. So, this is chemical process example and here x 1 is a regressor variable x 1 and x 2 are regressor variable. So, x 1 stands for the temperature reaction temperature and x 2 stands for concentration and the response variable y it stands for percent conversion of a chemical process. So, we have two regressor variable and we have one response variable and we have given some data. We have to fit a model like this. This is a second order polynomial model in two variables x 1 and x 2. So, this one is nothing but a multiple linear regression variable like y equal to x beta plus epsilon. So, here x is the coefficient matrix or design matrix sometime we say. So, here the first column is corresponds to beta naught I mean or say x naught here and then the second column is corresponds to x 1 and then x 2 then x 1 square x 2 square and then x 1 x 2. Then how do you get this matrix? You are given x 1 here. So, the first column is corresponds to this x 1 values. You are given the x 2 values also. So, the second column or the column associated with x 2 is corresponds to this column and then you can compute x 1 square you can compute x 2 square and you can compute x 1 into x 2. So, 200 into 15 for the first observation. So, this is how you get the coefficient matrix x and then this one is same as multiple linear regression y equal to x beta plus epsilon where beta is beta naught beta 1 beta 2 beta 1 1 beta 2 2 and then beta 1 2. So, you have to estimate this coefficients and you know how to do that. So, beta hat is nothing but x prime x inverse x prime y. So, you know x matrix you know y. So, you can compute beta hat. So, here is the fitted model now. So, these are the this is beta naught hat and this is your this is the fitted model for this chemical process example and then here is the ANOVA table for this one. So, we had their 12 observations. So, that is why SS total has degree of freedom 11 and as you can see here that there are 6 parameters 1 2 3 4 5 and 6 and that is why you will have 6 restriction on residuals. So, there are total 12 residuals because there are 12 observations and on this 12 residuals EI you have 6 restriction. So, you have the freedom of choosing 6 residuals independently and then the remaining have to be chosen in such a way that they satisfy those restrictions. So, that is why the residual degree of freedom is 12 minus 6 which is 6 again and the regression degree of freedom is 5. So, you know how to compute this residual you know how to compute this regression SS regression. So, this SS regression it involves sort of SS regression due to beta 1 plus SS regression due to beta 2 plus SS regression due to beta 1 1 beta 2 2 plus beta 1 2. So, this is the total SS regression and then you have the MS residual here and here is the F statistic. What it does is that it test this F statistic is used to test the significance of this model whether this model is significant that means whether the parameters are significant. So, test for the significance of regression whatever you have fitted which is same as testing the hypothesis that beta 1 equal to beta 2 equal to beta 1 1 equal to beta 2 2 equal to beta 1 2 is equal to 0. So, all of them 0 means the null hypothesis says that the regression fit is not significant and the alternative hypothesis H naught here sorry alternative hypothesis H 1 that says no H naught is not true that means the fit is significant. So, in order to test this null hypothesis you have the F statistic and which has value 58.86 and now this follows F with degree of freedom 56 and you get the tabulated value from the F table that is 4.39. So, the observed value is greater than the tabulated value that means H naught is rejected which says that the regression fit is significant. So, overall whatever model we have fitted like second order polynomial involving two variables that model is significant. Now what we are going to do is that we are we will be testing whether what is the contribution of the linear terms what is the contribution of beta 1 and beta 2. So, that is what we will test the significance of the linear terms in terms of beta 1 and beta 2 and then we will be testing the significance of the quadratic terms. So, to test the contribution or significance of linear terms of the model what we have to test is that we have to test the null hypothesis H naught that beta 1 equal to beta 2 equal to 0 because beta 1 is the coefficient of x 1 and beta 2 is the coefficient of x 2. So, if this is true that means if the null hypothesis is true then the contribution of the linear term is not significant against the alternative hypothesis H 1 that H naught is not true. So, to test this one we need to find S S regression due to beta 1 and beta 2. So, this is the contribution of beta 1 and beta 2 in total S S regression. So, S S regression due to beta 1 and beta 2 in the presence of beta naught. So, this measures the contribution of first order terms. So, how to get this one is that you fit a model you fit the model y equal to beta naught plus beta 1 x 1 plus beta 2 x 2 plus epsilon. So, you fit the this model to your given data x 1 x 2 y. So, you are given x 1 x 2 y for you have several observations on x 1 x 2 and y here specifically you have 12 observations on x 1 x 2 and y. So, you fit this model on the given observations and then the regression sum of square for this model is basically this quantity S S regression is beta 1 beta 2 given beta naught. So, how to get S S regression due to beta 1 and beta 2 given it is given that beta naught is in the model. So, basically to get this S S regression you have to fit this model and then you find the S S regression for this model that S S regression for this model you know how to do that. So, S S regression for this model is same as S S regression due to beta 1 beta 2 in the presence of beta naught for the model we consider like no second order model involving two variables. So, this can be found that this is equal to 914.4 with the degree of freedom 2 may be I will explain why it is 2. So, the F statistic is so why it is 2 you can construct an overtable for this one total degree of freedom is 11 that is total and then the residual residual has degree of freedom 12 minus 3 that is 9 that is why the regression degree of freedom is 2 for this model. So, the F statistic is 914.4 by 2 minus 2 by 5.89. So, but this test is for the full model and the MS residual for the full model is 5.89. So, this part is just to explain how to get S S regression due to this in the presence of beta naught. So, this is the MS residual and this is equal to 77.62 and you now this follows F distribution with degree of freedom 2 and 6. So, you find the tabulated value 0.0526 from the table that is equal to 5.14. So, you see that the observed value is greater than the tabulated value. So, which implies that H naught is rejected and H 1 is accepted. So, H naught is rejected means beta 1 and beta 2 are not equal to 0. That means the linear term contribution is significant. So, which implies linear terms contributed significantly to the. So, we observe that the contribution of linear terms is significant in the model. Now, we test for the significance of or the contribution of the quadratic term to test the contribution of linear terms. Quadratic terms given that the model already contains the linear term to test this thing the contribution of the quadratic terms given that the model already contains the linear term. We have to test the hypothesis that beta 1 1 is equal to 0. So, beta 1 1 is beta 2 2 is equal to beta 1 2 is 0 against the alternative hypothesis H 1 that H naught is not true. So, I hope that you can recall the model. So, the model we are considering is y equal to beta naught plus beta 1 x plus beta 1 x 1 beta 2 x 2 plus beta 1 1 x 1 square plus beta 2 2 x 2 square plus beta 1 2 x 1 x 2 plus epsilon. So, this is the full model. Now, you know how to test this hypothesis using the technique of extra sum of square. So, the F statistic for testing this hypothesis is equal to I will use the notation that S S regression due to beta 1 1 beta 2 2 beta 1 2. So, the S S regression due to beta 1 1 beta 2 2 and beta 1 2 is equal to beta 1 2 in the presence of the linear model linear terms in the model that is beta naught beta 1 and beta 2. This is what we want to test. I mean this is the notation for S S regression due to this quadratic term in the presence of linear term by M S residual and S S regression and of course, I need to divide this by the degree of freedom that is 3. I will explain why it is 3. Now, this one is this regression S S regression due to this quadratic term in the presence of linear terms. This can be computed using the extra sum of square technique. So, what you have to do is that you compute S S regression for the full model. So, you compute the S S regression for the full model. This is the full model and we already have that. We have a ANOVA table for this one. So, this is the S S regression for the full model. Now, what I will do is that this minus S S regression for the restricted model. What is my restricted model? My restricted model is the model under H naught that is Y equal to beta naught plus beta 1 X 1 plus beta 2 X 2 plus epsilon. So, S S regression due to this model is nothing but S S regression under the restricted model by M S residual. Now, you can see just now we have both the things. We have S S regression for the full model from the ANOVA table that is 1733.6 and for the restricted model also we have it that is just now we computed that is 914.4. Now, this has degree of freedom 5 and this has degree of freedom 2. So, the difference is 3 that is why you have to divide it by 3. So, this by 3 by M S residual is 5.89 which is equal to 46.37. Now, you check the tabulated value. This F follows F distribution with degree of freedom 3.6.05 level of significance. So, this one is nothing this is you will get this value from the table from the F table that is 4.35. So, you can see the observed value is greater than the tabulated value which implies that the null hypothesis is rejected that means this coefficients are significance. So, the final conclusion is that so, this implies that the quadratic terms contributes significantly to the. So, we have observed that you know well. So, we have observed what we have done is that we have studied how to fit a second order polynomial model involving two variables and then first we computed we fitted that model and then we computed the ANOVA table for the full model and then we observed that the model is significant by the F test and then once the model is significant that means all the regression coefficients are not equal to 0. Some of them are non-zero and then what we did is that we tested the significance of the linear term separately and we also tested the significance of quadratic term and we found that for this particular example both the linear terms and the quadratic terms are significant. So, you cannot remove any term from the from the model. So, the model is quite significant. So, that is all about the second order polynomial fitting involving two variables and now we will solve some problems on orthogonal polynomials. So, we have the problem. So, fit a cubic equation using orthogonal polynomials to the y values the values are 13, 4, 3, 4, 10 and 22. So, we have 6 observations for y corresponds to the x value minus 2.5 minus 1.5 so on and you can see that the x values are equally spaced. So, the question is we are asking to fit you a cubic equation. So, is the cubic term needed? If not what is the best quadratic fit? So, to solve this problem first what we have to do is that we have to fit 3 degree polynomial involving I mean using orthogonal polynomial because that is the orthogonal polynomial has some advantage and then we will test the significance of the quadrate significance of the cubic term. So, I wrote the observations here again. So, this is by x and y and I have 6 observations. And, the question say you fit a cubic model. So, basically you have to fit y equal to beta naught plus beta 1 x plus beta 2 x square plus beta 3 x cube plus epsilon. You have to fit this model and then we know that you know instead of fitting I mean we know how to use orthogonal polynomial to fit third order polynomial. So, instead of fitting this model we will fit this one. So, this one is this is also third order, but involving orthogonal polynomial. So, this is orthogonal polynomial of order 3, order 2, order 1 and we have 6 observations. So, you know how to compute this orthogonal polynomial for 6 observations. So, P naught, so here you have P naught x. So, you know that P naught x is always equal to 1 for all x and P 1 x is minus 5 minus 3 minus 1 1 3 5 right. I hope that you can recall that P 1 x is equal to lambda 1 minus 1 minus into x minus x i minus x bar by d. So, here you can check that x bar is equal to 3.5. I mean you do not need to consider this x value you can just replace them by 1 2 3 4 5 6 because they are all equally spaced. So, you can code them by 1 2 3 4 5 6. So, using those values my x bar is equal to 3.5 and then for 1 it is 1 minus 3.5 d is equal to 1 and I have to take lambda equal to 2 to make it integer. So, this is minus 2.5 and then I multiply by 2 to get minus 5. So, this is how you know you have to compute P 1 x and then you see the formula for P 2 x and P 3 x. Generally, during the exam you know this table is given. So, you do not need to memorize all these things. So, what I want to do is that suppose well. So, it says that you fit third cubic equation and then you know what is this alpha how to estimate this alpha. We know that alpha naught hat is equal to y bar and alpha j hat is equal to p j x i into y i and then you know what is this alpha how to estimate this alpha. We know that alpha naught hat is equal to y bar and alpha j hat is equal to p j x i into y i by p j x i square. So, you know everything. So, if you want to say compute alpha 1 hat if you have p 1 x you know y. So, you can compute alpha 1 hat and similarly alpha 2 hat and alpha 3 hat. So, that is not a problem. Now, the problem says that you have to compute alpha 1 hat and alpha 2 hat and alpha 3 hat. So, that is not a problem. Now, the problem says that you know you test the significance of the problem says that is the cubic term needed. So, that means you have to test the hypothesis that h naught alpha 3 equal to 0 is equal to 0 is the cubic term needed. So, that means you have to test the hypothesis that h naught alpha 3 equal to 0 against the h 1 that alpha 3 is not equal to 0. So, how to do that alpha 3? I think that you first compute the ANOVA table for this one and then come back to come back for testing this because anyway you have to estimate the SS residual. So, let me construct the ANOVA table first. For that I need SS regression due to alpha 1 and alpha 2 so that one is nothing but alpha 1 hat summation y i p 1 x i. So, you can compute you know alpha 1 hat and you know y i's you know p 1 x i. So, you can check that this one is 58.51 with the degree of freedom of course 1. Similarly, you can compute SS regression due to alpha 2 that means the contribution of the quadratic term the alpha 2 x square right in the regression model. So, that is alpha 2 hat into y i p 2 x i this one is nothing but 210 that you can check with degree of freedom 1 and then SS regression due to alpha 3 this we need because we need to test the hypothesis that alpha 3 equal to 0 against alpha 3 not equal to 0 that is alpha 3 hat summation y i p 3 x i right. So, this one is very small. So, this clearly of course we will test it formally, but it clearly says that the significance of alpha 3 is negligible with the degree of freedom 1. So, if you add this 3 SS regression that will be the SS regression due to alpha 3 hat summation y i p 1 x i. So, this one total SS regression right and I mean what I mean is that SS regression for the cubic model involving orthogonal polynomial is nothing but SS regression is SS regression due to alpha 1 plus SS regression due to alpha 2 plus SS regression due to alpha 3 right. And then you compute SS total and then SS residual sorry SS residual can be obtained from that SS total minus SS regression that you can check that this SS residual is 207.7 0 with degree of freedom 2. Why it is 2? Because we have 4 parameters in the model. So, 4 parameters mean 4 restriction on the residuals and there are total 6 residuals and you have 4 restrictions. So, that means only 2 you can choose independently and you have the freedom of choosing 2 and the other 4 has have to be chosen in such a way that the 4 restrictions are satisfied. So, that is why the SS residual has degree of freedom 2. Now, I have the SS residual. So, I can compute the F statistic to test this hypothesis alpha 3 equal to 0. So, my F statistics is F is SS regression due to alpha 3 that means the contribution of cubic term in total SS regression by its degree of freedom is equal to 1 by SS residual divided by its degree of freedom that is MS residual. So, this one is nothing but 0.006 by 1 and SS residual we computed that is 207.7 by 2 which is equal to which is very small 0.000057 and I mean this clearly says that alpha 3 is not significant. So, still let me find the value of tabulated F that tabulated F 0.05 with degree of freedom 1 and 2 that is 18.51. So, this one is clearly not significant. Similarly, very small very smaller than 18.51. So, this F test implies that no H 0 is accepted that means alpha 3 can be 0 in the model. So, which in other words says that alpha 3 is not significant. So, I can say here that alpha 3 is not significant. So, not significance mean I can go for the model y equal to alpha naught hat plus alpha 1 hat P 1 x plus alpha 2 hat P 2 x. I can ignore the cubic term and finally, you can check that this is y hat is equal to 4.273 plus 1.8286 x plus 2.3750 x square. See you know that alpha 3 is not alpha 1 hat. What we have done here is that you know you find alpha naught alpha 1 hat and alpha 2 hat and then also you replace this you write this equation in terms of the in terms of x. Here it is in terms of the orthogonal polynomial. So, you have to replace this P 1 x by that P 1 x is equal to lambda 1 x minus x power by d. So, you will get this lambda is equal to 2 d equal to 1 and so finally, you have to get this equation in terms of x. Well, so the next problem is it says that so this is the original problem. So, we tested the third order term and we found that the alpha 3 is not significance and we found the best quadratic fit. Now, if the model say this is a third order polynomial had been fitted directly how would the extra sum of squares s s regression due to beta 1 given beta naught which is equal to 58.51 and s s regression due to beta 2 in the presence of beta naught and beta 1 which is equal to 210 in 0.58 and s s regression due to beta 3 in the presence of beta naught beta 1 and beta 2. How these things are related to the sum of square for the first, second and third order orthogonal polynomial. So, you understand the problem. So, what you have done is that we have fitted a third order polynomial in terms of using orthogonal polynomials. Now, you can without using the orthogonal polynomial you can use you can fit this model also and then the s s regression you have the s s regression due to beta 1 in the presence of beta naught which is equal to 58.51. How this one is related to the s s regression of the first order polynomial involving orthogonal polynomials. So, I hope you understood the problem. Now, if you see that this one is 58.51 and whatever we computed before that s s regression due to alpha 1 is also 58.51. So, the s s regression for the model y equal to alpha naught plus alpha 1 p 1 p 1 p 1 p 1 p 1 p 1 p 1 p 1 p p 1 x. The s s regression due to this model is same as s s regression due to the model beta naught plus beta 1 x plus epsilon plus epsilon because s s regression due to this model is nothing but s s regression due to alpha 1 and s s regression due to this model is nothing but s s regression due to beta 1 in the presence of beta naught. Now, see the question says how now this one is related how s s regression due to beta 2 in the presence of beta naught and beta 1 that is 21 naught 0.58. How this is related to the sum of square for second ordered orthogonal polynomials. Now, we have to check. So, these two quantities are same. This is same as basically s s regression due to beta 2 in the presence of beta naught and beta 1 which is same as s s regression due to alpha 2. So, this is same as alpha 2. So, what I want to the message I want to give here is that now I am talking about a model in of order 2. Let me write down that y equal to alpha naught plus alpha 1 p 1 x plus alpha 2 p 2 x. So, this is a second order model involving orthogonal polynomials plus epsilon and let me write the polynomial third order sorry second order polynomial that is beta naught plus beta 1 x plus beta 2 x square plus epsilon. Now, the contribution of what is this quantity the contribution of beta 2 in s s regression in the presence of beta naught and beta 1 is same as the contribution of alpha 2 in this model in the presence of alpha naught plus and alpha 1. But you are aware of the fact that you know in case of orthogonal polynomial fitting the s s regression are orthogonal. I mean the s s regression due to alpha 2 does not depend on s s regression due to alpha 1. So, the s s regression due to alpha 2 is same as s s regression due to alpha 2 in the presence of alpha naught and alpha 1. So, you understand I think you have to think about it. So, that is why you know s s regression due to alpha 2 is same as the s s regression due to beta 2 in the presence of beta naught and beta 1. And similarly you can check that s s regression due to alpha 3 does not depend on the other terms. So, this is same as s s regression due to beta 2 sorry beta 3 in the presence of beta naught beta 1 beta 2. This is same as this one and this is same as s s regression due to alpha 3 right for polynomial fitting using orthogonal polynomials because s s regression in polynomial fitting they are orthogonal also. So, this is same as s s regression due to alpha 3 in the presence of alpha naught alpha 1 and alpha 2. It does not matter you know because the s s regressions in polynomial fitting using orthogonal polynomials they are independent. So, that is all for today. Thank you.