 Welcome back, we will continue with our lecture. So we were discussing the encoded case in which the experimental data were not converted to minus 1 plus 1 and so on. The numbers are taken as they are and linear regression model is fitted. So here we have the so called variance covariance matrix V and we can see that the diagonal terms are the variances and the off diagonal terms are the covariances and based on the residual sum of squares we have the sigma hat squared as 382.56 and we can also see that the variance covariance matrix is symmetric in the sense that if you convert the rows into columns and columns into rows we retrieve the original matrix itself. So the variance of beta hat uncoded not the intercept on the uncoded model is 112.5 that would be 2.9341 into 382.56 variance of beta hat uncoded 1 is 0.5465 and the variance of beta hat uncoded 2 is 6.072. The standard errors for the regression coefficients we take the square root of the variance of the various regression parameters and the standard error for beta hat not is 33.5 for beta hat 1 is 0.74 and for beta hat 2 is 2.46. Now we may compare these numbers against the estimated parameters. The estimated parameters are 352.277 minus 0.6 and minus 23.5. So it can be seen that in the uncoded form of the regression analysis we are only considering the effect of the main factors we are not considering the interactions yet. So if you compare a standard error of beta hat 1 which is 0.74 that is of the same order as beta hat uncoded 1. So this is not quite good whereas for the other cases the intercept as well as for the beta hat 2 the standard errors are much smaller than the actual parameter values okay. The next step in our analysis is to construct the ANOVA table explaining the different calculations. So as we all know by now to construct the ANOVA table we need the total sum of squares regression sum of squares and the error sum of squares. The total sum of squares is given by sigma i is equal to 1 to n yi minus y bar whole squared and y bar from the given experimental data is 187.277 and we have 13 data points. This may also be written as y prime y minus i equals 1 to n yi whole squared by n. I will just make a small correction here. So y prime y is the transpose of the column vector of the responses and then y is the actual vector of the column responses. So when you do y prime y we get sigma equals 1 to n yi squared and then we also get this n y bar squared term or the sum of all the observations squared divided by the total number of observations. This we have covered in one of the previous lectures. So when you plug in the numbers again one small typo okay. So we have the total sum of squares after correcting for the intercept beta hat naught is 38869.34 and please note this value the total sum of squares for the main effects regression model is 494813.7. It would be a good idea at this stage for you to do these calculations on your own and see if your numbers match with mine. And the regression sum of squares as we all know by now is beta hat prime x prime y minus sigma equals 1 to n yi whole squared by n and that we get as 35043.74. Please note that we are doing the uncoded case here and I have removed the subscript uncoded uc for convenience but we will put it at the later stage okay. So all these things refer to the uncoded case and the regression sum of squares without removing the effect of beta hat naught is 49098.14 and then you have the n y bar squared term here. So the regression sum of squares is 35043.74. So we have the ANOVA table and we have the different sources of variations the regression residual and total and the sum of squares are given as shown here obviously the regression sum of squares has been corrected for the intercept beta hat naught okay. So that is why you have 35043.74 the residual sum of squares we know how to calculate and that is 3825.59 you have the model predictions you have the responses you take the difference and then square and then add them up you will get the residual sum of squares. So the total sum of squares is given by 38869.33 and that we saw earlier as well the total sum of squares after excluding the effect of beta hat naught is 38869.34 here and I have just put 33 okay a small difference may arise when you do the calculations from different ways. So you have the degrees of freedom 2 for the regression sum of squares in fact you have 3 parameters beta hat naught beta hat 1 and beta hat 2 but since we are not considering beta hat naught we have only 2 degrees of freedom and we have 10 degrees of freedom for the residual sum of squares we have 13 data points and then we have 3 parameters beta hat naught beta hat 1 beta hat 2. So the residual sum of squares as I said earlier in the previous slide the psi is equal to 1 to n yi-y predicted for the ith observation squared okay and that is expressed as y prime y-beta hat prime x prime y and if we do not consider this because they cancel out you can see that 494813.7-490988.14 that we get as 3825.56 which is what was reported here. Now we have to see whether the regression parameters are significant or not so we compare the mean square for the regression with the mean square for the residual and we get the corresponding f value by just looking at the f value itself we can see that the regression is significant because the f value is so high the p value is likely to be very small in fact it is 10 power-6. So as an exercise show that the first regressor variables temperature is not significant in the recent model. Next we move on to adjusted R squared we know that the adjusted R squared is a more realistic estimate of the quality of the regression fit okay. It is not enough if we match the model to all the experimental data points we have to see whether the parameters we have used in the model are really contributing to the model bringing in value addition okay. So we have R squared adjusted is 1-sum of squares of residuals by n-p by total sum of squares by n-1. So the residual sum of squares is a quantity which tries to reduce the R squared adjusted and we are making it larger by dividing it by n-p and if p increases n-p will decrease and this term in the numerator will start to increase and once it increases the R squared adjusted will start to decrease okay. So that is very important and this n-p is sort of penalty for trying to overfit the model. So in this particular case we have 13-3 we are only fitting 3 parameters and so we get R squared adjusted as 0.882 which is not too bad. The actual regression parameter R squared is the regression sum of squares by total sum of squares and that comes to 0.9016 and the adjusted R squared is pretty close to the actual R squared value at 0.882. Next we move on to the sequential sum of squares okay what is meant by the sequential sum of squares? It is a regression sum of squares added to that due to beta hat uncoded knot okay the so called intercept from individual regression terms in sequence. The sequence considered for illustration is effect of adding the temperature factor t the effect of the mass of the powder m and the effect of interaction between the 2 that is temperature and mass. So we have the model which is given as beta hat u c knot plus beta hat u c 1 t plus beta hat u c 2 m plus beta hat u c 1 2 t into m okay. So we are again carrying out the matrix approach to linear regression and to account for the temperature into m term what we need to do here is to simply multiply the column temperature with the column corresponding to mass. In this column multiplication we do term by term multiplication of the values of temperature and mass okay. So we simply treat it as a new model involving t into m and we re estimate all the regression coefficients. Now we are doing it in the uncoded manner this is not an orthogonal design. So we cannot see the effect of t into m separately as we did for the coded case it was an orthogonal design and it permitted us to do in that fashion. But here we cannot take that shortcut because this is no longer an orthogonal design it is an uncoded case and so what we have to do is we have to re estimate all the parameters for the new model. We have to again estimate beta hat uncoded knot, beta hat uncoded 1, beta hat uncoded 2 and then beta hat uncoded 1 2 corresponding to the interaction between temperature and mass of the powder. So the design is no longer orthogonal we cannot simply see the effect of adding the additional interaction by modelling it uniquely. So we also notice that there was a covariance between the regression parameters the parameters were not independent of each other. So now we have this x u c u c means uncoded and then i n t refers to interaction. Let me make a small change here. So x uncoded interaction is given by the column of once the temperature given here 30, 40, 50 and so on and then you have the mass of the powder given here and the interaction term involving temperature and mass of the powder is simply found by multiplying 30 times 3 you get 90, 40 times 3 you get 120, 50 times 3 you get 150, 30 times 6 you get 180 and so on. So for the case where we did not consider the interaction we noted the parameters to be 352.277 for beta hat knot uncoded beta hat 1 was minus 0.6 and beta hat 2 was minus 23.5. Now we are considering the interaction term and we are re-estimating the model parameters. If this had been an orthogonal design then these parameters would not have changed. We would have simply got the contribution from the interaction term okay or we would have got the parameter associated with the interaction term but on the other hand when we look at the model where we have beta hat uc interaction the set of parameters are now completely different. The beta hat knot uncoded for the new model considering the interaction between t and m is 112.277. The new beta hat knot uncoded is 112.277. The new beta hat 1 corresponding to the temperature regressor variable is 5.4 for beta hat uncoded mass of the powder it is 16.5 and then the interaction term is minus 1.0. So the interaction term is quite small when compared to the other parameters we have estimated. Now we have to find the extra sum of squares due to the interaction term. So the full model is the one which is including the interaction beta hat uncoded knot plus beta hat uncoded 1t plus beta hat uncoded 2m plus beta hat uncoded 1 2t into m. So this is the model which is accounting for the interaction between the 2 main factors. Now you also have the previous model the simple model which we started doing at the very beginning of the uncoded analysis. We have beta hat uncoded knot plus beta hat uncoded 1t plus beta hat uncoded 2m. Now it is important to remember that the parameters beta hat uncoded knot in the full model and the beta hat uncoded knot in the previous model or the original model their values are not necessarily the same. They may not be the same in most cases okay beta hat uncoded 1 will not be equal to beta hat uncoded 1 for the original model and beta hat uncoded 2 for the full model will not be same as beta hat uncoded 2 for the original model. Here we are estimating beta hat uncoded 1 2 for the first time okay. Now we are looking at extra sum of squares due to interaction. What we do is first we consider the regression sum of squares brought by the full model and that is given by beta hat uncoded interaction model prime x uncoded interaction prime y. We are including the effect of the intercept beta u c knot okay. So this is the full regression sum of squares and that comes out to be 494813.15. Next we have the sum of squares of regression brought in by the additional parameter the interaction parameter beta hat uncoded 1 2 in the full model okay. We wanted to consider the effect of the interaction and so we now want to see what is the additional contribution to the regression sum of squares by the freshly introduced parameter beta hat uncoded 1 2. So that is what we refer to here sum of squares of regression beta hat uncoded 1 2 given that beta hat uncoded knot beta hat uncoded 1 and beta hat uncoded 2 are already present in the model. So this was the original model or the old model which we had considered at the beginning and now to that model we are adding the interaction term and so what is the additional sum of squares brought in by the interaction term addition. So for that we have to take the total regression sum of squares for the full model which is including the interaction term that is why we have beta hat uncoded interaction term considered prime x uncoded interaction term considered y minus beta hat uncoded prime x uncoded prime y okay. And we have these values as 494813.15 minus 490988.15. So rather than only listening to the lecture I would suggest that you please carry out the calculations on your own and then listen to the lecture so that or again listen to the lecture so that you can follow the thread and also make sure that the calculations have been done correctly. So if you recall we have 494813.15 let us see whether this number matches with the one we had estimated earlier for the full model and that is here 494813.15 that can be verified by simple calculations but now let us look at the regression sum of squares including the intercept term for the case where the interaction term was not present that is the old model. So let us see that it is 490988.15 so we have 490988.14 okay so pretty much the same right so I am just telling where we picked up this number from okay and that value comes out to be 3825 and the residual sum of squares with interaction no big deal we know how to do that it is y prime y minus beta hat uncoded interaction term considered prime x uncoded interaction term considered prime y, y prime y is 494813.74 we have seen this thing earlier as well and the total regression sum of squares is 494813.15 that we found in the previous slide you can see the numbers here and also here and then when we subtract the 2 we get the sum of squares of the error as small 0.59. So now what we can do is carry out the F test as before and see whether the interaction term is important or not and since the error mean square as considerably reduced it was 0.59 for the sum of squares and when you further divided by 9 we get a even smaller number. So the interaction term would be quite significant the F values 58347 the mean square error previously was 10 because we had considered only 3 parameters and out of the 13 data points so we had n minus p as 10 but now we are considering 4 parameters the intercept the main factor 1 main factor 2 that makes it 3 and then the new interaction term which is the 4th parameter. So n minus p is 13 minus 4 which is 9 and that is why you get the degrees of freedom for mean for the mean square error term is 9 and so we have 0.59 as the residual sum of squares divided by 9 as the degrees of freedom and so we get the mean square error as 0.59 by 9. So we get the F value as 58347 which is pretty high and so it is obvious that the interaction term is quite significant. So the p value is pretty much close to 0 and hence the null hypothesis may be rejected. So the variable t into m the regressor variable t into m the interaction between temperature and mass of the powder creates a significant contribution to the process. So now we move on to another important and interesting concept called as the adjusted sum of squares. Let me sort of add a note of caution here for those of you who are very much interested in knowing the depth and breadth of regression analysis which is very fascinating. You may continue from this point onwards otherwise you may stop at this particular sequential sum of squares calculation concept okay. But I would suggest that you give it a shot and see the adjusted sum of squares concept also it is quite interesting and informative okay. But for instructors who are going for a tight time schedule and time bound completion of the syllabus this adjusted sum of squares concept may be skipped. We are looking at adjusted sum of squares we want to demonstrate the effect of adding the variable t last after considering the variables for some reason m and t into m first. In other words the variable t is added to the model last after considering the main factor m and the interaction t into m okay. So we have the first new model okay we will call it as the first new model which is beta hat uncoded knot the intercept beta hat uncoded 2 which is corresponding to the regressor variable m and then beta hat uncoded 1, 2 t into m okay. Here the regressor variable t is absent okay. So we are having the first new model whereas the full model is beta hat uncoded knot plus beta hat uncoded 2m plus beta hat uncoded 1, 2 t into m same as the first new model so far and then you are having the beta hat uncoded 1 t which is added last. So this is the full model and as before the common coefficients in the two models are not the same beta hat uncoded knot is not equal to this beta hat uncoded knot, beta hat uncoded 2 is not equal to this value and this value would not be equal to this value and this is the one we are going to estimate newly. So we have the first new model let me refer to the first new model it is having only m and t into m okay it is having the matrix x uncoded 1 nu in the following form the usual vector of 1 and this is the mass of the powder and then we have m into t okay we do not have the t at all in the first new model we have only m into t which is 30 for temperature so 30 into 3 is 30 is 90, 3 into 40 temperature in degree centigrade is 120 and so on. And you have the full model which is comprising of the vector of 1 the column vector of the mass of the powder same as this and then you have the t into m and then you have the temperature coming in at the very end and you can also confirm that 3 into 30 is 93 into 40 is 123 into 50 is 150 and so on. So please see the arrangement of the various column vectors in the two matrices. So we have the adjusted sum of squares due to temperature we find in the following way first new model including intercept beta 1 nu uncoded not is given by the sum of squares of regression is given by beta hat uncoded 1 nu prime x uncoded 1 nu prime y and that turns out to be 492123.19 this is only for the first new model which was not having the effect of temperature for some reason. Then you have the full model sum of squares of regression beta hat uncoded 1 given that beta hat uncoded not beta hat uncoded 2 and beta hat uncoded 1 2 were already present in the model okay. So we have to find the value addition due to adding the temperature effect at the very last okay. So what is that value addition to the regression sum of squares when beta hat uncoded not beta hat uncoded 2 and beta hat uncoded 1 2 were already present in the model okay. So we have beta hat uncoded full prime x uncoded full prime y minus beta hat uncoded 1 nu prime x uncoded 1 nu prime y okay. So this is for the full model and this is for the first new model the regression sum of squares when we subtract from the full regression sum of squares the regression sum of squares due to the first new model we get the adjusted sum of squares brought in by the addition of the temperature variable at the very end towards the very end. So we get 494813.15 minus 492123.19 we get the value to be 2690. Now let us show what will happen when we add mass at the very end. I hope you had followed the discussion regarding temperature. Temperature was added at the very end and now we want to add the mass at the very end. So we have the second nu model as beta hat uncoded not plus beta hat uncoded 1 t plus beta hat uncoded 1 2 t m. So for some reason mass is considered to be not there in this second nu model we have only t and t m and then the full model of course will have beta hat uncoded not plus beta hat uncoded 1 t plus beta hat uncoded 1 2 t m plus beta hat uncoded 2 m okay. So the values of the regression parameters are not the same in the two models. Now we do the same manner sum of squares of regression to the second nu model is given by beta hat uncoded 2 nu prime x uncoded 2 nu prime y. Of course this includes the effect of the intercept beta for the second nu model you see not okay the intercept and so we get that value as 494186.1. The sum of squares of regression beta hat uncoded the second parameter for the full model given that beta hat not uncoded comma beta hat uncoded 1 comma beta hat uncoded 1 2 that is given by beta hat uncoded full prime x uncoded full prime y minus beta hat uncoded second nu model prime x uncoded second nu model prime y okay and that turns out to be 494813.15 minus 494186.1 which is 627.1 okay. So this looks a bit difficult but in fact it is quite simple. The full model is given as given here okay and we want to see the effect of the addition of mass and for the mass the regression coefficient is beta hat uncoded 2. So we want to see the effect of bringing in this regression parameter beta hat uncoded 2. So we want to see the sum of squares brought in by this beta hat uncoded 2. For that we take the regression sum of squares for the full model and we take the regression sum of squares to the second nu model and we take the difference. Obviously we have to subtract the regression sum of squares of the second nu model from the full model and that is what we are doing here. We are having the full model regression sum of squares and then the second nu model regression sum of squares and the difference between the 2 would be the contribution due to beta hat uncoded 2 which is the mass and that comes out to be 627.1 right. Next we move on to the confidence intervals on the regression coefficients. We have the 100 into 1-alpha percent confidence interval for the regression coefficient beta j in the multiple linear regression model that is given by this formula beta hat-t alpha by 2 n-p a standard error for the corresponding regression parameter beta hat less than or equal to beta less than or equal to beta hat plus t alpha by 2 n-p standard error for beta hat. The standard error for beta hat is obtained from the variance covariance matrix main diagonal. So the model parameters for the full model are 112.277, 5.4, 16.5 and minus 1 and the variance covariance matrix is given as shown here. So when you plug in the values in the formula we get the confidence intervals as given here. So for beta hat not it is 109.81 as the lower limit and 114.74 as the upper limit. So these are the boundaries for the 95% confidence interval and this is quite narrow. The parameters estimated to be within this 2 numbers and for the beta hat 1 it is 5.34 and 5.46 as the lower and upper limits of the 95% confidence interval. And then you also have the confidence interval for beta hat 2 as between 16.12 and 16.88 and for beta hat 3 is minus 1.01 and minus 0.99. There is nothing wrong if both the upper and lower limits are negative okay. You may think that it is a problem if the lower and upper limits are negative. It just means that the parameter itself is negative okay. There would be a problem only if the lower limit is negative and the upper limit is positive and then you are saying that the parameter itself is insignificant. So a simple clue to see whether a parameter is significant or not in the regression model is to identify the 95% confidence intervals for the different parameters. And if the parameters are having the lower and upper limits to be of the same sign then the parameter is significant. If on the other hand the lower and upper limits of the parameters are having opposite signs then the parameter is insignificant. It is pretty much saying that the parameter value is 0. Now we sort of summarize we have the full model beta hat uncoded knot, beta hat uncoded 1t, beta hat uncoded 2m and beta hat uncoded 1 to tm okay. For that the parameters are 112.28, 5.4, 16.5, minus 1. This we had seen earlier. Now when you look at the model at which temperature was added first we have c hat is equal to beta hat knot or rather beta hat uncoded knot plus beta hat uncoded 1t. So the parameters are 211.28 and minus 0.6 and these values are not same as 112.28 and 5.4 okay. So temperature is considered first in the model and remember all these models are dealing with uncoded numbers okay and so the parameters are different. And when you consider the mass first so that the model is c hat is equal to beta hat uncoded knot plus beta hat uncoded 2m then again the parameters change 328.28 minus 23.5. So this minus 23.5 is different from this beta hat uncoded 2m okay. And then when you consider the model in which you add temperature first and then mass of the powder second this is the only main effects model we had already seen this previously and the parameters are 352.28 minus 0.6 minus 23.5 and we can also correspondingly find regression sum of squares including beta knot they intercept and for the full model we have the regression sum of squares is 494813.15 the error sum of squares is 0.5931 and then we subtract the effect of the intercept that is why we are moving in y bar squared and we get this value. Similarly, we do it for the other models and we get the corresponding regression sum of squares and the sum of squares after removing the effect of the intercept beta knot okay. And so this is the effect of adding temperature this is the effect of adding mass okay and so we get these values. And then the regression sum of squares including beta knot for the main effects model alone is 49098.15 and if you remove the effect of beta knot we get 35043.75. And the regression is quantified by the coefficient of determination R squared and the adjusted R squared and these values are given here and it can be seen that the adjusted R squared is pretty close to the R squared value. Of course when you add the full model the R squared is 99.99 and then we have these numbers. So I request you to go through these calculations and get these values yourself and that concludes our discussion on the regression analysis. It has been a very interesting experience in understanding the various complexities and intricacies of regression analysis. By having a deeper insight into this concept it will be useful for us to compare various regression models and choose the one which is compact has a high adjusted R squared value and less number of terms in the equation so that it is easy to use in further applications involving the same variables. Thank you for your attention.