 we will consider one example and this example will enable us to illustrate the theoretical concepts we mentioned in the last two classes. Well in last two classes we have learned you know how to how to fit multiple linear regression model and when the you know when the model has been fitted or has been constructed I mean next job is to you know test the statistical significance of the fitted model and we know how to test the significance of the fitted model using the global test and also using the partial test and also we learnt you know extra sum of square technique to test whether to test the test for several variable several regression parameters being zero. Well so we considered the data in the following table here you have two regression variable X1 and X2 and one response variable Y and here is the observation corresponds to X1, X2 and the response variable Y. So, this one is the basically you know this is the first observation on a regressor 1 this is the first observation on regressor 2 and this is the first observation on the response variable Y well. So, what is the first requirement is that using least square method estimates the parameters betas in the. So, the model here is you know Y equal to beta naught plus beta 1 X1 plus beta 2 X2 plus epsilon. So, we have to estimate the unknown parameters beta naught beta 1 and beta 2 well. So, first we construct the following matrices matrix Y which is you know basically the matrix of observations. So, this is 6 8 1 0 5 3 2 minus 4 10 minus 3 5. So, this is the vector of observation and it is a 11 cross 1 vector because we have 11 observations here. Now, we will construct the matrix X this X is equal to 1 1 8 1 4 2. So, this column is corresponds to the regressor X1 this one is corresponds to the regressor X2 and this way you know 1 9 minus 8 1 11 minus 10 this way you go up to 1 6. So, this one is a 3 cross 11 matrix and this is the vector and the next one is the beta vector which is basically the vector of parameters this consists of beta naught beta 1 beta 2. So, it is a 3 cross it is a 3 cross 1 vector and also we define you know epsilon is a 11 cross 1 vector which one is you know this is the vector of errors. So, the model now can be written in the matrix form like Y equal to X beta plus epsilon well and we need to estimate the unknown parameters using least square method well. So, L S estimates of beta naught beta 1 and beta 2 are beta hat which is equal to X prime X inverse X prime Y. So, this one is you know 118 1 4 2 1 6 minus 4 this one my X prime and X is 118 1 4 1 6. So, X prime X inverse X prime Y. So, 118 1 4 2 1 6 minus 4 this is X prime and Y is 6 8 right. So, see you know this one is X is a 3 cross 11 matrix X prime is 11 cross 3 matrix and this one is X prime. So, this is 3 cross 3 by 11 matrix and this one is you know 11 cross 1 matrix well. So, this one is you know you can check that X prime X is equal to 1166 minus 22506 minus 346484 and it is a symmetric matrix ok. So, this is my X prime X. So, X prime X inverse and X prime Y is 33 85 142. So, you can check that well. Now, the inverse of this matrix is equal to 4.3705 minus 0.84 minus 0.4086 well. So, this inverse is also of course, it is a symmetric matrix. So, this is the inverse of X prime X into X prime Y which is equal to 33 85 142. So, what we got is that you can check that this is equal to 14 minus 2 minus half. So, beta naught hat is equal to 14 beta 1 hat is equal to minus 2 and beta 2 hat is equal to minus 0.5. So, our fitted equation is you know the fitted equation is Y hat equal to 14 minus 2 X 1 minus X 2 by well. So, once we have the fitted equation you know next we need to check how useful is this fitted equation. That means, we need to test the statistical significance of the fitted equation basically first we go for the global test we which test the hypothesis that beta 1 equal to beta 2 equal to 0. That means, there is no linear relationship between the response variable and the two regressor variable and we will use the you know ANOVA approach to test the statistical significance of the fitted equation well. So, for that we know first we need to write down the ANOVA table let me do that first well. So, the next problem you know P 2 I will say the second problem you write out ANOVA table well. So, what we need to compute here is that we need to compute SS residual total and then SS regression can be obtained from these two I mean SS T minus SS residual is equal to SS regression. Now, in order to compute you know SS residual first we need to compute the residuals well. So, the table for we make a table of fitted values and residuals we know what is x 1, x 2, y we know what is y hat y hat is equal to 14 minus twice x 1 minus x 2 by 2 right. So, once we have y and y hat we can compute E where E is equal to the residual you know this is equal to y minus y hat. Let me just explain one two observation like first observation is 1, 8 and the response value is 6. So, the corresponds to 1 and 8 the fitted value of the response variable is equal to 8 you can check that. So, the first residual you know basically this is y 1 hat. So, E 1 is equal to y 1 minus y 1 hat y 1 equal to 6 and y 1 hat is equal to 8. So, the first residual is minus 2 similarly 4, 2, 8 this is a second observation. So, y 2 hat is equal to 5 and your E 2 is equal to 8 minus 5 this is equal to 3. So, this way you know you compute all the residuals you this way you can go up to E 11 because we have 11 observations and once you have all the residuals you know you can compute the SS residual now. So, SS residual is equal to summation E i square i is from 1 to right and the value of this one you can check that this is 68 next we go for SS residual. SS total is equal to summation y i minus y bar whole square which is nothing but summation y i square minus n y bar square right. This is this value is equal to 289 minus 289 minus n is 11 and y bar is 3. So, 3 square is equal to 9 and SS total is equal to 190 and once I have the SS total and SS residual I can compute SS regression which is equal to SS total minus SS residual 190 minus 68 which is equal to 122 right. Now, we can have the ANOVA table source of variation you know degree of freedom SS m s m s m s m s m s m s m s m s and the A value well the source of variation explained by the regression model residual and the total variation well. So, the degree of freedom of total SS total is 10 because you know we have 11 observations and one degree of freedom we lose because of the constraint that summation y i minus y bar is equal to 0. So, SS total has degree of freedom 10 SS regression has degree of freedom 2 and the SS residual has degree of freedom 8. Let me explain here you know SS regression has degree of freedom 8 which is equal to 11 minus 3 we are losing the 3 degree of freedom because of the 3 constraint on residual because the residual e i it satisfies you know 3 constraint. The first constraint is summation e i equal to 0 the second constraint is summation e i x 1 equal to 0 that x 1 is the first regressor and the third constraint is summation e i x i 2 equal to 0 well. So, the residual has degree of freedom 8 and the regression has degree of freedom 2 we call it SS residual this is called SS regression. So, the SS regression value is 0. 2 is equal to 122 SS residual is equal to 68 and this one is 190 and the m s value is 61. So, 122 by 2 and m s residual is 68 by 8 which is equal to 8.5 and the f statistical value is equal to 7.17. So, this is the ANOVA table next we move to the next problem which is problem 3 I mean requirement 3 you can say using you know alpha equal to 0.05 that means the level of significance is 0.05 you test to determine if the overall regression is statistically. So, the meaning of this one is that whether the overall regression is statistically significant that means we need to go for the global test that means we need to test the hypothesis that beta 1 equal to beta 2 equal to 0 this is the null hypothesis against the alternative hypothesis that H naught is not true. So, basically we need to test the hypothesis H naught beta 1 equal to beta 2 equal to 0. This says that there is no linear relationship between y and the regressors variable against the alternative hypothesis H 1 it says that beta i not equal to 0 for at least 1 i. To test this hypothesis we what we do is that we compare you know we have the f statistics value. So, we compare the f value observed value which is equal to 7.17 we compare this observed value with the f tabulated value f 0.05 and the degree of freedom is 2 8 from the statistical table you can check that this value is equal to 4.46. So, what we observed is that you know the observed f value is greater than the tabulated value. So, we reject we reject the null hypothesis H naught which says that beta 1 equal to beta 2 equal to 0 and we use that means the fitted equation is significance and we use the fitted equation y equal to 14 minus 2 x 1 minus well. So, what the result of this test is that the global test it says that the fitted equation is statistically significant that means there is you know linear relationship between y and any at least one of the response one of the regressor variable. Well, so next requirement or problem 4 I should I say you calculate R square that is the coefficient of determination this is you know basically R square is equal to S S regression by S S total well our S S regression value is equal to 122.6. And S S total is 190. So, this is this R square is one parameter which measure you know sort of the performance of the fitted equation. It measures the proportion of variability in y about y bar that is explained by the fitted equation well. So, the proportion here is 122 by 190 which is equal to 64.21 percent which is not that good you know that means the 64 percent of the total variability in the in the response variable has been explained by the two regressor variable. It is not that you know not that good well. So, next we move to the next problem. So, problem 5 it says that you know you calculate the estimated variance. What we know is that you know beta hat is an unbiased estimator of beta. We know that expected value of beta hat is equal to beta. And also we know that we know that the value of beta hat is equal to beta. We know that the variance of beta hat is equal to sigma square x prime x well. And what you want is that we want estimated value of this variance well. So, here to estimate this variance of beta hat only you know sigma square is not known. So, and also we know that that m s m s residual is an unbiased estimator of sigma square. So, basically we will be replacing sigma square by m s residual well. So, here I can write you know variance of beta hat or the estimate of this one is equal to m s residual into x prime x inverse, which is nothing but 8.5 is the m s residual 8.5 into this matrix x prime x inverse. Which is equal to 4.3705 minus 0.849 minus 0.4086 well. Now, estimated variance of beta 1 is equal to m s residual x prime x inverse 1 1 th element. You know I have used the notation for the variance of beta j hat j j here. So, the meaning of this one is I call this one as 0 0 th element, because this one is corresponds to beta naught. This element is corresponds to this is a variance covariance matrix for beta naught hat beta 1 hat and beta 2 hat. The variance of beta 1 hat is m s residual into the 1 1 th element of this matrix. So, this one is nothing but 8.5 into 0.1690, which is equal to 1.4365. Similarly, the estimated variance of beta 2 hat is equal to 8.5 into m s residual into m s residual x prime x inverse the 2 2 th element, which is nothing but 8.5 into 0.0422, which is equal to 0.3587. Well, so next problem is p 6 it says that what does x 2 contributes given that given that x 1 is already in the regression model. Well, so it basically says you know the contribution of the second regressor that is x 2 in the presence of the first regressor in the model. You know if you can recall the partial test. So, basically here you have to test that test the null hypothesis that whether beta 2 equal to 0 against the alternative hypothesis that beta 2 is not equal to 0. Well, so the meaning of this hypothesis is that what is the contribution of the second regressor x 2 in the presence of the first regressor x 1 in the model. So, the hypothesis to test this is beta 2 is equal to 0 against the alternative hypothesis h 1, which says that beta 2 is not equal to 0. And we know that the test statistic to test this hypothesis is t, which is equal to beta 2 by m s residual x prime x inverse the second regressor x 2 in the presence of the first regressor x 1 inverse 2 2 th element. Now, you know the meaning of this one. So, this is basically the variance of beta 2 estimate of the variance of beta 2. And this quantity is equal to minus 0.5 by just now you have calculated this one this is 0.3587. So, this is going to be minus 0.8348. Now, here t follows t distribution with degree of freedom 8. So, you find out the tabulated value of t 0.025. So, two sided test with degree of freedom 8 this value is equal to 2.306, which is. So, this value is equal to 2.306, which is equal to the observed value the mod of the observed value is not greater than the tabulated value. So, the conclusion is that we accept we accept h naught, which says that beta 2 equal to 0. So, the meaning of you know what we concluded is that x 2 does not have any contribution in the regression model in the presence of x 1. So, let me test the other thing also you know what does x 1 contributes in the presence of x 2 in the model. So, for that I need to test similarly I need to test h naught, which says that beta 1 equal to 0 against x 2. H 1 that beta 1 is not equal to 0 and the test statistic for this one is say t, which is equal to beta 1 hat by m s residual x prime x inverse 1 1th element. This is equal to minus 2 by by 1.4365, which is going to 1.668. And again you know this t this t is not greater than or equal to 0, which is equal to 0, which is equal to 0, which is equal to 0, which is equal to t tabulated value t 0.0258, which is equal to 2.306. So, the conclusion here is that we again we accept we accept h naught that beta 1 equal to 0. That means you know the conclusion of this partial test says that there is that x 1 is not significant or x 1 does not have significant contribution to the model in the presence of x 2. And also the first partial test that is beta 2 equal to 0 that also is accepted. So, the conclusion of the other partial test says that x 2 is also not significant in the presence of x 1 in the model. So, this is you know whereas the global test says that x 1 x 2 is significant that means the model is significant, but individually x 1 is not significant in the presence of x 2 and similarly x 2 is not significant in the presence of x 1. So, this is a nice example you know global test is reject you know global test says that x 1 x 2 are significant to explain the variability in y whether whereas the partial on the other hand the partial test says that neither x 1 is significant in the presence of x 2 nor x 2 is significant in the presence of x 1. So, here I know this example explain the problem of multicollinearity in the regressor variable. So, anyway I will be going to talk about multicollinearity later on. So, this is the result of the partial test. So, what I want to do next is that I want to test how useful you know problem 8, how useful is the regression using the regression x 1 alone. That means if you only consider x 1 in the model, then how much of the variability in y is explained by only x 1 that is what we want to check. So, what I am doing is that I am just removing the second variable x 2 I will be only talking about x 1 and y that means I have the observation like 1 6 4 8 6 5 and I want to fit a simple linear regression between y and x 1 you can check that you know how to fit simple linear regression model. So, you can check that y is equal to the fitted model is y equal to 9.162 minus 1.027 x 1. So, once you have this fitted model you can compute you know y hat you can compute the residual. For example, y 1 hat is equal to 8.135 and then e 1 is equal to minus 2.135. Similarly, for the second observation it is 5.054 and the e 2 is 2.946. So, you do it for all the observations and you compute s s residual which is equal to summation e i square 1 to 11 which is equal to 74 and s s t is same s s t is equal to 190 and s s regression is then equal to 1. So, you can see that y is equal to 16. Now, a annulable source of variation degree of freedom s s m s and the F statistics. So, the regression residual. So, degree of freedom here for total degree of freedom is of course, strain regression has degree of freedom 1 and residual has degree of freedom 9. I hope you understood why it is 9 s s regression is is 116 residual is 74 total is 190. So, m s is 116 and total is 116 and total is 116 and total is 116 and m s residual is equal to 8.2 and the F value is 14.15. Now, you check here this F follows F distribution with degree of freedom 1 and 9. So, you find out the value of F 0 5 1 9 from the table it is equal to 5.12. So, the observed value of F is greater than the tabulated value which is equal to 5.12. That means, you know the regression the fitted model is the fitted model is significant and also one more thing to observe here you know here before this s s regression was 122 and now the s s regression I mean the s s regression involving two regressors was 122 and the s s regression involving one more regression only one regressor is 116. So, perhaps you know x 1 is more capable to explain the variability in in y compared to x 2 because you know out of 122 which was the s s regression before involving the two regressors x 1 and x 2. Now, if you keep only one regressor in the model say x 1 then it is explaining you know almost the same well. Now, one more thing I want to discuss here is that let me define one thing s s residual p. So, this the meaning of this one is that this is the residual sum of square when there are p minus 1 regressor in the model and of course, I hope you know that this s s residual involving p minus 1 regressors in the model this decreases this decreases as p increases so if you involve more regressor in the in the model then s s residual decreases well, but this is not true for m s residual m s residual is equal to s s residual by n minus p this may increase as p increases you know this is not true for m s residual m s residual is equal to s s residual by n minus p this may increase as p increases you know this may increase why it is so just you consider the previous example there you know s s residual initially it was 68 when we had both the regressor x 1 and x 2 in the model. So, I will say this is s s residual for the full model and this one is less than the s s residual for the restricted model restricted model I means the model y equal to beta naught plus beta 1 x 1 plus epsilon. So, for this model also we have fitted this model and we checked that the s s residual for this one is equal to 74 right. So, this one has more regressor that is why the s s residual is less compared to this model this model has only one regressor this model has the full model has two regressors. Now, you compute the m s residual m s residual for the full model that means two regressors is equal to 68 by degree of freedom is 8 which is equal to 8.5 and for this restricted model you compute m s residual for the restricted model which is equal to 74 by 9 which is equal to 8.2. So, the m s residual for the full model is greater than the m s residual for the restricted model. So, this explains you know I said that s s residual always decreases as p increases, but the same is not true for m s residual. The reason is here you know the increase in m s residual occurs when the reduction in you know if you increase one more b s residual variable in the model of course, s s residual decreases, but the reduction in s s residual here for I mean for adding one more regressor in the model is not sufficient to compensate the loss of one degree of freedom in the denominator. So, here if you add one more regressor in the model this one is model involving one regressor x 1 only this one is the model involving two regressors x 1 and x 2, but the reduction in s s residual is not sufficient. So, s s residual is not enough to compensate you know the loss of one degree of freedom in denominator that is why the m s residual for the full model I mean the model involving more regressor is more than the m s residual in the restricted model. So, this is one thing we are going to use later on like m s residual as a parameter to select the best model. We will learn those things later on. Thank you for your attention.