 this is my fourth tutorial and today we will be solving some problems involving coefficient of determination and also you know model fitting with auto correlated errors. So, here is the first problem. The problem says that your friend says he has fitted a plane to 33 observations on x 1, x 2 and y and his overall regression is just significant at 0.05 level of significance. That means, your friend has fitted a multiple linear regression model with two regressors x 1 and x 2 and one response variable and his test is significant. The fitted model is significant at 5 percent level of significance. Now, you ask him for his r square value that is the coefficient of determination, but he does not know. You work out for him on the basis of what he has told you. So, what are the information given to you is that he has fitted multiple linear regression model with two regressors x 1 and x 2 and his fit is significant at 0.05 level, but he does not know the r square value where r square is the coefficient of determination and it is sort of measured the proportion of variability that is explained by the model. Now, you have to work out this r square value from whatever he has told to you. So, what you know is that your friend has fitted a model like this y equal to beta naught plus beta 1 x 1 plus beta 2 x 2 plus epsilon and this model has been fitted for n equal to 33 observations. So, from this observations I mean from this of information what you can do is that you can construct an overtable for this one. So, here is the an overtable source sum of square degree of freedom and m s and the f statistic. So, the total degree of freedom is 32 because there are only 33 observations and then the regression degree of freedom is 2 because there are three parameters and the residual degree of freedom is 30 and of course, your friend has you know s s regression, s s residual, s s residual and s s total and hence the m s regression, m s residual and f value. So, what you know is that this f is significant. So, this f follows f distribution with the degree of freedom 230 and his test is just significant. So, that means, the observed f value is just greater than or equal to the f value 0.05 at the level of at with degree of freedom 230 which is equal to 3.32. So, you can assume that the observed f value is close to 3.32 and from here you have to compute the r square value. What is r square? r square is s s regression by s s total that is the proportion of variability in y about mean that is explained by the model. But, what you know is that you know f which is equal to m s regression by m s residual. So, you know the f value, you know their degree of freedom but, of course, you do not know separately the m s regression value and m s residual value from f you have to compute r square. So, this is the problem, well we will see whether this r square can be written in terms of f. So, r square which is equal to s s total is s s regression by s s t that I can write as s s regression and s s total is equal to s s regression plus s s residual. And this one I want to express this one in terms of f. So, this I can write as s s regression by m s residual. So, I will divide both the numerator and denominator by m s residual. So, this is s s regression by m s residual plus m s residual plus m s residual by m s residual. So, this one is equal to s s regression can be written as v 1 m s regression by m s residual. Where v 1 is the m s regression by m s residual. So, regression degree of freedom is v 1 and similarly here also I can write that this one as v 1 m s regression by m s residual plus v 2 m s residual plus v 2 m s residual plus m s residual by m s residual. Where residual has degree of freedom v 2. So, this one is equal to now I can write. So, m s regression by m s residual is nothing but f. So, this can be written as v 1 f by v 1 f plus v 2. So, this one can be written as v 1 f by v 1 f and now we can compute r square. So, this is how we know we can express r square in terms of f. So, we know that f is close to 3.32 and we know that our v 1 v 1 is the regression degree of freedom that is equal to 2 in our case and v 2 which is the residual degree of freedom that is equal to 30. So, r square can be now written as 2 in the into 3.32 by 2 into 3.32 plus 30. So, which is equal to 0.1812 that means you know what is r square. R square is the proportion of variability in y about mean that is explained by the model. So, you can see here only 80 percent of total variability has been explained by the model. So, r square is this means only 80 percent of total variability has been explained by the model. So, you can see that from this example r square is a very good parameter to measure how good the fit is. See the test is significant. So, the model is significant according to the global f test, but only 18 percent of the total variability has been explained by the model which is quite low. So, we will consider another problem of similar type. So, this problem says that you are given a regression print out that shows a planar fit to x 1, x 2, x 3, x 4 and x 5 plus intercept of course, obtained from a set of 50 observations. The overall f for regression is 10 times as big as the 5 percent upper tail f percentage point. So, you have to compute how big is r square. Well, the here you are considering a model involving 5 regressors. So, y is equal to beta naught plus beta 1 x 1 plus beta 2 x 2 plus beta 5 x 5 plus epsilon. And this model is fitted on 50 observations. So, we can quickly have an invertible just. So, total degree of freedom just the degree of freedom is important. Total degree of freedom is of course 49 and regression degree of freedom is there are 1, 2, 3 total 6 parameters. So, the regression degree of freedom is 5 and the residual degree of freedom is then 44. Well, so what we know is that we know that my v 1 is equal to 5 here, v 2 is equal to 44. And it says that the observed f value is 10 times the tabulated f value. So, for this test here f has degree of freedom f 5, 44 and you have to find the tabulated value for this one at 0.05 and you can check that this value is equal to 2.43. So, what were given is that your observed f value is 10 times bigger than this one. So, your observed f value is then is equal to 10 times of this tabulated f value that is 2.43. So, which is equal to 24 point and this is and you know v 1, v 2 you know f. So, you can compute r square now. So, problem is that how big is r square. So, you know the formula that r square is equal to v 1 f by v 1 f plus v 2. So, here your v 1 is 5 and f is 24.3. So, 5 into 24.3 plus v 2 is that is the residual degree of freedom that is 44. So, here 73, 43. That means, you have to find the tabulated 73 percent of the total variability in the response variable has been explained by the fitted model which is quite good. So, next we will be considering one problem from regression models with auto correlated errors. So, here is the problem considered the simple linear regression model y t equal to beta naught plus beta 1 x t plus epsilon t where the errors are generated by second order autoregressive process. So, I hope that you can recall. So, here you can see this observations are collected sequentially in time. So, these are time series data basically that is why it is denoted by t here and in case of time series data we know that this epsilon t the error term they are not independent. They are basically correlated and here it is given that the errors are second order errors are having second order autoregressive relation. So, epsilon t is equal to rho 1 epsilon t minus 1 plus rho 2 epsilon t minus 2 plus z t. So, this z t is independent with mean 0 and variance sigma z square. Here rho 1 and rho 2 are called auto correlation parameters. So, the problem is that you know you have to discuss how Cochrane or cut iterative process could be used in this situation. If you can recall we talked about how to fit the regression parameters beta naught and beta 1 in case of first order autoregressive error. So, here instead of first order autoregressive error we have second order autoregressive error. So, this is quite straight forward problem. So it says that what transformation would be used on the variables y t and x t and y t and how would you estimate the parameters rho 1 and rho 2. So, this is the problem how do you estimate this parameter beta naught and beta 1 hat beta 1 and beta naught in the presence of second order autoregressive error. So, let me start the solution for this problem. So, you need to fit this model y t is equal to beta naught plus beta 1 x t plus epsilon t, where this epsilon t is equal to rho 1. This is second order autoregressive epsilon t minus 1 plus rho 2 epsilon t minus 2 plus z t and this z t follows normal distribution with mean 0 and variance sigma t. So, this z square. So, usually you know when you fit a simple linear regression model we assume that epsilon t follows normal distribution with 0 mean and constant variance and they are also independent. So, if this is true then we can estimate beta naught and beta 1 using the ordinary least square technique, but here it is given that this error term they are not independent they are correlated and they follow the second order autoregressive process. So, in this situation how to fit this model. So, what you do is that we transform this response variable y t to y t dashed which is equal to y t minus rho 1 y t minus 1 minus rho 2 y t minus 2. So, this is equal to this is this is equal to this is this is equal to this. So, y t prime is equal to beta naught plus beta 1 x t plus epsilon t minus rho 1 beta naught plus beta 1 x t minus 1 plus epsilon t minus 1 minus rho 2 epsilon t minus 1 minus beta naught plus beta 1 x t minus 2 plus epsilon t minus 2. So, you understood. So, this is my y t this is rho 1 y t minus 1 and then rho 2 y t minus 2. So, this can be written as beta naught minus rho 1 beta naught minus rho 2 beta 1 minus rho 2 beta 1 minus rho 1 beta naught plus beta 1 x t minus rho 1 x t minus 1 minus rho 2 x t minus 2 plus epsilon t minus rho 1 epsilon t minus 1 minus rho 2 epsilon t minus 2. And this one is equal to z t. So, this term is equal to z t. So, I can rewrite this one as say beta naught dashed plus beta 1 call this as x t prime. So, what you do is you transform y t to y t prime and similarly x t to x t prime. So, this is my x t prime plus z t. So, now in this transform model. So, we have transformed y t to y t prime where y t prime is given here. Similarly, we have transformed x t to x t prime and y t prime is given here. As a consequence the error term has been transform to z t. This z t is now independent. Now, z t are independent. So, and also we know that z t follows normal 0 sigma z square. So, the advantage of this one is that now we have a model like y t prime equal to beta naught dashed plus beta 1 x t prime plus z t. And here the error term is error term is x t prime plus normal distribution with mean 0 and variance sigma square and they are also independent. So, now you are in a position to apply your ordinary least square technique to estimate the regression coefficients beta naught prime and beta 1. But, the problem is that you know this y t you see here this y t prime that involves rho 1 and rho 2. Similarly, x t prime involves rho 1 and rho 2. So, we cannot use this transformation unless we know the value of rho 1 and rho 2. So, y t prime and x t prime cannot be used directly as y t prime which is equal to y t minus rho 1 y t minus 1 minus rho 2 y t minus 2 and x t prime which is equal to x t minus rho 1 x t minus 1 minus rho 2 x t minus 2 and x t prime which is equal to x t minus rho 1 x t minus 1 minus rho 2 x t minus 2 are x t prime minus rho 2 x t prime minus rho 2 x function of unknown parameters rho 1 and rho 2. So, unless you know see you are given the data x x t and y t you do not know rho 1 rho 2 unless you do not unless you know the value of rho 1 rho 2 how do you use this transformation. So, we need to estimate them and we know that these are that autocorrelation parameter what is this rho 1 and rho 2 are autocorrelation parameter for this second order autoregressive process rho 1 epsilon t minus 1 plus rho 2 epsilon rho 1 epsilon t minus 1 plus rho 2 epsilon t minus 2 plus z t. So, we need to estimate this rho 1 and rho 2. So, how to do that you know that the E t the observed value of E t epsilon t are t th residual E t. So, what we do is that we fit this model y t equal to beta naught plus beta 1 x t plus epsilon t using ordinary least square technique. And obtain the residual E i. So, we will fit this model we do not ignoring the fact that they are autocorrelated. So, you fit this model and once you have the fitted model you can compute the residuals and then you regress E i on E i minus 1 and then you can compute and E i minus 2 that is you fit a model like E i is equal to rho 1 E i minus 1 plus rho 2 E i minus 2 plus some error z t. So, you know epsilon i's. So, you can fit this model epsilon i's. So, you can fit this model right. So, this is nothing but multiple linear regression model with 2 regressors. So, how do you estimate the parameter rho 1 and rho 2 you compute the least square function s rho 1 rho 2 which is equal to E i minus rho 1 E i minus 1 minus rho 2 E i minus 2 and you minimize this. So, you estimate rho 1 and rho 2 in such a way that this least square function is minimized. And what you do is that you just differentiate this s with respect to rho 1 that equal to 0 will give you one normal equation. And similarly partial derivative of s with respect to rho 2 is equal to 0 this will give you another normal equations let me write down those things. So, E i minus rho 1 E i minus 1 rho 2 E i minus 2 into E i minus 1 this is equal to 0 this is the first normal equation. So, and the second normal equation is E i minus rho 1 E i minus 1 minus rho 2 E i minus 2 into E i minus 2 equal to 0. So, you have 2 normal equations and 2 unknown. So, you can estimate you can find rho 1 and rho 2 call them rho 1 hat and rho 2 hat these are the least square estimate of rho 1 and rho 2. So, once you have this estimated values now you can use this rho 1 hat and rho 2 hat to get the transform values y t prime is equal to y t minus rho 1 hat y t minus 1 minus rho 2 hat y t minus 2. And similarly you get x t prime which is equal to x t minus rho 1 hat x t minus 1 minus rho 2 hat x t minus 2. So, now you can write obtain y t prime and x t prime. So, these are the transform data and now you are in position to apply ordinary least square to the transform data y t prime equal to beta naught prime plus beta 1 x t prime plus z t and here for the transform data you know this z t the error term on transform data this follows normal 0 sigma z square. So, normal with mean 0 and constant variance sigma z square and they are independent. So, you are in position to apply ordinary least square technique and you apply ordinary least square technique here to get the estimated value and the final fitted model is y t prime hat is equal to beta naught prime hat plus beta 1 hat x t. So, this is how we can fit the model in the presence of second order autoregressive error. So, this is called Cochrane Orcut procedure and in a module called regression models with auto correlated errors we talked about the same technique for auto for autoregressive errors with I mean first order autoregressive order. We solve the same problem when there exist a first order autoregressive error and just now we solved for second order autoregressive error. So, let me consider one more problem this is also you know to check whether there exist lag 1 autocorrelation or serial correlation. It says that the following 24 residuals. So, these are the residuals from a straight line fit are equally spaced in time and this is the first order autoregressive error are given in time sequential order. That means, this residuals are obtained from time series data and they are equally spaced I mean the times are equally spaced. So, the question is is there any evidence of lag 1 serial correlation for this 24 residuals. So, it says that you use a two sided test at level alpha equal to 0.05. So, you are given E i for i equal to 1 to 24 and how do you test that there exist lag 1 serial correlation or not. We know that for testing lag 1 serial correlation we need to go for Darby Watson test. So, what we will do is that. So, what is lag 1 correlation is that the correlation between epsilon u and epsilon u plus 1 this correlation is equal to rho which is not equal to 0 if there exist lag 1 autocorrelation. So, what you have to test is that. So, here you can see the errors are one step apart. So, we need to test that H naught rho equal to 0 against the H 1 that rho is not equal to 0. So, H naught says that there is no lag 1 autocorrelation in the residuals. H naught says that and H 1 says there exist lag 1 autocorrelation. So, what we do is that we compute Darby Watson test statistic. What is that? That is D is equal to 0 and equal to sum over E u minus E u minus 1 square u is from 2 to 24 here by E u square for u equal to 1 to 24 and that you can check that you are given E i's right. You are given E u for u equal to 1 to 24. So, we can compute this one this is 2225 by 834 which is equal to 2.67. So, my D is 2.67 and we know that this D is the range of D is from 0 to 4 and it is symmetric about 2. We compute 4 minus D also 4 minus D is equal to 1.33 and what you do is that for testing this two sided alternative we compare this D value compare with D L and D u and this value we will get from the D table. So, for alpha equal to 0.025 because it is two sided test that is why I am taking for alpha equal to 0.025, n equal to 24 and k equal to 1 because it is a straight line fit. So, there is only one regressor in the model that is why k is equal to 1. You can check that your D L and D u is equal to 1.16 and 1.33. Now, we need to think about the critical values for this one. So, what we have is that we have we know that my D is equal to 2.67, my 4 minus D is equal to 1.33 and my D L and D u is equal to 1.16, 1.33. So, if D is less than D L or 4 minus D is equal to 4 minus D is equal less than D L you reject H naught. That means, if the Darby Watson test statistic is small then you reject H naught. Rejecting H naught means there is no autocorrelation. Let us see what is this D value? D is 2.67 which is not equal to D L and similarly, 4 minus D which is equal to 1.33 which is not strictly sorry which is again not less than D L value which is 1.66. So, here D value is large. So, we reject H naught. Rejecting H naught means well so D is not true. So, this is not true. So, we are not going to reject H naught. We accept H naught because as my D which is equal to 2.67 which is not equal to D L which is not less than equal to 1.16. So, this is not true. So, I will accept H naught. So, accepting H naught means no there is no autocorrelation. So, this is not true. So, I accept H naught there is no lag 1 autocorrelation or serial correlation. So, this is the first one and also you can do what you can do is that you check with this one if of course, you will get the same result if D is greater than D U and 4 minus D is greater than D U then it says that you accept H naught which is equal to rho is equal to 0. So, this is the test in terms of D U value D upper values. So, D is greater than D U yes D is 2.67 which is greater than D U 1.33 and also my 4 minus D which is equal to 1.33 is greater than or equal to D U that is 1.33. So, both are true. So, we accept H naught. Accepting H naught means there is no lag 1 autocorrelation or serial correlation in the data. So, we accept H naught means there is no lag 1 autocorrelation or serial correlation in the data. So, we accept H naught means there is no lag 1 auto correlation. Here we have used Darby Watson test to test whether there exist serial autocorrelation lag 1 autocorrelation or not. So, intuitively you know when the D value that is the Darby Watson test statistic whether when that is small that means there exist autocorrelation. And here you can see that the D value is not smaller than the D lower value that is why finally, the conclusion is that there is no lag 1 autocorrelation in the data. So, we talked about this problem and still we have some time. So, we can sort of solve this problem recall the parameter estimation technique in case of in the existence of first order autoregressive errors. So, this is what the Cochrane Orcote method you know and this says that you know how to estimate this parameter beta naught and beta 1 when the data is not in the data there exist first order autocorrelation or among the errors well and this epsilon t they are not independent 0 sigma square they are having the first order autoregressive error. And, what we did there is that we transform y t to y t prime which is equal to y t minus rho y t minus 1 and we can finally, check you can finally, check that this y t prime is equal to beta naught prime plus beta 1 x t prime plus z t and here z t is equal to epsilon t minus rho epsilon t minus 1 and we know that this z t this transform error they are independent and that is why you know you can use ordinary least square technique to this transform data but the problem again is that you know this x t prime and y t prime they involve some unknown parameter rho and we talked about how to estimate those parameters. Those rho I mean the similar technique I mean just know repeating the things here. So, you feed the model using the ordinary least square technique forget ignore the thing that assumption is not true here and fitting the ordinary least square technique you will get the residuals E i's and then you regress E i on E i minus 1 right. You feed this model that is E i is equal to rho E i minus 1 plus z t and the least square estimate of rho is this one you can check that you know you have to estimate rho in such a way that this is minimum. So, the rho value is equal to this and using this rho now you can transform the data y t to y t prime similarly x t to x t prime and now you know that this transform data or the transform model has error z t which is normal 0 sigma square and they are independent. So, you can apply ordinary least square technique to this transform data and you can happily you know estimate beta naught hat and beta 1 hat. So, this is sort of just you know we solve the same problem for second order autodegressive error today. Well, so that is all for today again in the next tutorial class we will be solving some randomly selected problems. Thank you very much.