 So, today, we will be talking on a new topic called measurement errors and calibration problem and here is the content of this topic. So, case where the response and regressors are jointly distributed random variable and measurement error in regressors and also we will be talking about the calibration problem which is called inverse problem. Let me talk about the objective of this topic in almost all regression model we assume that the response variable is a random variable where the regressor variable like x 1, x 2, x k they are called controlled variable they are not random variable. So, what we will do in this topic is that we will talk about two variations of this situation and as I told you know that y is a random variable and x is a controlled variable which is not a random variable. Let me just recall why I say y is a random variable. So, in simple linear regression model what we consider the model is like y equal to beta naught plus beta 1 x plus epsilon and we assume that epsilon is a random variable which follows normal distribution with mean 0 and variance sigma square. So, this is the assumption you know we make in simple linear regression also in the multiple linear regression model. So, assuming that you know sigma is a random variable which follows normal distribution and they are independent if you put i here is same as you know assuming that y is a random variable. So, what we assume is that the response variable y is a random variable and the regressors like x 1, x 2, x k they are not random variable they are called controlled variable or also we called you know deterministic variable. So, here we will be talking about two variation of this situation the first one is both response and the regressor variable are jointly distributed random variables and the second case we will be considering the second variation is there are measurement error in regressors. So, let me talk about the first variation where both the response and the regressor variables are jointly distributed random variable. So, let me assume that the first case that x and y that is the regressor and the response variable are jointly normally distributed. So, usually x is not a random variable, but here we are considering both of them are random variable and they are jointly normally distributed. So, then the joint pdf probability density function of x and y is 1 by 2 pi sigma 1 sigma 2 1 minus rho square exponential 2 1 minus rho square y minus mu 1 by sigma 1 square plus x minus mu 2 by sigma 2 square minus twice rho y minus mu 1 by sigma 1 into x minus mu 2 by sigma 2. So, this is the joint pdf of x and y when they are jointly normally distributed. So, here expectation of y is equal to mu 1 variance of y is equal to sigma 1 square expectation of x is equal to mu 1 sorry mu 2 and variance of x is equal to sigma 2 square and there is one more parameter that is called the correlation coefficient between x and y. So, the rho is the correlation coefficient which is equal to the covariance between x and y. So, y minus mu 1 into x minus mu 2 by sigma 1 sigma 2. So, notation for this one is sigma 1 2 by sigma 1 sigma 2. So, this is the correlation coefficient between y and x and once you have the joint distribution we know we can find the conditional distribution also. So, let me write down the conditional distribution of y given x is equal to mu 1 by sigma 1 by y given x follows normal distribution with mu 1 plus rho sigma 1 sigma 2 x minus mu 2 and variance 1 minus rho square sigma 1 square. So, this is the conditional distribution of x of y given x which is normally distributed with some mean and variance. So, what I can write here is that the expectation of y given x is equal to mu 1 plus rho sigma 1 by sigma 2 is equal to mu 1 plus rho sigma 1 by sigma 2 x minus mu 2 and this I can write as beta naught plus beta 1 x right where my beta naught is equal to mu 1 plus mu 2 plus mu 2 plus mu 1 minus rho mu 2 sigma 1 by sigma 2 and beta 1 is equal to rho sigma 1 by sigma 2. So, why I derived all these things because just to say that here you can see the conditional expectation of y or the expectation of y given x is this one. So, this is the model we need to consider when both x and y are random variable and what we do when x is not a random variable when x is a controlled variable or deterministic variable what we do is that we fit the model expectation of y which is equal to beta naught plus beta 1 x. So, this is the model we fit in the case when y is a random variable and x is a deterministic variable. It is not a random variable. So, writing this is same as y equal to beta naught plus beta 1 x plus epsilon where epsilon follows normal 0 sigma square. So, similarly this is the difference only. So, this is the model we need to consider here whereas when both x and y are random variable and this is the model we consider when y is a random variable, but x is not a random variable that is you know in almost all cases this is the situation that y is a random variable, but x is not a random variable it is a controlled variable. Now, how to fit this model because we are talking about the case when both x and y are random variable right. So, from here we know the conditional distribution of y given x. So, from here we can say that this y i given x i this follows normal distribution with mean beta naught plus beta 1 x i and the variance is 1 minus rho square sigma 1 square and they are independent. So, I am given the observations y i x i both are random variable and I know that this is true that is the random variable y is 1 minus rho square sigma 1 square given x is random variable y i given x i are independent random variable and they follow normal distribution with mean beta naught plus beta 1 x i and variance a constant variance. Now, to estimate this parameter and we want to fit the model y given x is equal to beta naught plus beta 1 x right. So, fitting this model means we need to estimate the coefficients beta naught and beta 1. So, what we will do is that we will go for maximum variance likelihood estimator because here we know the distribution of conditional distribution of y given x i. So, the likelihood function is and they are independent. So, the likelihood function of y 1 y 2 y n given x 1 x 2 x n is just the product of marginal density. So, that is nothing, but 1 by root over of 2 pi sigma 1 square 1 minus rho square and e to the power of minus 1 by 2 sigma 1 square 1 minus rho square 1 by 2 sigma 1 square y i minus beta naught minus beta 1 x i square for i equal to 1 to n. So, this is the likelihood function and this can be written as 1 by root over of 2 pi sigma 1 square 1 minus rho square to the power of n e to the power of minus 1 by 2 sigma 1 square. 1 minus rho square summation y i minus beta naught minus beta 1 x i whole square. So, what the maximum likelihood estimator technique suggest is that you construct the likelihood function and here you can go for and find the parameter beta naught and beta 1 in such a way that the likelihood function is maximum. So, maximizing this likelihood is same as minimizing this thing. So, we find beta naught and beta 1 such that this is minimum. So, this is so find beta naught and beta 1 such that y i minus beta naught minus beta 1 x i whole square i equal to 1 to n this is minimum. So, this is nothing, but the least square function we consider when while estimating beta naught and beta 1 using the least square technique in simple linear regression model. So, we know what is beta naught hat and beta 1 hat. So, beta naught hat is equal to y bar minus beta 1 hat x bar and beta 1 hat is equal to summation y i minus y bar x i minus x bar summation x i minus x i whole square. So, this is nothing, but s x y by s x x. So, these are identical to those given by least square estimate because we are minimizing the same function here. We are minimizing the least square function here also in case where x is a controlled variable. So, here we are trying to estimate the model expectation of y given x which is equal to beta naught plus beta 1 hat sorry beta naught plus beta 1 x and we found that the estimate the maximum likelihood estimate for beta naught and beta 1 are the same as obtained by least square technique in case when x is a controlled variable and here we have a new parameter called the correlation coefficient rho. So, this is the correlation coefficient between x and y and what we want is that we want to draw some inference about this correlation coefficient and first we find an estimator for this one. The estimator of rho which is a correlation coefficient is just simple is the sample correlation coefficient sorry is sample correlation coefficient that is equal to r. So, r is the sample correlation coefficient which is y i minus y bar into x i minus x bar right by square root of summation y i minus y bar whole square into x i minus x bar whole square. So, this is the sample correlation coefficient and this is the estimator for population correlation coefficient rho and this can be written as in the standard notation s x y by square root of s x x s y y. So, s y y is this one and we know that this one is also called s s square sorry s s total. So, I will write this as s x y by square root of s x x and then s s square root of s x y and then s s square root of s total well and now note that whether we there is a relation between this sample correlation coefficient which is an estimator for rho and the regression coefficient beta 1. So, we know that beta 1 hat is equal to s x y by s x y by s x y by s x y by s x y by s x which can be written as in terms of r this can be written as s s total by s x x square root of this into r you can check that just plug r here you will get back this one. So, this says that you know beta 1 hat and r are closely related and also what we will do is we will see what is r square here. So, r square is equal to r square is equal to r square from this expression r square is equal to s x x square you know s x x by s s total into beta 1 hat square and this one can be written as we know that beta 1 hat is s x square root y by s x x. So, this can be written as beta 1 hat into s x y by s s total you can you can check that you know just take out 1 beta 1 hat and plug this value here and we know that this one is nothing but s s regression this is interesting. So, s s regression by s s total and you know what is this quantity this is called the coefficient of determination that is the capital r square this is called coefficient of determination. So, what does this r square do is that it measures the proportion of variability of the variable in the response variable that is explained by the regression model and what does r do is that it measures the linear association between x and y and here we observed that you know r square is equal to the small r square which is the sample correlation coefficient is equal to the capital r square which is the coefficient of determination. Well, so what you do is that we just we have a new parameter rho which is the correlation coefficient between x and y and we will learn how to test the significance of this I mean whether this correlation coefficient is significant or not by testing the hypothesis that h naught is rho equal to 0 against h 1 that rho is not equal to 0. So, this is a useful test and the test statistic for this hypothesis is t naught which is equal to r root over of n minus 2 by 1 minus r square square root of this thing and this function of this follows t n minus 2 degree of freedom under h naught and here is the rejection criteria you reject the h naught if the modulus value of this one is greater than t alpha by 2 n minus 2. So, here we learn about how to test this test the correlation coefficient is equal to 0 against the alternative hypothesis that the correlation coefficient is not equal to 0. Well, so we talked about one case here where both the regressor variable x and the response variable y both of them are random variables and we observed that the linear model we need to fit here is very similar to the case when y is a random variable and x is a controlled variable that is the situation in almost all cases and here we assume that the random variable x and y they jointly follow the normal distribution by variate normal distribution and so we have a new parameter called rho here which is the correlation coefficient between these two random variable and if you see and we learned how to test the hypothesis that rho is equal to 0 against rho is not equal to 0. So, if you see from this testing that you know rho is equal to 0 that means there is no linear relationship between the regressor variable and the response variable which is same as you know testing the beta 1 is equal to 0 against beta 1 is not equal to 0 they are also we say that if beta naught sorry if the beta 1 is equal to 0 then there is no relationship between x and y. So, next we will be talking about the another deviation we talked we mentioned that is measurement errors in regressors. So, here we wish to fit the simple linear regression model, but the problem here is that, but the regressor, but the regressor is one of the errors measured with error. So, what I mean by this here suppose x i is the observed value of the regressor. So, this is the observed value. So, in usual case you know we considered that y is sorry x is a control variable and then there is no error while measuring the value of x i which is equal to small x i, but here you know the regression is measured with error. So, x i is equal to small x i plus a i. So, what is this small x i small x i is the true value and this a i is called the measurement error. So, I will give an example to illustrate this situation. Let me consider this example suppose x i is the true value and this a i is x is regressor variable which stands for the current flow in an electric circuit and the current flow is measured with an ammeter which is not completely accurate. So, here a measurement error is experienced. So, this is the observed current flow capital x and the small x is the true current flow and this one is the measurement error. So, what we are given is that we are given the observation say y i that is the response variable and we are given x i capital x i we are given the observed value, but we want to find linear relationship between y i and the true value of the regressor variable. So, we will be talking about this how to deal with such situation now. Let me go to my previous slide here is the observed value you understood this is the observed current flow this is the true current flow and here is the measurement error and here we make the assumption that with this expected value of measurement error is equal to 0 and variance of a i the constant variance sigma a square and of course, the response variable is a random variable the response variable is a subject to the usual error epsilon i for i equal to 1 to n well. So, now what we want is that we want to find a relationship between the response variable y and the true value of x. So, we want to fit we want to consider the model the regression model is y i is equal to beta naught plus beta 1 x i plus epsilon. So, here only the problem is that we do not have a small x i this is the true value right and, but what we have is that we are given y i the response variable and the capital x i that is the measured value of the regressor variable. So, this can be written as beta naught plus beta 1 in terms of capital x i i can write this as x i minus a i because of the fact that we assume that the measured value is equal to the true value plus the measurement error plus epsilon. Yeah, because see we are given y i and capital x i. So, we have to convert this model in terms of capital x i that is quite clear. So, this is equal to beta naught plus beta 1 x i plus epsilon i minus beta 1 a i. So, this is equal to beta naught plus beta 1 a i. So, this is equal to beta 1 capital x i I should write I mean I should not mix here capital x i and small x i I used to do that before I mean because both of them are same, but here I have to be careful. So, this is equal to gamma i. So, where this gamma i is equal to epsilon i minus beta 1 a i. So, now, this appears to be the result now we have the model in terms of capital x i y i is equal to beta naught plus beta 1 x i plus some error term. So, you may think that we are done because this is the model we know how to fit this model before also, but the problem here is that see this capital x i is a random variable and this gamma i is also a random variable. Now, we need to check whether before when x was controlled variable there was no correlation between these two. Now, here we need to check whether there is a there is whether they are correlated or they are independent. So, what we will do is that we will compute the covariance of x i and gamma i which is nothing but expectation of x i minus e x i into gamma i because expectation of gamma i is of course equal to 0 because both expectation of epsilon i is equal to the expectation of a i that is 0. So, here I can write that this is equal to capital X i now the expected value of capital X i is equal to small x i that is small x i and this one is equal to epsilon i minus beta 1 a i. Now, this one is equal to expectation of what is this capital X i minus small x i is equal to a i. So, a i into epsilon i minus beta 1 a i right and we assume that you know perhaps I forgot to mention this that here while I was talking about this model here we assume that expect these two are independent a i and epsilon i a i and epsilon i they are independent this is equal to 0. So, then this one is equal to minus beta 1 expectation of a i square. So, expectation of a i square is nothing but the variance of a i square which is equal to minus beta 1 sigma a square. So, here as you see that you know this observed value of X i of the regressor and the model error they are correlated. So, you cannot apply the standard or the ordinary least square technique to estimate the parameter beta naught and beta 1 here. So, it says that so, if we apply this standard least square method to the data we estimates the estimates of the model parameters are no longer unbiased. So, if you apply just simple or ordinary least square technique what we will get is that beta 1 hat is equal to summation y i minus y bar into capital X i minus X bar because this is the observed regressor value by summation X i minus X bar whole square. But you can check that the expected value of this beta 1 hat is equal to beta 1 by 1 plus theta that is the beta 1 hat we got here is not an unbiased estimator of beta where where theta is equal to sigma a square by sigma X square again the sigma X square is equal to you need to check you know this one sigma X square is this. So, what this indicates is that if beta 1 hat is a biased estimator of beta 1 unless this theta is equal to 0 that means unless sigma a square is equal to 0. So, sigma a square is the measurement error variance. So, this will be 0 that is there is no measurement error in regressor. And also you know if this sigma a square is very similar to sigma a small relative to sigma X square the bias will be then theta will be small. So, the bias will be I mean once theta is small this quantity is almost close to 1 then the bias will be small. So, finally the technique says that you know if if variability in the measurement error that is sigma a square is small relative to the variability of the X value then it suggests that the measurement error will be the measurement error can be ignored and ordinary least square method can be applied. So, here we have learnt you know how to how to fit a model in the presence of measurement error in the regressor variable. And next what we will be doing is that we will be talking about the calibration problem which is also called the inverse problem. So, here usually you know given a value of regressor variable X we estimate the response variable Y here the problem is just opposite here you know you are given a value of Y you have to estimate the corresponding regressor value that is called the calibration problem. So, here is the calibration problem. So, here is the statement of this problem is that given observed value of Y say Y naught you have to determine the X value corresponding let me give an example know why we need this calibration problem. The example is we know that the temperature reading given by thermocouple is a linear function of the actual temperature. So, what I mean by this is that the observed temperature this observed temperature given by this thermocouple is a linear function of the actual temperature. So, this is observed temperature is equal to beta naught plus beta 1 actual temperature plus epsilon. So, in such situation you know the observed temperature and you want to know the actual temperature. So, given a value of Y say some observed temperature Y naught you can determine the corresponding X value that is the actual temperature. So, this is what the purpose of this calibration problem is and how do we solve this problem is that. So, suppose we have some Y i X i values for i equal to 1 to n. So, what you have to do is that you just first feed the model Y hat equal to beta naught hat plus beta 1 hat X and let Y naught be the observed value of X sorry observed value of Y and what you want to know is that you want to know the value of X for which Y equal to Y naught from this straight line feed. So, a natural point estimation of the corresponding value of X is say call it X naught hat is equal to just put Y naught here and then what is the X naught hat corresponding is equal to Y naught minus beta 1 sorry beta naught hat by beta 1 hat assuming that assuming that beta 1 hat is not equal to 0. So, this is a very simple problem like the calibration problem is called the is also called the inverse problem. So, here you given a value of X sorry given a value of Y say Y naught you have to find the corresponding X. So, once you have a fitted model there is no problem finding a point estimation for the corresponding X values. So, what we have learned in this topic is that the usual situation in almost all cases what happen is that Y is regressor variable which is a random variable and X is regressor variable and it is usually deterministic variable or controlled variable and here we have learned about two variations of this situation unlike when X is a random variable X and Y both are random variable then how to fit the model and also we have learned when there is a measurement error in regressor variable how to deal with that situation also. So, we need to stop now. Thank you.