 Okay, then let's get started. Any questions so far on what you have discussed? So, so far we have discussed likelihood function, maximum likelihood estimator, method of moments, bias methods discussed. And there is one more thing called expectation maximization algorithm. I will see that if I have time, I will come back to that. Any questions so far on this? So, we basically looked into how to get the different estimators based on the different principles. One was based on sufficiency principle, then based on likelihood principle, then other methods we looked into method of moments and Bayesian methods. And one thing is these are all like a point estimators. What is a point estimator? You are given a data sample, random sample. Now, you use some statistics on that or you do maximum likelihood estimator on that whatever, you are going to get a value, one value based on your data sample and which you are going to take as a estimate of that underlying parameter. So, from data you convert to one value which you are going to tell, okay, this is an estimate of this underlying. That's like a point estimator. So, later we will see also something called interval estimators. So, there you will say that, okay, this is not point, just point. I am not just giving you a point, but I am given to give an interval and say that the parameter that you will be interested is going to lie in this interval. Okay, now we have seen so many estimators, right, so far based on all this method. Then the question naturally comes to all of us is which among these estimators we get is a good one. Okay, then there has to be some criteria to evaluate the estimators we obtain. So, we are going to start talking about how to evaluate my estimators. Okay, then we will look into one of the fundamental principles or property of these estimators which is going to be captured through this Kramer's law of inequality. Then we talk about some terms related to that called as information inequality and future inequality. So, what should be the criteria for evaluating your estimators? Okay, as we say different methods may give different estimators, what's the criteria? One possible criteria is take just the mean squared error. What's the mean squared error is doing? Suppose let's say w is your estimator and of course this depends on your data samples x which I have suppressed in this definition and this is an estimator for parameter theta let's say. Now I am going to define its mean squared error as simply expected value of difference between this w and the theta. So, theta is a constant, w is a random variable here, it depends on your data. So, you are trying to find the expected value of this squared difference and we are denoting it with theta here that is just to denote that these are all with respect to the parameter theta. So, I am interested in knowing how well this w captures my theta, I am just looking into this is like a w is a point estimator here. I am looking into the difference between these two points and since w is in random quantity, I am going to look at the expected value of this difference, square difference. You may ask why you take the square and then take expectation why don't we simply take this quantity absolute value of the difference and take why don't we take simply this instead of taking the square one. This is also have the same property right it is also penalizing both negative and positive values, but this is like penalizing larger by taking the square yeah. So, from the computational point of view m s is better and one is that is tractability and as we will see it has also one good interpretation. What is that interpretation? Let us write this quantity here, what we can do is notice that this w is not necessary that this w is going to be unbiased I am not making any assumption here ok. This w is just I am starting with an estimator it need not be an unbiased ok. So, if it is an unbiased quantity what is the expected value of w? If it is an unbiased value this is going to be theta if not this need not be this ok. So, whatever is the mean value of the w let us simply take it as expected value w what I will do is then expected value I will do w plus expected value of w minus expected value of w plus sorry minus theta I will do this and this is same as this quantity right and this is same as this quantity see because simply have added and subtracted this quantity. Now, if you simply square it and expand it you will see that one term you are going to get is w minus expected value of w squared which is nothing, but variance of w and another term you are going to get is expected value of w minus theta squared that is the quantity I have written here and in addition there is been another quantity here which is 2 times expected value of w minus expectation of w into expectation of expectation of w minus theta right if I had squared this. So, squaring I have basically grouped them maybe what I should have done is maybe let me do w I will do first I will do w and then minus ok. So, these two quantities and these two quantities I have grouped the first when I expand this the first term will give me variance when I expand the second term will give me this quantity and their cross product is this, but what is the value of this cross product 0 because expectation of w is going to cancel with expectation of w this is going to be 0 ok. So, we will get this now this term here expected value of w minus theta we are going to call it as bias of my estimator w is that fine is that definition. So, if w happens to be unbiased estimator what is its bias is going to be it is going to be 0. So, in this case it is if it is not an unbiased estimator it could be positive, but we are going to simply take that the difference of these two quantities we are going to simply define as bias of my estimator w. So, I just said that if an estimator w for parameter theta ok. So, now we are basically defining earlier we have already said this definition, but we are redefining it. An estimator w for parameter is called unbiased if expectation of w is going to be theta this should happen for all theta and now that means, basically alternately I can define it is bias is 0. So, we are simply putting it like if it is going to be unbiased if its bias is going to be 0 and we know what is the bias mean here ok. Now, let us look into some estimators we have. So, we let us take w of x to be simply x bar what is x bar sample mean we already know this right and x bar we use it used it as an estimator for my mean parameter. Now, if I am going to compute this x bar this is going to be mu ok. So, if I am going to use this estimator to estimate the mean parameter the what is the bias of your sample mean with respect to the mean parameter it is going to be 0 right. So, sample mean has bias 0 bias equal to it is same as saying that it is unbiased. Now, I am interested in variance term. So, this is for mean let us now look into variance. Now, my estimator for variance I am going to simply take it as s square. Now, what is the expected value of s square sigma square. So, is sample variance what is the bias of sample variance to estimate the parameter sigma square that is going to be 0 right. So, sample variance has bias again 0 ok. So, that is fine. Now, let us look into this quantity that does let us look into for each one of them the mean squared error that we are interested in ok. First case I am again taking this w to be with respect to my mean parameter ok I am going to take the mean as mean and we know that this is nothing but variance of w plus bias of w with respect to my parameter mu these are all with respect to parameter mu I am computing now. We just should notice that if it w happens to be a sample mean this quantity is 0 what about this quantity sigma square by n. What is sigma there is the variance of the samples ok. Now, let us do the same thing for variance. Now, I am taking w to be s square that is a sample variance and now I am interested in this quantity. So, first of all what is its bias is going to be 0 what is its variance though. So, what is going to be the variance of your variance estimator? Have you calculated it before? What is the variance of your sample variance? Ok I am going to directly right here this is going to be 2 sigma 4 divided by n minus 1 and this is you just need to do computations ok. So, how you are going to do this? Variance of s square is nothing but expectation of s square minus expected value of s square whole square right by definition this and you know definition of s square what is the definition of s square? Ok, let we will write it later this part, but what is the expected value of s square? We know it is going to be sigma square ok. Then you have what all you need to do is 1 upon n minus 1 summation xi minus xi bar whole square minus sigma square whole square. So, you can compute this expression variance of your s square and I am just writing it for you this is going to be 2 sigma square divided by n by 4. This is for the case of unbiased estimator ok, but now let us look into the case where I am just let us look into the case. So, we always took s square to be 1 upon n minus 1 expectation of we took it as summation xi minus xi bar whole square. But why we deliberately chose it to be n minus 1 though naturally we felt that it would have been s square let us put it as s square prime to be 1 by n summation xi minus xi bar whole square right. So, now let us say that, but this could be also an estimator who is stopping you to take this as an estimator you if this is let us say this is your one estimator for variance this could be your another variance let us call this w prime and I know that this w prime is not unbiased right. So, now let us write it n minus 1 n minus 1 I have just multiplied it and 1 by n and let us write it as xi minus xi bar square. So, this is nothing, but if I do this and now if I want to compare it with this quantity this is going to be n minus 1 by n s square right. My w bar of x is not s square, but it is scaled by factor n minus 1 by n everybody agree this is another estimator. Now, let us compute its mean squared error. So, what is the mean squared error for w by our definition this is w prime minus we are still considering it to be an estimator for sigma square right. So, this is sigma square whole square this I know as sorry variance of w prime plus bias how is bias defined bias is s sorry w prime minus theta sorry sigma square right. So, now let us compute each of these terms. So, simpler is to compute expected value of w prime minus sigma square can you compute it ok. Now, let us take this. So, bias of w prime is this quantity which I am going to write it as simply expected value of w prime minus sigma square everybody agree and now what is w prime w prime I expressed in this format which is nothing, but n minus 1 into s square minus sigma square, but n minus 1 by n is a constant for me I can pull out this from the expectation n minus 1 expectation of s square minus sigma square. But what is expectation of s square I know that to be sigma square minus sigma square and if I simplify that this quantity what I am going to get. So, n and n cancels minus by n and now this is why this is unbiased because this quantity need not be 0.