 Let's get started with today's lecture. So in the previous lecture after talking hypothesis testing, we discussed about how to construct confidence sets, how to get interval estimations and from that how to get confidence intervals and we also discussed about how hypothesis test can be used to get confidence test. So in this test, we are going to talk about, given a test to estimate some parameter, how good is it, like how confident we are in that about our null hypothesis, for that we are going to define study various test like p test, t test and f test and how to quantify them using something called significance of a test. Okay, so let's get started with p test or rather p value, maybe it should be more appropriate to call it a p value rather than p test. Okay, as a motivation, suppose let's say you have samples and let's assume that each of this xi is coming from Gaussian with parameter mu and sigma square. You have already seen that if I take their empirical mean, do this centralization and this normalization, assume that right now I don't know mu but I know sigma square. I know that this is going to be Gaussian distributed with mean 0, 1 and now I am interested in estimating mu hat. I know one estimate is 1 by n summation xi, I want to, now the question is now our claim is mu hat is the population mean, but now the question is why one should take it as a population value, what you are giving is simply an estimate, if somebody has to believe in whatever you are giving is going to be a good representation or in fact the true value of the population mean how one can believe it or what should be the criteria and that is where p values comes into picture. Now we are also denoting it as x bar but this is one possibility, we know that x bar minus mu divided by square root of sigma square n is a normally distributed with parameter mu and 1 and we know well about its distribution especially about its CDF and its tail behavior. We can use this property to see how good is my claim that mu hat is a good representation of the underlying population mean. So for that one possible way is let's say I am going to define z to be x bar mu and sigma square and now what I am interested in is what is the probability that my, what is the probability that my x bar is going to be close to the population mean. Now think about this, let's draw normal distribution and let the mean is here mu. Now for a given sample, so this is like a z, let's say for a given x, this is my realized sample I got my x bar and I get a particular realization of this which I am going to denoted as mu by sigma square by mu sorry n where n is the number of samples we have. Now if suppose I get for a given realization z here, maybe this is a good representation of my population mean but suppose this z happens to be here instead of here that means it is little far away from you if z happens to be here maybe this is actually not a good representation of my underlying population mean. Now depending on how far this z value the estimated value I got is going to be away from the true population mean maybe I can say something about this. Let's say if this is this quantity is very small maybe I have a good confidence that yes whatever the z value estimate I got it is going to be a good representative of my underlying population mean and if this is large maybe that's not the case. So to do that instead of directly looking into z what to be instead of directly looking into the x bar okay so maybe this should have been x bar here not z like so suppose let's say is x bar is here if they are close I have a maybe more confidence or I may be more happy that this is representation of my true population mean and if this happens to be little far away that's not the case. So naturally what we should be looking into is the difference like if the difference is small maybe it is good if the difference is large maybe that is not a true representation of my population mean and also in this case the variance plays the role like how this is the dispersion around your population mean. So because of that instead of just taking the difference x bar minus mu maybe I would also want to normalize based on the variance and that's what like we are defining z which is the difference divided by the variance okay now let's ask the question what fraction of the time the difference is going to be small if the fraction of the time this difference is going to be small then maybe it is good your test is good and if the fraction of the time if this difference is going to be large maybe you are not actually capturing the true population mean. So using this quantity what we are going to sorry using this quantity what we are now going to call it as a statistic in this case we can ask the question what is the probability that my z is going to be greater than or equals to z so notice that the z is what you have computed based on your sample. If this z happens to be large let's say let's say somewhere here let's say now I am going to look into a Gaussian distribution with the mean 0 and variance 1 and now this is mean 0 and if this happens to be small let's say I this z happens to be here that means that this probability is going to be probability z like I will be basically covering the probability of all this so if this z is small my probability is going to be higher on the other way if this z happens to be let's say here then this probability is going to be smaller. So based on that we can think that if this probability is large and I am going to call this value as my p value p is equals to probability sorry probability that p is equals to probability that z is going to z and if the probability of this happening is let's say large that means basically my z is small that means this is actually good so maybe I would like to accept maybe like if large I would like to accept the null hypothesis okay so null hypothesis I mean to say this so your test is what your test is basically now trying to check whether these samples are coming from a distribution like my null hypothesis here probability that my my let's say my theta is mu and my null hypothesis alternate hypothesis is that your theta is not equals to mu or maybe yeah so if this quantity is large it is likely that the samples you are observing and by the way this probability is under you were assuming that your samples are coming from your to distribute true parameter that you want to test against if this quantity is large then you want to accept the null hypothesis saying that yes indeed they are coming from mu and if small if not or small then maybe it's possible that your samples are not coming from this parameter mu that is because in this case the z is is large and that is why this probability is going to be small so if if not or small then you may want to consider the alternate hypothesis or it is equivalent to saying that you want to you you want to consider alternate hypothesis maybe like in this case possibly your null hypothesis is not true and this is kind of a providing a evidence that maybe you are claimed that it is to be null hypothesis is incorrect and if large maybe actually what we will do is okay let's let's put it slightly differently so in this setup what we are saying let's say you having sample and you have this hypothesis to check one is you are going to check whether these are coming from with the population mean mu and alternate hypothesis what we are going to call is something challenges somebody's your claim is that this is going to be a null hypothesis with parameter mu and somebody is going to challenge you saying no no this is not the case it is something else now in that case if this quantity happens to be small then it is kind of indication that you are null hypothesis may not be true and alternate hypothesis is true and on the other hand if this probability is large it is in favor of null hypothesis it is basically saying that it is saying that not basically you can say that let me see what's the correct word here you can say that here you can say that researcher failed to reject the null hypothesis so in this setup we are basically saying that whatever the test you have provided to verify your null hypothesis whether to accept or not and you have a challenger here somebody is challenging you and if you could show that this probability or this p value in this case is large then you can say that researcher failed to reject your null hypothesis ok and on the other hand this is small then you are basically validating that are basically the researchers validating that whatever your claim on the null hypothesis is incorrect ok so then the question arises here is what is this large and small how to quantify this is there a way and that is where we need to put some threshold here and significance level come to picture so you want to say that the p value you have let us say z is greater than or equals to z if this is less than or equals to c then you can say that let me see what is the exact term I can use then we can say that you are null hypothesis is false on the other hand if this p value you have is greater than alpha then null hypothesis is not rejected and then question comes what is the good value of this alpha usually alpha is taken to be 0.05 if your p happens to be smaller than 0.05 then you are claim that the null hypothesis is true is not accepted or it is your null hypothesis is false on the other hand if p happens to be larger than 0.05 then your opponent who was challenging you has is not providing sufficient evidence to say that your hypothesis is false or in other words the null hypothesis is not rejected you are still in the business ok. So, because of that if you are making a hypothesis and the p value for that hypothesis the p value computed under that null hypothesis happens to be larger than 0.05 then your hypothesis is not rejected and so you should always make sure that if you are making a claim on a null hypothesis the p value should be as large as possible or at least it should be more than 0.05 and this 0.05 is just like a kind of a thumb rule people take it is not necessary that one has to stick to 0.05 depending on your application if you feel that my application is such that I do not need to be too stringent or I need to be relaxed depending on that you can set your alpha to be maybe any value between maybe let us say 0 to half or whichever is more applicable ok. Now when you are doing this test there are two possible ways right like when you are making the hypothesis one is let us called single side and another is like a two side. In the single side like let us say your hypothesis is like let us say theta is some mu and your alternate hypothesis is just saying that maybe just like your theta is greater than mu. They are just saying that no like somebody is claiming that the true parameter is going to be mu and you are challenging by saying that no the true parameter is simply larger than mu this is like one sided. On the other hand the two sided say that ok you are going to say that ok the true null hypothesis is simply parameter mu but your challenger may say that no he simply say that no this is not mu it is something else it could be larger or smaller that is called two sided. But in both the cases like this is like standard normal so if it is like one sided you have said here you will look into this this is like a one sided case but if it is a two sided you need to consider this probability as well. So, both this together will give you the probability the p value for the two sided test and the good thing about this p values is you can readily compute them by looking into the tables for your standard normal distribution right like I mean if I give you z you know already given z we can readily get p value from Gaussian tables or maybe maybe just I can just say normal tables ok. Yeah one point to note here is this normal tables they usually are given for probability like z is equals to less than or equals to z that is basically cdf of your normal distribution. But notice that the p values are the complement of this. So, the p values we need to get the complement of this by simply taking the negation of that and by doing this you will end up finding this value which are to the right of your z value ok fine. So, notice that basically this all worked well when your samples are Gaussian distributed or this quantity x bar minus mu normalized by square root of sigma square by is normal distributed. And here another crucial thing we notice this this sigma square is known. Now the next question happens is how does this change when the sigma square is unknown and which could be off on the case ok. We will next study that by looking into our t test. So, we actually so far computed the p value yeah before I jump into the t test I want to just conclude what this t test is by looking into an example. Suppose let us say you have a you are in a case where a statistician wants to test the hypothesis that the true population mean is 120 and somebody is challenging him saying that ok the alternate hypothesis is the one sided value of mu being greater than 120. And the threshold the significance level is been said to 0.05 here and the number of samples we have is 40 the variance is taken to be 37.5. And from the samples it has been computed that the sample mean is 105.37. Now using the p value we want to conclude whether this x bar is statistically significant or not. So, now how to do this first we are going to compute the statistics for that I need the variance of my estimator. So, I have this x bar and I know that its variance is going to be sigma by square root n and sigma has been told to me as 37 32.17 n is 40. So, I got this value 5.0865. Now I will compute my I have been denoting this as z value. So, the z value here is simply x bar minus 120 and divided by. So, notice that here you actually know this 120 because that is the one Hull hypothesis against which you are testing. And here you will end up with the z value which is minus 2 8 minus 2 0.8762. Now for this using z score table you can calculate z being larger than minus 2 8762 in the following fashion. We know that from the Gaussian tables probability that t is less than or equals to this because of the symmetry this is also equals to t greater than 2.8762 is 0.003. And now probability that t being greater than minus 2872 is 0.997. So, the p value we have obtained it to be 0.997 which happens to be significantly larger than my significance level of 0.05. So, therefore, in this case the p test says that my the p test leads to the conclusion that we have failed to reject the null hypothesis. That means, null hypothesis claimed by the statistician is still valid and there is no evidence to provide that this is not true that is basically we have failed to reject the null hypothesis.