 One of the more important ideas in probability and statistics is known as hypothesis testing, and one of the central ideas of hypothesis testing is known as the null and alternate hypotheses. These are based around the two fundamental problems of probability and statistics. In probability, the problem is this. Given the true state of the world, predict the outcome of a random experiment. So for example, if you know that, in fact, a coin will land heads with probability one half, what will happen if you toss a coin ten times? On the other hand, in statistics, we have the fundamental problem given the outcome of a random experiment infer the true state of the world. So suppose we see a coin land heads six times out of ten. What is the probability the coin lands heads? Now generally speaking, answering a question like what is the probability of something is a little too hard for us to answer. And to make it easier to answer these statistical questions, we consider two hypotheses. The first is known as the null hypothesis, sometimes indicated H0, and the other is called the alternate hypothesis, indicated HA. And you can think of, as the ultimate goal of statistics, the question, which of these two hypotheses do I choose to believe? More formally, we can state the problem this way. How likely is the observed outcome if the null hypothesis is the true state of the world? As an introductory example, suppose we have a bag which either consists of all black jelly beans, or a mixture of black and green jelly beans. Now we don't know which bag we have, so to decide we'll pick out a handful of jelly beans and find that it contains a mixture of black and green jelly beans. What do you conclude? Well, any normal person would say that we have the bag of green and black jelly beans, but statisticians are not normal people. So let's think about this statistically. The null hypothesis is that the bag consists of all black jelly beans. We'll discuss why this is the null hypothesis later, but for right now we'll just take this as the null hypothesis. The observed outcome is a handful of black and green jelly beans. And remember the problem that we want to answer is how likely is the observed outcome if the null hypothesis is the true state of the world? And so we can ask the question, how likely is it that we get a mixture of black and green jelly beans from a bag consisting of all black jelly beans? And that probability is zero. It is an event that cannot happen. And so we reject the null hypothesis. What's important to understand here is the logic. You do not get to choose your facts. The observed outcomes are what they are. So you have a choice between two hypotheses. One hypothesis requires a fantastic stroke of luck in order to produce the observed event. In order to reconcile your belief with your observations, you either have to reject all of the evidence or you can change your mind. Now if you're a politician, you reject the evidence. And if you're a thinking person in a free democracy, you change your mind based on the evidence. So let's go back to that first question. Why did we choose bag consisting of all black jelly beans as the null hypothesis? And here it's useful to keep in mind the following idea. You might be a null hypothesis if you could make a specific quantitative prediction and you could be falsified by an observation. For example, let's consider our two possibilities. Either all jelly beans in the bag are black or some jelly beans are black and some are green. So again, let's go back to our bag and reset so we don't know which bag we have. Now the hypothesis all jelly beans in the bag are black allows for a specific prediction. Every jelly bean we pick will be black. In contrast, our other hypothesis doesn't allow us to make a specific prediction. The best we can say is some are going to be black and some are going to be green and that's not very specific. The second important observation is that this hypothesis all jelly beans in the bag are black can be falsified. If we pick out a jelly bean and it's not black, we know this is false. In contrast, this second hypothesis, some jelly beans are black and some are green, no observation can falsify. And the next jelly bean we pick could be green. So for example, if you're trying to decide whether a coin is fair or not, so you flip the coin 100 times and record the result. Identify the null and alternate hypotheses. Well the two hypotheses are the coin is fair or the coin is unfair. And so the first question is, will one of these allow us to make a specific quantitative prediction? And this first hypothesis will. This allows us to make a quantifiable prediction. The coin will land heads about 50 times. Human psychology being what it is, sometimes it's easier to see when something doesn't work. This second hypothesis does not allow us to make a quantifiable prediction. The coin will land heads, I don't know how many times. And so what that says is that the coin is fair is going to be our null hypothesis. Now an important idea for what's known as Bayesian statistics is the following. You're trying to decide whether a coin is fair or weighted so it lands heads 40% of the time. To do so, you'll flip the coin 10 times and record the number of heads. Identify the null hypothesis. And again we have two possibilities. The coin is fair or the coin is weighted to land heads 40% of the time. Our first hypothesis leads to a quantifiable prediction. The coin will land heads 5 times in 10 flips. What's important about this example is that the other hypothesis also leads to a quantifiable prediction. If the coin is weighted to land heads 40% of the time then the coin will land heads 4 times in 10 flips. Both of these hypotheses lead to quantifiable predictions which means either could be the null hypothesis. Now you might wonder why do we call this the null hypothesis? Why don't we just call it the hypothesis that gives us a quantifiable prediction? And this example suggests why that terminology is used. Suppose you're trying to decide whether a treatment increases the growth rate of plants. On that end you grow 5 plants without treatment and 5 plants with treatment. At the end of the experimental period you measure the heights of the plants and let's identify the null and alternate hypotheses. So there are two hypotheses. First the treatment did not affect the growth rate. And second the treatment did affect the growth rate. This first hypothesis produces a quantifiable prediction. The height of the plants will be the same in both groups. The second hypothesis, not so much. The treatment did affect the growth rate does not produce a quantifiable prediction. The best we can say is the height of the plants will be different. So in this case notice that the null hypothesis says there is no difference between the two samples. And in general in many cases the null hypothesis corresponds to no difference between the two samples.