 So, let's talk Shakespeare. Well, actually, let's talk more statistics. When we test a hypothesis, we can either reject the null hypothesis, and if we do, we say the evidence is statistically significant. The other thing we can do is we can fail to reject the null hypothesis, and in this case we say the evidence is not statistically significant. But remember the problem of statistics is to try and infer the true state of the world from our observations, and so there's the possibility that we make the wrong inference. And so the question we have to ask is, are we correct? And this leads to the following idea. There are different types of errors. Since the null hypothesis might be the true state of the world, there are four possibilities. The null hypothesis might be true or false, and we might have rejected or failed to reject the null hypothesis. Now, first of all, if the null hypothesis is true and we failed to reject it, we've made the correct decision. Our conclusion, let the null hypothesis stand, is actually the true state of the world. Likewise, if the null hypothesis is false and we've rejected the null hypothesis, well, that's what we should actually do, and so we've made the correct decision. On the other hand, we might make the wrong decision. There are two possibilities. The null hypothesis is true, but we rejected it anyway, and this is what's called a type one error. Or the null hypothesis is false, and we failed to reject the null hypothesis, and this is what's known as a type two error. Now, in case you haven't noticed, mathematicians aren't very good at coming up with names for things. And if you're like me, you always have trouble remembering what a type one versus a type two error is, and so sometimes we'll use more descriptive terms for these errors. So if you're in medicine, you might talk about false positives and false negatives. Or if you're in a branch of mathematics known as signal theory, you might talk about false alarms and missed alarms. What you call these things isn't really that important. What's more important is the consequences of these errors. Ideally, we don't want to make any errors at all, but the universe rarely gives us something for nothing. As we make one type of error less likely, we make the other type of error more likely. A good example of this is a smoke detector. A smoke detector can be so sensitive that it sounds false alarms all the time, or its sensitivity can be reduced so that it doesn't detect a fire until the smoke detector itself is on fire. Well, obviously we don't want to reduce the sensitivity this much, but as we increase the sensitivity, we get more and more false alarms, and at some point we have too many false alarms, and we accept the current level of missed alarms. The most important thing to understand here is that point is not computable by mathematics. It must be decided by consensus among the decision makers. For example, suppose we have a device that tests for the presence of dragons. The dragon will eat everyone present, so if the alarm sounds, everyone must immediately leave the area. If the null hypothesis is that there is no dragon, which type of error is preferable? So it helps to set up a table, the true state of the world, either a dragon is not present, or the dragon is present. And we have two possibilities, either we detect a dragon and sound the alarm, or we do not detect a dragon and don't sound the alarm. So if there's no dragon present, but the alarm detects a dragon, that's a false alarm. Meanwhile, if there's no dragon present and the alarm does not go off, that's a correct decision. If there is a dragon present and the alarm goes off, that's a correct decision. And if there's a dragon present, but the alarm does not go off, that's a missed alarm. What we'd like is to make the correct decision. What we have to accept is that from time to time we'll make the wrong decision. And so the question is which type of error do we want to make if we must make one? Well if we have a missed alarm we don't detect a dragon when there actually is one. Since the dragon is going to eat everyone present a missed alarm will cause people to be eaten. On the other hand a false alarm occurs when a dragon is not present but the alarm goes off anyway. And if the alarm sounds everyone must immediately leave the area so a false alarm will cause people to leave the area. And we might reason as follows. Since a false alarm leads to an inconvenience leaving the area and a missed alarm leads to fatalities, people getting eaten, a false alarm is preferable. What if we use the same device but this time the dragon will only make bad puns? And again if the null hypothesis is that there are no dragons which type of error is preferable? So we can do the same analysis. Either a dragon is present or not and either the alarm detects a dragon or it doesn't. We have our correct decisions. We have a false alarm and we have a missed alarm. But now let's consider the consequences. In this case a missed alarm will cause everyone to hear bad puns. Meanwhile a false alarm will cause an inconvenience. And in this case we might decide that a missed alarm is preferable. We can survive some bad puns but having to leave the area and reset everything that's truly inconvenient and we don't want to have to do that. And here's what is vitally important to understand. You must decide on a case-by-case basis which error is worse.