 In this part of the course, we will work on understanding how to make a good experiment. Now that we've seen that just observing is not quite enough, we are in the business of dirtying our hands and getting involved in the real world. We will look how experiments work and why they help us identify causal relationships. Let us quickly look back at why we thought observation was not quite enough. If we want to see a specific causal relationship, observation alone is not quite enough because there are many, many alternative explanations and unwanted influences that we cannot disentangle from what we are truly interested in. In a way, it's like trying to count the hair on the torso of a certain bee while she's crawling around in a bee hive with lots of other bees. There's so much going on, it's pretty much impossible to get it right. You have no control over the situation, which makes it hard to answer your question. Instead, we want to create a controlled environment. That means isolate the bee, zoom in on the thing that we're interested in, the hair on her torso. The same idea applies to experiments. We try to gain experimental control over the situation by eliminating all distractions so we can study exactly the thing that we're interested in. Let's say we want to test if a new medication we have invented in our home laboratory helps against myopia. We have developed a carrot-based pill that we think cures people who currently have to wear glasses because they're short-sighted. Our speculation is that if you take this pill for 14 consecutive days, your vision will be restored to 100%. How would we test if our new medication works? Our first step would be to find a number of people who will take this medication for 14 days. That's how long we think it takes for the medication to work. We would make them promise to take the pill every day. Since we want to study myopia, it makes sense to give the pill only to people who are myopic. After all, we want to cure this condition, and if someone doesn't have it, we couldn't cure it. So far, that pretty much sounds like the woolly head observational study we had planned in the last video. So where does the experiment come in? The important part is that we want to make sure that our pill is the cause for restored eyesight. So we need to make sure that we can exclude alternative explanations. We want to make sure that a 14-day period in which someone is participating in a medical study alone does not lead to improved eyesight. We want to make sure that spontaneous improvements of eyesight are accounted for. And that's why we need a control group. And control group is a second group of participants or other subjects in our experiment which we observe under conditions that do not differ from those in the experimental group one bit. In our study, we would measure the eyesight of the experimental and control group at the same moments in the study. We would ask the people the same questions and we would treat them the exact same way in every other regard too. In our experiment, we would also want to make sure that the act of giving our participants a pill is not the reason why their eyesight has improved. For this reason, we would give participants in the control group the exact same number of pills as in the experimental group and give them the same information about it. The only difference would be that the pill in the control group would be a placebo. That means it would be a pill that contains no active ingredients whatsoever. It doesn't actually do anything at all. This way, we can be sure that our pill is effective above and beyond the effect of ingesting a pseudopill and thinking it might help you see better. Hooray! This is a basic experiment. The basic feature of an experiment is that it deliberately contrasts different observations. In our case, there's one control group and one experimental group. There could also be circumstances in which there are several experimental groups, such as when you want to try out which dosage of our carrot pill is the most efficient. But for now, working with this simple design suffices for our purposes. Now it's your turn. Practice designing an experiment. Revisit your item from the previous video. How can you make an experiment to test your speculation? In my case, I wanted to know if loose versus snugly fitting woolly hats keep people warmer. So how would I make my basic experiment? I would want two groups of people. A loose hat group and a snug hat group. Each member of the loose hat group is asked to wear a loose hat. And they all are the exact same model. And they're made of the same fabric in the same knitting style as the loose hat. Identical, except for the fit. Then I ask everyone to stand in a wind tunnel blowing at a certain speed with a room temperature set to, let's say, 5 degrees Celsius. I'd ask them to report how warm they felt in this room before putting on the hat. And then they put on their own clothes. I'd ask them to report how warm they felt in this room before putting on the hat. And then they put on the hat and stand in the room for 15 minutes. After 15 minutes, I ask again, how warm do you feel? The difference in the warmth between the loose hat group and the snug hat group is what I need to know to address my hypothesis. So far, so good. But you may remember that we were wondering how hair would influence finding out which hat works the best. This brings us to another consideration. We want to make sure that the conditions in our experiment are identical between groups. That also means that we should make sure that disturbances to the causal relationship we want to study are identical between groups. In the case of the woolly hat, that might mean we want to keep constant how much hair our participants have. Maybe it would be best to only study bald men. By only studying a narrow type of participants, this also means we should ask ourselves if our conclusions can be generalized beyond that group to the general public. We will come back to this point in the next module of this course. There are a number of other factors we would try to keep constant, such as the gender and age of the participants in both groups. Identical twins would basically be our ideal test participants. But we can't run all of our experiments just with identical twins, right? So are there other things we can do if we can't control everything? There are two tools that can help us with this problem. We can try to assure that disturbances, if there are any, occur in all groups to a similar extent. We can do that by studying a large number of people or other subjects. In our hat example, if we study lots of people, we can be pretty sure that all kinds of hairstyles will come up as we go along. We could also record which hairstyles our participants have and use statistical tools to control for the effect of different hairstyles, so this doesn't cloud our effect of interest. Moreover, we can randomly assign our subjects to the groups investigated. That means chance decides if someone is in the one or the other group. In our example with the carot pill or the placebo pill, which group of pill takers you would be assigned to could be determined by a coin toss. If you get heads, you'll take the carot pill. If you get tails, the placebo pill. This brings us to another important issue. Neither the participants in our carot pill study nor the administrators who interact with the participants should know who takes the carot pill and who takes the placebo pill. Why? Because if participants knew they were taking the placebo, its effect would be meaningless because participants would no longer believe that the pill could help and might therefore respond differently to our experiment. If the research staff knew who was taking which pill, they might treat the two groups differently, even without wanting to do so, which may cause them to have different reactions. Therefore, in our experiment, both the research staff administering the pills and interacting with the participants, and the participants themselves should be blind to which group in the experiment, or we also call these groups conditions, so to which condition they have been assigned. We called this type of procedure double-blind. A quick side note on working with human subjects. Participants would of course have to be informed beforehand about this procedure to make sure that they are okay with the potential of only receiving the placebo pill instead of the carrot pill. Finally, we would also want to make sure that our participants take the carrot pills or the placebo pills. This type of control is called manipulation check. We make sure that our intervention, our manipulation was successful. In this example, this means making sure that everyone took their pills. There are several ways of doing this. You might ask participants to report if they missed a pill. Oh, they might be reluctant to admit that, fearing negative consequences, such as being judged by the researchers administering the study. You might require participants to come to the lab to take the pill while you watch them. And you could then check their mouth to make sure that they aren't hiding the pill and make them stay in the lab for two hours until the pill actually has been digested. Or you could measure the blood levels of pill-specific indicators that would suggest that the pill indeed has been taken. There are a number of ways to do this, but the idea is that you want to make sure that your manipulation was successful and that not finding the effect you're after is not because the manipulation didn't work. All in all, now we have a list of things we want to keep in mind when designing our experiment. Let's put it to work. Revisit your item from the previous video. How can you improve the experiment you just designed to test your speculation? In my case, I would study only bald men, as we've discussed already, sample a lot of them and randomly assign them to the snug versus loose hat conditions. Keeping the researcher administrating the study blind will not work unfortunately because they can see the hats that they distribute. We could use different researchers for the different conditions and not inform them about the other condition. Now that might be a solution. And we could not tell participants about the other hat condition until after the experiment for the same reason. Finally, in terms of the manipulation check, we would need to make sure that the participants put the hat on properly. The researchers administrating the study could be in charge of doing that to make sure that the hat is positioned in the right way. Finally, let's quickly go back to the identical twins case. We said it would be ideal to invite identical twins as human participants because they're very similar in many regards. Another way to go about this is to study the same participant in both conditions. Arguably, you are the most similar to yourself that anyone can be. And we can sometimes take advantage of this. In the example of the woolly hat, we would make sure that the same subject first wears one hat, then let them acclimatize back to their original body temperature and then wear the second hat. To make sure that the order of which hat was worn first does not affect our results, we would randomly choose for each participant which hat they wear first. There's a procedure is appealing because you don't need to find as many participants. And at the same time, it can be problematic because it might be harder to keep participants blind to your hypothesis. So let's sum up. In this part of the course, we have seen how to make an experiment. We have seen what experimental and control groups are and why it's important to keep disturbances constant between them. We have also looked at ways how that goal can be accomplished. In the next part of the course, we will see if we can spot bad experiments and improve them. We will also discuss more generally how to deal with shaky insight we might get from experiments.