 In this brief video, it talks about problems with sample selection. In a prior video, you saw six different methodologies for polling samples. Within any of those, more in some than others, there are possible problems with actually conducting the survey based off the sample. Here they are. Bias, under coverage, non-response, response errors, poorly worded questions. In fact, the four errors starting with under coverage all create bias. And remember, bias is systematically favoring a certain outcome, for example, tending only to get views of a certain segment of the population, tending only to get views of those with strong opinions, tending only to get old, only to get young, only to get men, for example, those all create bias. And what we're trying to do in a good random sampling methodology is avoid that. Even if we attempt to draw a good random sample, there can be problems that create bias. Let's talk about those. Under coverage. Under coverage means we've defined a population that we'd like to get to and get opinions from, but for whatever reason we can't reach them. So I'm doing a phone survey, but a certain element of the population doesn't have phones. Or my methodology only includes landlines, and I miss people who have cell phones. I'd like to do a survey by a computer with pop-ups, but not everybody has a computer at home. I'd like to do a survey that really hits the population in a city, but a certain element to that population for whatever reason might be difficult to get to. So under coverage means for whatever reason we're missing a part of the population that we'd intended to get to. I don't know if you recognize this guy or not. This is Jesse Ventura in this state probably elected him governor not so long ago, over 10 years ago. I actually remember coming home from a trip and getting into my car the night of the election and turning the radio on and hearing that he'd been elected and just being blown away just shocked that he'd actually been elected governor. The polling leading up to the election had him third in nearly every poll. Now it was close and there were three people running, but he was third in nearly every poll. And the pundits who sort of looked at this after the fact concluded, not 100% sure they're right, but concluded that one of the reasons he was third and ended up getting elected is that a large portion of the young population, new voters, late teens, early 20s, loved the idea of a non-traditional governor and voted for him. But in the polling that led up to the election, they were missed. They were difficult to get to, often because they were living at home. And so parents answered calls instead of the 18 to 27-year-olds. And so they were under-covered in the survey leading up to that and that's why the polls missed the idea that he would end up being a governor. Non-response simply means what it says. The phone rings, excuse me, may I have a moment of your time and you hang up. The pop-up comes up on the computer and you click through it. The guy at the mall asks you if you can ask a few questions and you speed up and walk by. In doing that, you've eliminated yourself from the survey and if that happens with a pattern, busy people hang up, old people stay on to have a conversation and then you're creating bias. You're missing busy people and getting responses that tend to favor old people. I'm taking a survey, so tell me, son, have you used drugs or alcohol in the last three months? Well, geez, no, I'm going to say no to that. Well you can see what might happen here depending on how a survey is conducted. Folks might be wary about giving an honest answer to a question. You get a phone survey on a controversial topic or you get a survey at home perhaps about taxes. Excuse me, Sir Madam, I'd like to ask you a few questions about your approach to taxes. Do you pay your taxes every year? Well, if you answer it all, you're nervous and perhaps careful in what you say. So response errors can come from lying, but that would be oversimplifying. It could also come from misinterpretation of the question, where you don't really understand the question that was asked. And so you give a response that doesn't get to the point of the question. A warning of questions is interesting. Read these two. Which is the better question? Well, if you found a wallet with $100 in it, would you turn it in? If you found a wallet with $100, what would you do? Well, I think the second question is a little better because it's open-ended. The first we call a leading question. It suggests what the right thing to do is and probably buys the answers in favor of turn it in. Here's another. Read these. Which is the better question? Well, again, I think the first one is a bit leading. It tells you what proper is and wouldn't we all like to be proper. So I'm going to pick the proper thing, as opposed to the second question, which leads it a little more open-ended and allows somebody to form their own opinion. So again, how you ask a question could lead to a certain answer. These two questions were asked in the same survey really to get at this point to see if people's answers would change based on the way the question was asked. So these questions were randomly mixed up in a real survey. They basically asked the same thing, but they got two different responses. Pause me and read these briefly. So a government had some extra money. And the question was, should the money be used for a tax cut or should it be used to fund government programs? Second question says, should the money be used for a tax cut or should it be spent on programs for education, the environment, health care, crime fighting, and military defense? Which question is the Republican question? Which question is the Democratic question? And it's a little bit of an oversimplification, obviously. But the first question sounds like a Republican question because it's a simple choice, tax cut or bigger government. The second is a little bit more Democratic in that it's tax cut versus programs that might be favorable to the population at all. Well, interestingly enough, that same question, 60% were for the tax cut in the first. But once options were introduced, only 22% were for the tax cut in the second. Same survey, same population, same folks answering questions. You get two wildly different responses based on how the question's asked. So those are problems you should know and be able to identify. Bias is introduced by the four problems listed below. Under coverage, I miss a certain element of the population. A non-response, I might only get responses from those who have strong opinions. Response errors, I might not get what I'm looking for. Poorly worded questions would tend to lead people towards a certain answer that doesn't really reflect what they might think if allowed to think on their own.