 Welcome to this presentation on how to do survey research. As many of us must be knowing, one out of four research work on media and communication studies employs a survey research method. And many of our allied fields including marketing research and political communication and other social science fields employs a survey research in details. And it's important for us to understand the basics of survey research, what are the various steps involved there, how do we construct a sampling frame, how do we do the sampling, how do we construct the questionnaire, what are the dos and don'ts of constructing a questionnaire and how do we actually perform the survey process and how do we analyze it. So in today's discussion, we are going to talk in details about all these steps. So before we start the discussion, it's important to understand this very important definition of what is a survey. So survey is a systematic method. The method is systematic because these steps are decided well in advance. For gathering information, so we are gathering data from a sample of entities for the purposes of constructing quantitative descriptors, or we want some information from a sample of a population so that we can extrapolate from that sample to the entire population. So we are basically studying the characteristics of a population by asking them questions and based on that, we are going to extrapolate for the entire population. It could be about association, it could be about correlation, it could be just description of the population and it could be many other things as we'll see as we go along. So journalism and marketing research grew to use survey because it was seen as one of the most important ways or one of the most effective ways of finding out the view of the man on the street. So it tells us the view of the man on the street and we'll see that even public opinion polling is one very effective way of finding out the opinion of people. So if you want to find out an aggregate opinion, this survey research is a very important tool for that and we'll also see that it also helps demonstrate correlation links between the variables. So if there are variables and we want to draw correlation links or whether we want to see whether there is difference between two groups and such things, that can be done very easily using the survey method. So just to give you an overview of what the survey method is and we'll go in detail in future slides as well. So the first and the most important thing is to define the research objectives. Before we get to work on the research, it's important to define what our research objectives are and then we do two things at the same time. We first of all choose the mode of collection, whether it will be face to face, whether it will be by cell phone or whether it will be using a computer software or whether we'll be using some kind of email list or those kind of things. And then at the same time, we choose the sampling frame. We'll discuss the details of what is a sampling frame and how we can design sampling frame and how we can get our sample from those sampling frames. And then we design and select the sample at the same time after we've chosen the mode of the collection. We construct and pre-test the questionnaires. So these are the two things that we're going to talk in details in today's presentation about how do we do that sampling process and how we construct the questionnaires and how we go for the pre-test as well. And finally, we recruit and measure the sample and then we code and edit the data and then we perform analysis after cleaning that data. So we'll describe the entire process in today's presentation. In mass communication research, for example, especially in the field of agenda setting, there's been a lot of research based on the survey method where we ask individuals what they perceive to be the most important problem. And at the same time, we measure what is the media agenda and we try and draw correlations between the media agenda and the public agenda. Also, a lot of research in cultivation studies and in spiral of silent studies is also based on individuals' reports of their perceptions of social reality. So these are three very important areas where we do survey research. And of course, there are many, many other areas where survey research is a very, very effective measure. And before we go ahead with the details, it's important to understand two major types of survey work that we do. One is a cross-sectional survey, which we do for one point of time and we just ask the opinion of people or we get their idea about their attitude, their behavior, the knowledge and all those things. And we do it at one point of time or we just do it for one time. And then there is the longitudinal survey, which is done at different times, either with the same respondents or with other respondents. So the important distinction between the cross-sectional and the longitudinal research is that in cross-sectional research, we do it just for one time. And in longitudinal research, we do it over a period of time. So if we do trend studies in the longitudinal research, we study a population at different point of time. So we could be drawing on from different samples from the same population and we do that survey over different periods of time. Or we could be doing cohort studies where we collect data over a period of time from the same subgroup of the population. And that subgroup of the population is known as the cohort. That's why we describe it as cohort studies. And when we do panel studies, we gather information from the same respondents over time. So it could be open in polls with the same people about how they feel about the work of the executive at that point of time and at different times. So these are two very important distinctions of survey research that is known in our field. Let's now discuss about the eight most important stages of survey research. So as we discussed in an earlier figure, the most important thing is to identify the focus of the study and how we are going to do that. So first and foremost is the focus of the study or what are our research questions or what are the answers that we are, what are the answers that we are looking for or what are the questions that are important for our research. And then we establish an information base even before working on that it's important to work on the theoretical basis or on information available on that field, on that subject, on that subarea. And before we get to design questionnaires or before we start to think about doing the survey work, it's important to have an information base about that particular subject. And after we've done that, we determine the sampling frame. Sampling frame is a measure of the population. So if we have a particular population in mind and we have to draw a sample from that population, so we have to get a sampling frame. So for example, if you're wanting to study the people in a particular district and the sampling frame would be kind of a voter's list for the entire district. So that is something which is there in a systematic format. And after that, we determine the sample size and the sample selection procedures. We'll talk about random and non-random sampling and probability and non-probability sampling and all that. And the sample size is also a very important decision that we have to make, because that will determine the effect size and a lot of other things that we'll need to know in the research that we're doing. Next is designing the survey instruments. Generally, it's a questionnaire or an interview schedule and that's the most important part of the survey work. After we have designed that questionnaire or the interview schedule, we have to pre-test that survey instrument because there might be certain things that need to be corrected. So we have to do a pre-test. We cannot just go on working with the first draft question that we've prepared. Finally, we implement the survey and then we analyze the data and prepare the final report. So these are, in effect, eight stages of survey research that we have to do. So designing a survey includes, first of all, the choice of the topic. So as we've discussed, we have to know what the topic is. Then we have to know what are the most important variables. And that's where we have to ask the questions. So variables will help us know what are the questions to ask. And the choice of operationalization. So if, for example, we are trying to measure the attitude of people, how do we operationalize it in terms of questions? So do we ask them whether they like a particular program or how much do they agree with a particular statement or those kind of things? So that's how we start beginning to design a survey. Then, as we discussed just now, we have to test the quality of the questionnaire as well, because the initial questionnaire that we have just prepared or we've prepared will have to be, first, taken to a pilot test. And then that pre-test and where we have to probably look for errors or look for certain things that might need some streamlining. Because at the design stage, a lot of these things may not be obvious. And only when we put it to work in a particular sample of the same population, that's when we get to know about the problems that can arise when we're doing the actual survey work. And then we decide on the final questionnaire. And afterwards, we go for the choice of population and the sample design. So this is just to repeat. So just to repeat what we've just discussed, there are these two things that we have to keep in mind while we do the survey design. The first is we have to be very clear about the construct and the kind of questions that we have to frame and how do we get answers for that. And at the same time, we have to be very clear about what is the target population, what is the sampling frame that we've decided upon, what is the sample, what are the number of people that have to be administered the survey, who are the respondents. And finally, we might need to clean the data and then we analyze it. So these two things take place at the same time. While we are deciding on the survey, it is very important to understand the context of the questions and what are we trying to find out? So one of the first things that we could be interested in is to find out the behavior of the people, what people do in a particular situation so that we have to be very clear whether we are asking a question, which is about behavior. The second type is on the belief, what people think is true. So we must know whether we are asking about a belief question. The other thing that we could be asking is about a knowledge question. This is to test the accuracy of what they believe in. Then we have the attitude questions about what people think is desirable and what is not desirable. And it could also be attribute questions. It could be about the rage, their income, their time spent on media and all those kind of things. So before we start administering the survey, we have to be very clear about the context of the questions and what are we trying to find out as part of our survey research. So there, these concepts of the conceptual definition. So what are the key constructs and what, for example, does attitude or knowledge or these kind of things or for example, participation means, we must be very clear about these before we start administering the survey. Also, the operational definition has to be about the actual measurements of a concept which we just saw two slides ago where, for example, if we are talking about an attitude, so what are the specific questions we could be asking about that attitude? So whether we ask whether they find a particular advertisement was good or whether it was interesting, for example, or whether it was informative, whether it was appropriate, whether it was easy to understand, whether it was objective. So there are various ways in which we can try and measure the attitude of a respondent towards an advertisement. So that's where we have to be very clear about defining it operationally, what is the concept that we are measuring and how we are actually measuring it because there will be questions of reliability and validity as we go along. Also, we must be very clear about the kind of measurement we are doing. And I've discussed that in another video in details, but for today's context, it's important to understand what is a categorical variable. So then we are just naming distinct entities. It's categories that we are talking about. If there are only two categories, then it's a binary variable. If we use numbers to just distinguish one thing from the other, so there the numbers don't have any value. For example, your Adharkar number, it could be just a number, but it just distinguishes one from the other. So it's just a nominal scale in that sense. And when we talk of ranks, we are talking of the ordinal scale. And when we talk of continuous numbers, where the distance between one and two and between one thousand, one and one thousand and two is the same, we are talking of numerical scales or interval scales. And if we are talking of an actual zero, we are talking of ratio scales. So we must know about what we are measuring in the survey instrument. And the information that we are gathering is by asking people questions. So that is at the basis of survey research. We ask questions from people. And we ask those questions by either having interviewers ask them questions. So we recruit interviewers, we tell them these are the questions that you have to ask. Or we ask people, or people could be answering their own questions or recording their own kind of answers. Or there are different ways of administering that survey. But important that we have to talk to people and the way we talk to people can be different and we have to talk to a sample of the population. So that these are the three important areas of the survey research that we are talking about in today's presentation. So this is from the International Encyclopedia of Media Studies by Suman Mishra. And this talks about different kinds of methods of administering the survey. So we could be doing face to face interviews where we are actually sitting in front of people with an interview schedule in our hand or we could be doing it on telephone. But in this context of computers and computer-mediated communication, telephone service also have a computer-mediated element these days. So as we can see, the advantages of face-to-face interviews are very much. It gets a very good response rate. So you're actually sitting in front of people. So it allows interviewers to clarify questions and then we can also incorporate video and audio. For example, we can be shooting videos while they're answering. And it could also involve open-ended and lengthy questions also. So if it's a lengthy questionnaire, you could be sitting there and asking them or we can also deal with open-ended questions. But there are a lot of disadvantages. Number one takes lots and lots of time. So one interviewer maximum can do not more than 15 to 20 or even less number of interviews in a day. It requires lots of training. Interview bias is possible there. Reaching some locations physically is very, very, it can be very difficult. And getting a response to sensitive questions may also be very difficult. A telephone service, of course, are cheap and they have a greater response rate than mail service. So we can be just mailing them by post and they are very cheap to conduct. But one of the problems is that there's a very low response rate. People might not respond to the mail then also. Again, that's much cheaper because it's anonymous. So whoever is answering, you know, and they can answer sensitive questions in a better manner there. One of the best methods these days is the online or the web survey, which we know is very fast, very efficient and very less expensive. For example, you could be doing survey using Google Forms and it hardly costs anything and it allows interactivity also. So on an online survey, you can have interactive element and you can have multiple types of questions. You can have multiple choice questions. You can have drop-down questions. You can have linear skates and so many other things. It can help us reach people in a wide geographical area. It can help avoid the interviewer bias because everybody is being administered the same questionnaire. So the disadvantages are that those people, they have to have access to internet technology and they need to be, you know, literate and sampling is also a problem there. And it also ends up with a lower response rate. We can also do the same thing with multiple modes, for example, for example, the same, for some respondents, we can go for telephone survey and we can have a face-to-face component for those without telephone and we can have a male survey with web response options. So there are lots and lots of options. So for example, for one main part of the interview, we can have face-to-face contact and then we have computer-aided self-administered interviews for sensitive items also. So we can have these kind of mix and match processes as well. So before we start designing questionnaires, important to understand that every question we ask should be related to the service objectives or it must provide me with data that I require for my research questions or to reach the kind of inferences that I want to reach to. And also, if we have written a question that is not related to the objectives, then we must have very strong reasons to know what will we do with the data that we collect from this kind of a questionnaire. So we can have two different types of questions. I'm sure this is quite a known thing. So we can have open-ended questions where we do not give options to people who are answering the questions or we give closed-ended questions where we give them choices from which they have to. They can either take one or they can choose more than one there. And also, if it is an administered interview, we can also have pre-coded. So we have two options in the open-ended questions, but it's important to understand why do we go for open-ended or why do we go for closed-ended in certain cases? So in many cases, if we are providing them with closed-ended options then we are probably priming them for certain kinds of answers. So many researchers would probably start off with open-ended questions and then go to closed-ended, but there are views and arguments for against both these types, starting with closed-ended and then going for open-ended also. So this is the checklist for a questionnaire. Is the language simple? Can the question be shortened? Is the question double-barreled? So is it asking two questions at the same time? Are the questions leading? If you're providing leading questions, then as a researcher, you will end up with problems of validity and reliability as well. Is the question negative? Negative questions might confuse the respondents and we might not get the right answers. Is the respondent likely to have the necessary knowledge? So are we assuming that all the respondents will have the knowledge that is the premise of the question? So when we're framing these questions, we must be very clear about this. Will the words have the same meaning for everyone? Is there a possibility of different people drawing two different meanings from the same word? If so, then we must correct that kind of a questionnaire. Is there a prestige bias? Is there a bias for a particular kind of answer which our respondents might be tempted to answer? Is the question ambiguous? Is it too precise? Is the frame of reference sufficiently clear? Does it artificially create opinions? Is it personal or is it impersonal? Is there anything which is objectionable? And does it have any dangling alternative? So there are lots of things that we have to keep in mind before designing the questionnaire. Also to know the order of the questions, we generally start off with questions with the respondent will enjoy answering and they will easily answer those kind of questions. Then we go for questions which are factual. So which are only about facts where people do not have to decide upon their knowledge and their attitude and those kind of things. So we generally advise not to start with demographic questions such as age, marital status and income and all that and it can come later in the questions. We have to ensure that the questions are relevant to the stated purpose of the survey. There are too many irrelevant questions and it will put off the respondents. And we generally go from easy to the more difficult questions and we go from the concrete to the abstract. So we start off with concrete and then we go to the abstract questions. The open-ended questions should be kept to a minimum and if possible they should be placed at the end. So we have to group them into similar sections. We have to make use of filter questions also. And when we are using positive and negative items, it's important to just mix them up so that people are not answering in a particular pattern when they're answering or they're responding to the survey instrument. Also to know as we've discussed that it's important to even pre-test the item. So whether there is enough variation in the answer, if you are getting the same kind of answer, then probably that's not a good questionnaire whether the meaning is clear to everyone, whether people think that it's difficult, whether the respondents find it interesting and they pay full attention and whether there's naturalness of the section. As we just said, we have to divide the question into sections and whether these sections, they are natural and whether the order is logical and whether people are skipping certain questions, how much time is required and again, you know, responded interest and attention. So these are the things that we measure in the pre-test item and if there's anything that can be reworked, we will rework the questionnaire again and then we will administer it once again. We've discussed about the problems of validity. So validity is about questions, measuring what they're supposed to measure. So we must be, as we discussed before, so whether we're measuring the attitude, the behavior or the knowledge or the attribute and the questions actually measure that kind of a thing. So if you are having an attitude question, we must, or when the survey wants to measure attitude, our question must be actually measuring attitude and not something tangential to that and that's what in a crux, validity is. And reliability, again, is very important because we want the same kind of answer from same kind of people. So if they, you know, at different times provide different answers to the same questions then it is unreliable and, you know, we'll have problems with replication of the survey in a different context. So that reliability has to be taken care of and one of the problems that causes reliability is the social desirability bias because a lot of people have a tendency to portray themselves as someone whose thoughts and attitudes and behaviors are socially acceptable or they want to portray themselves as a good respondent and that is why they're not providing the most honest answer but what they think is the most socially desirable kind of an answer. And that causes, you know, problems of reliability for the researcher. So while designing questionnaires we must be very clear about these biases that are being present. So before we begin the sampling process we just, you know, kind of providing you that it's basically the sample is drawn from a population and then there are respondents who are part of the sample and based on their responses we are kind of extrapolating the results to the entire population. So that is why the process of sampling is very, very important and in the next part of today's presentation we're going to talk about the sampling process. So I'm going to give you a very quick overview of probability and non-probability sampling and what are the different aspects of probability sampling and non-probability sampling and how we can go about it. So let's start with the simple random sampling which is a probability sample. So before we talk about a sample it's important to understand what is a sampling frame. So sampling frame provides us an idea about the individuals or the units from the population from which the sample is selected. So as we just discussed some time ago it could be list of registered waters, it could be employment files, it could be list of driver's license holders, it could be a telephone directory, list of students in a university and from such kind of places where we have the names of people in a numbered kind of a list. So that is the sampling frame and from that sampling frame we will draw the sample for our work. So when we talk of probability sampling we suggest that each individual or unit has an equal chance of being selected and that's one of the most desired kinds of sampling because it provides us with an estimate of what is a sampling error and we can kind of estimate the difference between the sample characteristics and the population characteristics. So when we want to extrapolate from a sample to population then probability sampling is the best way to go about it. So simple random sample is one of the most desired methods of sampling where every potential respondent is given a number. So for example that voters list everybody has a number and then we choose numbers randomly by a process that does not favor certain numbers or pattern of numbers. So if we have say for example 10,000 people and we want to draw 100 people as sample from there so we have this one to 10,000 numbers and then we just randomly draw in 100 numbers from one to 10,000. So it's just like those 10,000 numbers are in a hat and we just draw in 100 numbers randomly. But as you know that's not a very practical process so we'll use computer programs or we'll use a simple computerized techniques to do the random sampling. So for example SPSS provides us with a very important case. So if you go to data and select cases you just have to say exactly how many cases you require. So it will randomly select Microsoft Excel hazard and to all other programs including Python and R and even on simple Google search you can draw in random numbers. If you just type in 10 random numbers between one to 10,000 it will just randomly give you those 10 numbers without any pattern. In systematic random sampling we cannot draw in or it is not possible to draw in all these numbers randomly so we start off with one number and then we go on for every other interval. So just let me give an example. So say for example there are 250 students and out of those 250 students we just want nine a sample of students. So if we divide 250 by nine and round it off it comes off to around 27. So the first number we select is random. So say for example if we select the first number as three then every 27th person is selected. So after three it will be the 30th member. After 30th it will be the 57th member. After 57th it will be the 84th member. So we go on by that process. So that is also random sampling but this is systematic and we are not just randomizing all numbers. So this is again a very convenient way of sampling. In a cluster or in a multi-stage sample we sample the hierarchy of units. So say for example if I want to sample people in a state for example then say a particular state has 25 districts then we randomly choose two districts out of those 25. So this is by the random sampling method. And after that we will choose and say for example any four or five blocks from these districts. And from those districts we select the household. So we do it at a multi-stage or we do it in terms of cluster. So we are not doing it for the entire state but first of all we are dividing the state into districts and districts into blocks and from blocks we choose in a certain number of people. So that is also a very convenient way of doing random sampling or probability sampling. The other is the stratified sampling where we divide the population into strata. Say for example we could be dividing them into religious units. So each say for example if we divide a particular community into say for example 75% Hindus and 20% Muslims and 5% Sikhs and we will make sure that of the entire population 75% are Hindus and 20% are Muslims and 5% are Sikhs. So we are drawing samples from each stratum. So we suggest that it has to be that then this provides us with a kind of a representative population. And one of the major problems with probability sampling and with the survey research is the non-response bias when people start responding can't say don't know or they don't answer that. And that's when the randomness is lost because although the sampling is random and people have been randomly chosen and they are answering randomly but we don't know that which are the questions and who are the people who have not responded to a particular kind of questions. And that is why non-response bias is an important thing to consider in survey research. Non-property sampling is a convenient sampling and that's very easy of doing it especially when somebody is doing your private research and you do not have the logistics and you do not have the money and you do not have the means to go for priority sampling then we go for non-property sampling as we understand that it does not give us power to generalize the survey data with any known degree of accuracy. So that's one major problem of non-property sampling but the easy thing is that it's very convenient and one can do it very easily and effectively. So one of the forms of non-property sampling is the sidewalk survey. So you just talk to whoever is available as you go on walking on the sidewalk or onto any kind of a place or you could be standing in front of a mall or a university or whatever and whoever is available and whoever is willing to talk, you talk to that person. Snowball sampling at the name suggests you ask, you talk to a few respondents and then you ask them to identify others who might qualify as respondents. So one person, he might be identifying two or four more people and they keep on identifying others. So that's again, a very convenient way of doing this non-property sampling. Then we have purpose of sampling where we actually reach out to key respondents and we ask them to respond to the questionnaire or to the interview schedule or we have a volunteer sampling where we set out the questionnaire and we ask people to volunteer to participate in the survey. Sample size is a very important decision that we have to make. It depends on the project type. It depends on the purpose of the research. It depends on the complexity of the research questions. It depends on how much error that we are willing to tolerate and how much time we have, what is the budget and what is the normal optimum size on research in similar areas, what other scholars have done. So these are the things that determine what is the sample size. So anything around between 300 to 500 is regarded as a good sample for private researchers. After we have collected all the information, so in the next three or four slides, I'm going to very quickly talk about what are the results or what are the measures that we could use for finding out the results and extrapolating them to the entire population. So one of them could be the T test where we find out how statistically significant the difference between two groups are. So if you have male responded and female respondents and you want to find out whether the marks they've obtained in examination are statistically significant, we do a T test for that. And when there is more than two groups and we want to find out whether the difference between the groups on any statistical measure are statistically different. For example, we could be having people on various age groups and the time they spend on media on internet, for example, in any given week. So whether the time spent on internet across the groups, across the income groups, the low income group, the middle income group, the high end, very high end across these groups, whether the difference is statistically significant. Difference in our case, as I just said, is about the time spent on internet. So whether there is any statistical difference there. So that is where we go for the analysis of variance test or the ANOVA test. We also go for correlation test to measure association. So when we have both the measures which are quantitative or which are measured on the interval scale, so we go for the Pearson correlation coefficient. That's a very effective way of suggesting association between two variables. Another very important test, it's also used for categorical test. It is the chi-square test where we test the relationship between variables. So that, again, is a very important test that we can do after we get in data from a survey. And before we end, we must talk a little bit about the problems with research misconduct. So there are the three major problems. So if you end up making up data and record a reporting them, then that's a major crime that a researcher can do and also falsification. So changing or omitting results so that the research is not accurately represented. And of course plagiarism is something that we must be very, very clear about. So theft or misappropriation of intellectual property is something that has to be avoided at all counts. And a lot of survey requires that we disclose important information at the end of the survey. So questions about who was the sponsor of the research and what was the survey instrument that we used or we have to put up the questionnaire that we use for the survey. We have to provide a full description of the population that we are studying and also the sampling frame and also a description of the sample design, the sampling error and the weighting procedures and the methods of the data collection and when we actually collected it. So all this has to be put up upfront and when we do this kind of a thing, we are said to be doing ethical research. Thank you for your participation in today's lecture.