 So today we are going to discuss about experimental methods in media and communication and that is one very, very important method that very often we do not give the importance that it deserves and it's probably because of the logistics that are involved there and probably the rigor that's required here which becomes a difficult to maintain at times. So the moment I talk of experiments, this is the figure that comes into our mind and this is what we see in television, this is what we see in movies, this is what is generally regarded as experiment or this is the stereotype image of an experiment. Much of it could be true but in social sciences experiment also have many other perspectives. So we'll talk about what are those perspectives and how we can do that and what are the things that we have to be careful about and I will probably give you one or two examples about good research which has adopted this experimental technique. So it's not only about these things, experimental research in social science has a lot more other aspects to it so we'll talk about that. As I said, experimental designs are regarded as the most rigorous because that's the standard against which all other designs can be judged and one reason is that because if I have to establish cause-effect relations that this causes that or the example that I gave you earlier that if time spent on online classes causes greater understanding. So if I have to establish this cause-effect relation then I have to go for an experimental design then a mere correlation test would not be enough. That is why this is regarded as such an important test. So if I have to give you a definition of what is an experiment, so this is how we describe especially in social sciences, it's a methodological design which shows how one or more variables that have been manipulated by a researcher, how it influences another variable. So in other words, how an independent variable causes some change in the dependent variable. So I am manipulating the independent variable and I am trying to find out what is the effect on the dependent variables and the most important effect or the most important purpose of an experiment is to identify causal relationships between the variables as I said that this is caused by that or this causes that. So this can be established only through experimental methods or through the experimental design. So this is a classic definition of an experiment. It means recording of observations. It could be quantitative or qualitative made by defined and recorded operations. So we have to first of all define what are those operations or what are those manipulations and we have to record those and in defined conditions and these are very, very important factors I will discuss about these conditions in today's presentation followed by an examination of the data by appropriate statistical and mathematical rules for existence or we are trying to find out whether there are significant relationships or not. So this is just, as I said, just a little definition of that that we are trying to find out record data when that dependent variable has been influenced or impacted by some independent variable and we examine that data and we try and find out through statistical means whether there is some relationship between these two variables. So basically that is what we do in an experiment. So I'm going to talk about what is a classical experiment design. So in a classical experimental design, we first of all need to know or we have to have an understanding about what is an independent variable or a condition or treatment introduced into the experiment. So it could be in today's time it could be the vaccine kind of a thing. So that is the independent variable. So you introduce that. So that's of course not for social science research. But in social science research, it could be that I make some some person consume some kind of a content or I expose that person to some kind of a content. And then I will try and see what is the impact of that condition on the dependent variable. So in the classical experimental design, first of all, we must know what is an independent variable or we must have a very clear defined understanding of what is the independent variable. Now dependent variable is something that provides us with the data. It could be physical conditions, it could be social behaviors, it could be attitudes, it could be feelings or belief of the subject. So whether after consuming, say for example, violent content, did your attitude change or after consuming a lot of content, which was which were about patriotic movies, say for example, did your attitude or feelings or beliefs, etc, did they change? So as you can understand, the first part is about introducing you to those conditions. And the second part is to measure these things. And in an earlier presentation, we have spoken about how to measure that it could be through Likert scales, it could be through standardized instruments or whatever, but we are measuring this data through certain subjects. It could also be self reported that we ask you that, okay, what do you think about this or what, what, what, what, or how would you behave about this one and so forth. So the dependent variables are measured in exactly the same way or in very much the same way as it would be in a survey research. So we introduce that condition means I introduce that independent variable. It is regarded as one of the condition or a treatment. And if I have to compare it with clinical test, it could be some medicine or it could be some vaccine or whatever. And then we find out and we study the response to that treatment. So we are using the same term, but it means very different in our media and communication perspective. It could be introduction to say, for example, public speaking classes. We'll talk about that as well. So dependent variables are often tested through the survey instrument or there can be some automatic way of testing that as well in, in certain more sophisticated experimental research designs. So for example, there is this secondary task reaction time. So how much time do people take to respond to a question or how much time they take to respond to the click of a button or whatever. So we can have very many different ways of measuring the dependent variable. The important thing to understand is that this is what, this is the, at the center of what we're trying to study, the dependent variables. Then there are two important things that we do not measure the dependent variables straight away. There are two cases. First, I measure the dependent variable prior to the introduction of the independent variable. So the independent variable could be some television content or could be some OTT content, for example. So I measure certain attitudes, certain behaviors, certain knowledge. Before I show this content to the subject and before showing that I do the pretest and it would, it would ask the same kind of questions. So probably it could be say, for example, if I were having a training program, so I would give you a pretest. So what is your idea about experiment or what do you know about experiments on and so forth. And after this presentation, I give you the same test and try to see whether there has been some kind of a difference after you were exposed to this test. But that's a very crude example because that would be just one short test. I'll talk about all these in details in a moment's time. After the pretest, the post test also has to be done. So that would give us an idea about how much the dependent variable has changed because of the treatment or because of the condition that was provided. So in our case, I will repeat, I was talking about the condition could be exposure to OTT content or it could be exposure to some violent content or patriotic content or any other kind of exposure or treatment, as I said. So we have to have the post test as well. Just pretest is probably not enough. We also have to have two different groups. First of all is the experimental group that receives the independent variable or that receives the treatment or that receives the condition. And then we have the control group because control group is the one which does not receive the independent variable. It is the one which does not receive the vaccine, say for example. So we are having one which is known as the control group and the other which is the experimental group. In the experimental group, if I have to give you that kind of clinical example, then it would be the experimental group has been given that medicine. The control group has not been given medicine. They probably could have been given a placebo. So because if you know that you have not been given medicine, then your response would be different. You do not know that whether you have been given the medicine or not. So very important to understand that in a true experimental design, you have to have both the experimental group and the control group. So we'll explain why control group is required. So finally, we have to make the, as I said in the beginning, we have to manipulate the independent variable before a change happens in the dependent variable. So the changes in the dependent and the dependent variable, they must be correlated. So we have had a presentation on correlation means either one is causing an increase in the other or decrease in the other, but they must happen together. Now, why are we having an experimental group and a control group? Because we have to be extremely sure about one thing. And it's that that any change in the dependent variable, any change in your attitude or behavior or habit or whatever, should be explainable only by the independent variable and not by any other intervening variable. That is why we have two very similar groups so that we can find out whether the change that is observed is due to the treatment or is due to some other factors. So since we have two very similar groups, we give them to one group, we give the treatment to the other group, we do not give the treatment. So that tells us whether the treatment and as I said, the treatment can be of many types. So that treatment, whether that has some effect or not. So this is a very important part of experimental design and that is why we need to have the control group, the experimental group, the pre-test and the post-test. So we must have a knowledge of all these things and when we have all these things, then it is a classical experimental design. Very often we'll see that it may not happen or it's not possible to have these classical designs. So we'll have another explanation or another way of going around that. So before we start an experimental design, these are the questions that we must ask ourselves or these are the questions that need to be answered. So what are the variables under investigation? So which are the variables that we are trying to find out about? So what are the dependent variables and what are the independent variables? What data are being collected and analyzed? So whether it could be your attitude. So if I'm collecting data about your attitude, I must be very clear about how it is being collected and how it is being analyzed. Where is the source of your data? So whether if I'm measuring the attitude, am I measuring it through a questionnaire that I'm giving you or am I measuring it through some other manner? How is the experiment being conducted? And we'll tell you and we'll show you that our idea of the laboratory or our idea of the experiment has to be a lot more creative because we are dealing with very, very, very different things, very different from the stereotype image that I showed you right at the beginning. And finally, how will the experimental data be analyzed? So there are very many statistical techniques to analyze this data. We'll talk about the analysis also as we go along. So first of all, we have to state the research problem and the hypothesis or the research questions because that's very important. If our hypothesis or research questions are not clear, then we will not be able to convince ourselves and to convince everybody that the experimental methods are the most appropriate ones. So we have to be very clear about this decision that the experimental methods are appropriate for this kind of a research. And then we have to, as I said, in a classical experimental model, we have to define what are the independent variables, what are the dependent variables, what are the potential intervening variables, and we have to take care that it does not impact our experiment. And then we have to choose the appropriate measures to find out the data. So as I said, first part is manipulating the independent variable. The second part is to get data. So we must have appropriate measures to use in the experiment. So as I said, you need to have an experimental control and why control group is required. I just gave you the reason, the reason being that if I have two similar groups, then I can be sure that whatever changes are being observed in the post-test from the pre-test is because of the independent variable or not because of any other reasons. That is why we need to have the comparison group. That is why we need to have people being randomly assigned because if people are not randomly assigned to these groups and if you choose people, then of course there will be errors and we'll talk about these errors as well. And you need to have a pre-test and post-test as well because pre-test is before you administer the treatment or the condition and post-test is after that. We'll talk about some of the designs. Some of the designs are quasi-experimental. Some of them are not truly experimental, but they are under the umbrella of experimental design. So experimental control is about these three different things. This is just to repeat. The first one is about the comparison and control group. So the experimental group is often known as the comparison group. So we need to be very clear about these two and they must be randomized. That assignment to these two groups must be random. And often in the control group, the person is given a placebo. So say for example, in our experimental group, we give them some content which has, say for example, violent content inside. And to the control group, I give a content. I make them go through some content which does not have a violent elements there. So that is a kind of a placebo because both of them think that they have received a treatment because if to one group you do not show them anything and to the other group you show something, then obviously there will be a lot of design errors and people who are not part of an experiment, they might give answers very differently. So random assignment is very important because if you do not assign people randomly to the comparison and control groups, there can be a selection bias. There can be a bias of one group being very different from the other group and that will not be able to justify, will not be able to justify as researchers that whatever changes are being observed is only because of the independent variable because as you will remember that we are doing this experiment or the experimental design is practically to prove this cause effect relationship that this causes the other. This is not only about correlation, it's about causation. So that is why we are doing that and that's why all this rigor is so very important. And the pre-test and post-test is also very important because if you do only the post-test, then that is not enough. So the same person has to be given a question before the treatment is being provided and once after the treatment has been provided. So we have to be very clear about these three dimensions of experimental control. These are the three very important pillars of experimental design. If we are not able to do that, then our experiment will not be a true experiment. So I will talk about one, as I said, I'll be talking about one very important example and this is from a paper in this communication research reports. This is by Yun Casico and Billingsley. What do the, this is about the effect of taking a public speaking class on one's writing abilities. So if you take a public speaking class, does it impact your writing ability or not? So this is a kind of an experimental research and how did they go about it? So I will just use this example to demonstrate some of the things that I just spoke about. So what these people did where they spoke of these five hypothesis. I'll just zoom in the hypothesis. So the first one is that individuals exposed to a public speaking class will have greater gains in their writing skills of writing context than those not exposed to a public speaking class. So the first hypothesis is that individuals who have done a public speaking class, they will have greater gain in their writing skills. So their writing skills will improve. So that is one of the hypothesis. The second is individuals exposed to public speaking class will have greater gains in their writing skills of content development. So it's about the first is about writing content. The second is about content development. The third is about writing structure. The fourth is about the use of source and evidence. And the fifth is about the control of syntax. So these are very specific hypotheses that somebody who has been exposed to this kind of a class will have or will perform better than somebody who has not been exposed to that public speaking class. Now, as you can understand that this experiment requires a lot of creativity because you cannot force someone to do a class or you cannot ask someone not to do a class. So what these researchers did, I mean, these Yoon Costantini and Billingsley, what they did was they found out students, some of whom who were doing this public speaking class and others who were doing some other class. So as you can understand, this was not random, but still there were many other things that they took care of. So just to demonstrate how experiments are done. So they had students write a three to five page paper at the start of the semester and the end of the semester. So this is the pre-test and the post test. So they were not asked questions or their temperatures were not taken or their blood pressure was not checked, et cetera, et cetera. Of course, this is about social science research. So it has to be about the dependent variables that you were talking about and dependent variables are the writing skills. So they were given this three to five page paper at the start and at the end of the semester. And there were people and there were experts who would grade these writing based on certain very standardized measures. So what they did was that they would measure the writing skill at the beginning at the end. Of course, the coders weren't said that this is beginning and this is end or whatever. It was just very random. So they were asked to measure that. And then they wanted to see whether the public speaking class group performed any better than the other class, they did a history class. So this is not a true control like group as you would understand because in the true control group, it must be very, very similar. So I'll talk about these examples when I talk about a non equivalent group but the idea is very simple. The independent variable was the public speaking class. So that took care of it itself because people who did that class were in one group, the students wouldn't do that class were in another group. And we gave them this test at the beginning at the end and then we evaluated that and then we started to or we wanted to find out whether there was anything available. And this is how psychology research or behavioral sciences research work. And they had more than 600 students and then this is regarded as a very good research in this particular area. So the reason why I chose this is to suggest that there are so many experimental measures available. The key is to be very creative about that. There was no laboratory involved there. People were not able required to sit through or whatever. It's just that they were identified and they were given these tests before and after the semester class. Of course, there are threats to validity and all which will be answered as I go along. So important to understand some of the notations. I'm now going to talk about some of the experimental designs. What is the true experimental design? What is the quasi-experimental design? And so on and so forth. And before we start, we must understand these three or four terms very carefully. So whenever we see an O, it will be about an observation of a dependent variable. And whenever we see an X, it will be a treatment of the independent variable. R is about a random assignment and O's are numbered with subscripts from left to the right and this is on the time order. So if it is a pre-test, it will be O1. If it is a post-test, it will be O2. And if you do a test after or if you do an observation after that, it will be O3. So O is the observations that you're making. X is the treatment you're providing and R is the random assignment. So whenever we see such pictures, we must understand that what this means. In fact, I'll be showing you some of these pictures to describe the experimental designs that I'm talking about. So very important to remember these things because later on, whenever you talk of experiments, you will be talking in terms of these notations. You will probably not even be writing about that. So just like music notations, this is about notation of experimental design. So there are situations where classic design is not possible. I just explained what classic design means. Classic design means where randomization you are able to do the post-test and the pre-test and all those things. So there are cases where it is not possible. Even in the case where I spoke of this wonderful paper which was published in this journal, even there we could see that this is not a classic design because we were not able to randomize students into these two groups. So there are cases when this is not possible. And that's where we have to do these kind of tests. So one of the pre-experimental design tests is the one short case study. So one short case study is a design where some manipulation of the independent variable occurs. So what we do is that we want to find out the effectiveness of political campaign on voting behavior. So some person has taken part in this political campaign or likes that political campaign or has been exposed to political campaign. What, how does he or she vote? Or it could be any kind of a thing. So it's just a one short case study. The person is exposed to some independent variable and we just do one post-test because we don't do any pre-test because that is not possible. And also there is no control group. There's only one group and we provide it with only one treatment or we don't even provide but we can see that there's a treatment which exists and we just do one post-test. And as you can understand there are lots of problems because if we do this kind of a study we cannot tell with surety that there is a cause-effect relation that people voted only because of this political campaign. There could be other variables also. So when we have to provide this cause-effect relation where we are suggesting that the change in the dependent variable is happening only because of the independent variable then we have to be very clear with the design but in case that's not possible these are the possibilities that exist. The other possibility is the one group pre-test post-test. So this is just an extension of the one I just showed you. This is where I'm just giving them the post-test where these people have been exposed to say for example a political campaign or they have been exposed to some social message campaign or any kind of a thing. In an extension of that the same group we are able to give them the pre-test and the post-test. Pre-test means we give them the pre-test before they went to the political rally and we give them a post-test after they return from the political rally. But again there are a lot of factors about time and all I'll talk about that in a moment's time but this is another type of experimental research where we are giving them, we are giving the same group the pre-test and the post-test. So if I have to show you in terms of notation this is how it looks. So if you remember in the one short case study there is no pre-test, there is only an observation. So there is a condition as you can understand when I spoke of x, x means a condition x means a treatment and there is an observation. So there is no two groups there is no other thing, there is no randomization just the treatment and the observation. And in the one short pre-test post-test there is an observation before the treatment and there is an observation after the treatment. So I'm sure this notation now makes some sense that when we talk of x we are talking of the treatment and o means observation. So these are the two types of pre-experimental designs as I said these are examples of pre-experiment designs. One is the one short case study the other is the one short pre-test post-test. Now I'll talk about the quasi-experimental design. Quasi-experimental design is a time series kind of a design that there I'm measuring the dependent variable at various points of time before and after the manipulation of the independent variable. So say for example I made you sit in front of some OTT content and before making you sit in front of that OTT content I measured the different variable before and after that I do at various levels of time I might be doing it after one day after one week after 10 days or whatever. This is done so as we can be clear about whether there is a degree of change over time. So as you can understand this is a quasi-experimental this is not a true experimental design because again problems of randomness and control they are not satisfied here or they are not met here. So this is a quasi-experimental design that's why. And in the design and in the experiment in the experiment of public speaking that I just showed you this one and this on public speaking class there as I told you this was a non-equivalent control group because the other group I had no control over them. One of my group which is a comparison group they were the students who took the public speaking class but over the other group I had no control they were the ones who would decide what class they would take so I took a similar one where they took a history class. So basically I was comparing public speaking classes with their history classes. So both of them are giving pretests and posters all those things are there but the control group over the control group we do not have any control. So if I have to express that this is how it will look in the time series if you can say O1, O2, O3 these are the observations I made three times before the treatment was provided X as you can understand is the treatment or the condition or the manipulation of the independent variable. And after that over three different time periods I take the observation again O4, O5 and O6. So I'm doing it over a period of time over three times before the independent variable was manipulated and three times after that. In the non-equivalent as you can see that in one of the groups this X condition is theirs X condition in that case was about exposure to public speaking classes. So we do one pretest one posters but the other one we have no control over. So they are not similar groups we just take the observation before and after and we'll try and see whether there are any differences in these two groups. So basically I'm trying to see whether the dependent variable changes in these two groups. The post test only control group design this includes a random test. So we can have only the post test also. So there are cases when it is not possible for us to find out pretest. Pretest would mean that you identify those people and then you do a test then you manipulate the independent variable and then you do the test again which is the post test. So there are cases where we will only be able to do only post test. Although all other things will be satisfied there will be comparison groups. Comparison group means experimental group there will be control groups. There will be random assignments but generally we are not able to perform all the other tests there. So that is what we are saying is different for these quasi-experimental tests. It's about how far the logistics apply or how far the logistics allow you to do it in the classical way. So as I said in a quasi-experimental way design we are not able to perform these things. So this is as good as as close you can get to the classic experimental design. Then there is one of the most, as I said rigorous designs which is known as the Solomon four group design. So this is a true experimental design and it goes even beyond the classic design. So it has two extra control groups. I mean they have one extra control group and one comparison group, random assignment and they have two pretest and four post test. So this is to take care of all that I'm going to talk about a lot of problems. So if I have a Solomon four group test then a lot of the problems of validity that I'll just talk about in a moment's time they will be taken care of. So I'll talk about 12 problems of validity in a moment's time. So if I have to explain that in terms of notation this is how it looks. So the pretest post test means there is an observation and it is randomized. I have a treatment and I do an observation before and after and since there is a control group so I have an observation before and after where there is no treatment. So if I have to just again compare it with the clinical example I gave you so the first group would have been given a vaccine and I see whether there are any changes. The second group is not given the vaccine and whether we see whether there are any changes or not. So this is the pretest. In the post test only control group we only do the post test. As you can see there is no observation before giving the vaccine. There is no thing before giving them or before manipulating the independent variable for them. And in the Solomon four group design as you can see that we have this one group where we are just having the post test. In the other group we are having there is no treatment. So this is just a control group. We are having the pretest and the post test. This is the classical kind of a thing that we do where we have a pretest, we have a post test and it is random as well. And in another group we just have the post test. So we have four post tests. We are having two control groups and we are having two pretests. So that is the Solomon four group design. Why do you have to do all that? I will just explain in a moment's time why such a rigorous method is required. This is to take care of all the possibilities of validity that might threaten the, all the problems of validity that might threaten my conclusion. So that's why we have this Solomon four group design. So basically we know about validity. Validity means the ability to eliminate alternative explanations of the dependent variable. So if I am suggesting that this is happening only because of the treatment that I have been given, then I have to remove all the other possibilities. So variables other than the treatment are threats to internal validity. So as I said that if my experiment is about public speaking leading to better writing then all the other explanations must be taken care of because if other explanations exist then I cannot show that cause effect relation. So now in the next few slides I'm going to talk about what are the validity threats in a very short time. So there are six time progression effects. First of all is, so I'm going to talk about these six threats to validity as far as time is concerned and what is validity test that what I'm trying to measure is not the true measure that I'm getting. So history means that there are even that happened during the experiment that is outside the study. So it might happen that there could be some things happening whenever I'm doing a test or a pretest or a post as obviously I'm doing it over a period of time. So there might be other things that might come up probably India is winning or losing or some other thing that I have no control over but it may affect the outcome of the study. So people might be happy only because maybe the Indian cricket team is winning that is not a part of my experiment and I have no control over that but that happens in the outside world. So these things might take place and that is why as I said we have that four group design. Instrumentation again that is a problem which I spoke of when I spoke about or when I did a presentation on a questionnaire design that say for example I'm doing a pretest and then from the example of the answers I think that okay if I add another question or I remove another question then it will be better. So I am changing the instrument I am changing the survey instrument from the pretest to the post test and if I do that then of course my validity will be under question. Maturation is as I said a naturally occurring process in experiments. So all my participants over a period of time they will be developing mentally, physically, emotionally, otherwise. So whatever they answered maybe a month back they will be learning a lot better now. So of course as you can understand that before online classes nobody would even tell you what was Zoom. Zoom would mean somebody who would move very fast zooming would mean fast or we didn't even have Google Meet then and but right now a lot more people are experts with online technology. So that is a special occurrence but it might occur very naturally also. So I must make my design take care of all these things. Mortality unlike its literal meaning is not about people not living. It's about the fact that many people who will start the experiment they will not finish. And unlike our good Hindi series we cannot change them just through some plastic surgery or whatever because if my participants change during the process or if they're not available then of course I will not be able to get results out of that because the effect on the same person is being seen. So this is again a problem of validity. Statistical regression would happen when my sample includes people who are outliers people who are extreme of the dependent variable. So that problem is there. And again there is a problem of the testing itself the testing effect itself because the same person has been given a pretest and consciously or subconsciously he or she might be sensitized to what is the right answer. So when I give him or her the post test again then they would know what is the right answer and they would try and take care of the answer. So if you can understand so all these things that we were doing here that in certain cases we were having control group in certain cases we were having just the post test in certain cases we were having you know only the control groups so all these things are to ensure that these things are taken care of. There are six more problems there as I said one is the compensation behavior because as I said there are two groups one is the experimental group and the other is the control group. And oftentimes we say that there is a comparison group. So if one group sees that the experimental group is getting special treatment then they might respond differently or they might even get angry. Again there's a problem with knowledge. So if you tell people that you're being tested for something then they may provide answers which they think that the researchers want. So this is again a thin line to walk because if you don't tell them then there's a problem with certain kinds of regulations and so on and so forth. But if I tell them that this is why we are doing the research then they might give answers which they think are desirable. So that again is a problem. And it's also related to something which is known as the Hawthorne effect. And what is Hawthorne effect? It means that whenever individuals know they are being watched in the workplace their productivity goes up. So if your boss is there or if your parents are there and if you're working in front of them probably it goes up. So it depends on the situation. But again the Hawthorne effect is if you're watched. So when you're observing people they might think that okay I'm being observed so they might be at their best possible behavior and they might not be giving you the true answers that you want. And as I said we are taking care of all these problems through our experimental research designs and also researcher attributes. So some people so if you're doing on some gender issues and if the researcher is a male then you might not get all the honest answers or the other way also. So there are problems. The selection thread I just told you that when we are not able to randomly assign participants into the experimental group or the control group. And in our term as I said we are often using experiment as comparison group also. So in the public speaking experiment that I just showed you we were unable to provide them randomly. So that selection bias can cause a problem. And the other problem and as you can understand that there are so many layers to it and it gets so sophisticated as you go into it. So one is that it's known as a treatment diffusion or the contamination. So participant with the treatment group will tell the control group about the treatment and then the answer. So as I told you the control group is required for certain reasons so that we take care of all the other possibilities. So because the control group is present the control group has access to information about the experimental group they might behave differently. So this is the presentation for today and I'm sure that this gives us a very clear idea about what experimental research is and how to go about it. Thank you so much.