 Good morning, welcome back to the second face to face day of the ISD workshop on research methods and educational technology and this is the next session. We are quite happy to see that there have been several submissions on for the idea proposal assignment during the last week. I think as of last night there were about close to little more than 1500 assignments and as was mentioned in the previous session, we will increase the we will extend the deadline for another day or two and we will also increase the number of attempts. So, you do not have to worry about the number of attempts and you do not need to send any more chat queries regarding that. So, let us actually get going and let us move more deeply into this session which we have called delving into the research methods of educational technology. So, so far in the last two days, in the last two weeks what we have seen is that how do we move along the journey from an E.T practitioner as faculty members and instructors and colleges to doing research in educational technology. So, this is where we started from right from the pre workshop assignment and you have seen the slide a few times. What we will concentrate a lot today is how exactly what exactly do we have to do to conduct systematic studies to get data and to provide evidence and so on. So, our goals as E.T researchers is after we have done the novelty and positioning is to make sure that we have we do conduct systematic studies and we have sufficient evidence to support our conclusions. So, what this part will do is establish soundness for your study which was one of the five things that we put up in the slide on what referees look for. So, let us actually see how one might go about doing it. So, before we talk about what one should do, let us look at what a beginning E.T researcher may be a PhD student or may be somebody who is doing an educational technology research study in the first the first time or the first few times might do. So, this is what we mean by a novice E.T researcher and this person knows that some systematic study has to be done and some data has have to be gathered. So, what this person might do is say that here is a problem I am trying to solve. They might start from the problem that they are trying to solve and they might even propose a strategy to solve the problem like all of you like what you all did in the idea proposal assignment. And the goal that this person is aware of is to conduct a systematic study and gather data. Now, what this person might do further is implement their strategy in class the same strategy that they have proposed. And after implementing the strategy they might conduct a test to check how well the students have performed by learning with my strategy or with the strategy that this person has proposed. So, this is similar to the marks that one of you mentioned last time and it is a very logical thing to do for a beginning E.T researcher that I implement my strategy and then I conduct a test to see how well students have done. Then this novice E.T researcher might make a claim saying that if my students got high marks, get high marks on the test it means that my strategy is working. So, here this story is what a beginning E.T researcher might do and the question we have at this point is, is this method which this method of conducting gathering data and establishing and trying to evaluate the strategy. Does this method establish that the strategy is actually working? So, what I like you to do is raise hands vote locally participants please do that and coordinators you can take a few minutes couple of minutes to do this. Coordinators please convey the percentage of yes and nos in your class. The question here is, is this method implement strategy and conducting post test is this sufficient to establish that the strategy is actually working. This is quite interesting because what I am seeing in your responses is that it is really very mixed. Some centers are saying that 100 percent of the participants are saying yes, other centers are saying 100 percent are saying no. Some centers are reporting there is a 50-50 breakup, others are saying that there is 70 yeses and 30 nos and some have the opposite proportion and so on. So, this is a challenging question and as we mentioned this is what we would tend to do in the beginning. So, what we will do before actually discussing the answer is see what a referee would do. So, the referee will say that the paper is rejected the method to establish the claim is not sound enough in most cases. So, you can stop voting now because we are going to discuss why the referee is going to make such a decision and a strategy or method that does look reasonably logical to establish a claim is not going to be accepted as a sound strategy by the referee. So, let us do this as a think-pair-share activity and it is the same method that we described earlier. I implement my strategy I do a post test to see how many people did well and if people perform well, if people if students get high marks I say that the strategy works. So, what we have to do now is try to understand why the referee said no that is what finally counts and in fact, the referee has a very clear there is a very logical reason why the referee is saying that this is not sound enough. So, the question is what are possible reasons this method is not the best to establish that my strategy works. What you all have to do as participants is first write one reason in your notebooks why this method is not sound why is it not enough to establish that the strategy indeed is working. Don't put anything on the chat window yet because we are not going to do any more voting at this point, but write one reason individually and I think you need about 2 or 3 minutes to come up with one reason. Think about it carefully. If you have written your answer what I like you to do is share your reason with your partner and we have been getting some replies on chat saying that the strategy is not accurate or this is not the best strategy and the question here is why is it not the good strategy. Some of you are trying reasons like resources may be limited and so on. Let us just actually go back to the question before you share your answer with your partner. If I implement a strategy and after implementing the strategy conduct a test that is all I do as a researcher. I base my claims on the results of the test that came after the strategy. Marks may be important here in this case. So, what you have to find out what you have to try to refute is the logic of this method why is this not very logic why is it not logically sound. So, if you have written down your answer please share your reason with your partner. You can assume that the test is actually measuring what you want to try to find out through your research objective. So, the test is a good test. Let us say you want to measure some conceptual understanding in an electrical engineering topic. Let us say the test has questions exactly to measure that. So, there is no problem with the test let us assume that for the moment. So, please share your reason with your partner another minute or so and then we will go to the next phase. So, we hope that you would have got some time to discuss your reason with your partner. And if you could please stop sending messages through chat for a few minutes we will get back to going doing this activity. A lot of you are saying that there is some problem with the test or the test does not measure performance. The test does not measure understanding and so on. So, our point here is that assume that the test is a good test it does measure understanding it is complete in terms of measurement of whatever your research objective is. In spite of that even if you have a very good test and even if the test can measure everything the logic of the method of doing this research is not sound. A few of you do seem to be heading on the right track. I saw one response which said that some students would have learnt even without the strategy would have done well on the test even without the strategy. And I saw a few of you saying that we did not know what the students knew before I implemented my strategy. So, those are the two reasons we are going to now try to explore a little further. What we will do in the share phase is do this locally the three reasons here. See if either you or your partner got one of these three reasons as what is wrong with this method. The first reason is that the result could have happened due to some reason other than my strategy. The second is that there is no comparison to a group that did not learn using my strategy and the third is that the result might have existed even before the strategy was implemented. So, these are three reasons why your method to conduct this particular research study is not sound. So, if you or your partner thought of this reason just raise your hand within your local centre. This is just to check your understanding and then we will try to understand what to do with all of these. What we mean by number one here is that well the strategy was implemented in class, but many other things happened or many other things could have happened. The students could have read the textbook, they could have gone and checked some site on the internet, they could have learnt through some other sources. So, there could have been several other reasons which could have been responsible for the outcome. So, it does look like approximately 70 percent of you in some of the centres have at least one reason thought of. And in the rest of the two hours today, what we will do is in fact go to each of these different methods of implementing the strategy and try to make the research method, try to make the logic of the research method sound so that we do not get a referee saying that the paper is rejected. What we saw just now, now let us move on to some technical jargon. So, again what we will do in this session today is start with the perspective of a ET practitioner and then introduce some technical words and technical terms which are necessary in ET research. The method, the research method we saw is called a single group post test only research design because you have only one group for whom you are implementing your idea and your strategy and this group only gets an exam or a test at the end. The reason a single group post test only design is not sound is that the outcome that is the result could have happened due to some other reason other than your treatment. We will come to all these words in the next, we will keep revisiting these words. Treatment is your idea or your strategy. There is no comparison to a group that did not receive the treatment. The outcome could have existed even before the treatment and the fourth one really covers all of these that if you have only one group and if you are only doing some measurement at the end, you as a researcher have no control of any other influence on the outcome. Several other things might be happening at the same time, several other variables might be playing out their effect and you have no control. So, that is why single group post test only research design is not sound. So, now our question is what to do? It did sound like a good idea, but now we have seen that it is not such a great idea anyway. And before we go on to looking at possible different possible methods of designing and implementing the study, what we will do is actually look at an example. And what I would like to do at this time is introduce a colleague, Madhuri Mavin Kurve and she will be talking to us about an example from her own research. Madhuri is a faculty member in Thakur College of Engineering which is an engineering college in Mumbai and she is also a PhD research scholar in the educational technology department at IIT Bombay and she present was an example where you can see all the things you saw until now and also some ideas on how to design a sound and systematic research study.