 Good morning and thank you for organizing the e-seminar on Steps to Research and I am very happy to be giving a talk in the seminar. The title of this session is From Teaching to Research on Teaching, so this is a seminar on research and the steps to doing research and most of the talks you might have heard so far might be pertaining to the talks related to your domain. In that sense this particular session will be slightly different and it is going to be different in two ways. One way is that the topic of the session is not research in your domain whether it is engineering or computer science or science or management, but it is actually research on your teaching practice. So what we will discuss in this session is all of you are instructors and teachers and you have been teaching for several years and all of you have been implementing interesting and innovative ideas in your classroom with the target of improving teaching and learning, your students learning and so on. So how do we formalize that and how do we think about that in terms of scientific research method is going to be that is the topic of the session. The second way in which this session may be different I am not very sure of that is that you will not only be listening in this session, but there will be several activities for you to do both individually and as groups. So whenever you see a blue slide which looks like this there will be something written on it. This means that you have to actually do the activity whatever is said on that slide. So for getting maximum benefit out of this workshop what is expected is that you sincerely participate in these activities, put in the effort to complete the task and challenge yourself to say something and do something beyond the first obvious thing. And if you sit and expect me to lecture and get something out of it I am sure that you have realized from the other sessions also that it is not going to be very beneficial. So with that let us actually get started on with the I am going to go back to the title slide for a moment because there is one word in it here my department is educational technology. So let us take a few minutes to see what is this educational technology because that has to do with research on teaching and how it is relevant to you. So educational technology actually comprises two parts. The first one is something which you will be more familiar with which is the technology for education and this includes the tools creation and use of these technology tools such as visualizations, animations, wiki, moodle. So all these tools which are useful in the teaching learning process become part of the technology for education. And educational technology also has another aspect another part which we call as the technology of education and this has to do with the strategies, the instructional strategies, the teaching learning practices that you implement in your classrooms and innovative strategies such as peer instruction, think-pair-share, debate and so on which are used to facilitate students learning and engagement. So research in educational technologies are actually include both of these and some integration and some combination of these. So what we will do today is look at some teaching learning practices on which combine both technology for education and technology of education and try to come up with some ways of conducting a research study using both of these. So what exactly comprises research in educational technology let us look at that before trying to see how to do that research. So when we are ET practitioners and a practitioner is somebody who practices which is you and me which are that means we are the teachers. So the teacher is a practitioner of teaching learning strategies and practices. So when we teach students, when we facilitate their learning, when we think about how to improve their learning and when we come up with ideas and implement these in class we are ET practitioners. So these are standard things which a good teacher does. But doing research in educational technology goes one step beyond that. So as ET researchers what we have to do is scientifically investigate how good our ideas are. We have to conduct systematic studies based on standard scientific methods to collect data and to analyze data which helps us decide whether our ideas work or not. And we need evidence, scientific evidence just like evidence we require in science and engineering labs that support our conclusions. So this particular session actually is about going from being an ET practitioner towards becoming an ET researcher. So that is going to be the goal of this session. And before we continue, you may be thinking I am doing teaching anyway in class, I am doing research in my domain perhaps. So what is the point? Maybe you are asking what is the point of doing educational technology research? So let me discuss one or two points where we show that each one of you is really well suited to conduct research of this sort. One is that anyway you are working on the problems coming in your class, the problem may be student disinterest or some topic is too complex and the student is not understanding. So anyway you are thinking about these problems and you are trying to fix them. What we are saying now is that if you would like to do research on these problems, why not systematically implement solutions to improve your teaching? And the key word here is systematically. Why is that the key word? Because as we saw in the last slide systematically means you have to scientifically investigate, you need data collection instruments, you need formal data analysis methods, you need to report it, you need evidence. So all this includes the systematic part of investigating and implementing solutions. And secondly why not take a few extra steps and close what you are doing? What does this closure mean? It means that if you systematically and scientifically do a research study on your teaching practice, others could adopt your solutions and others can be your colleagues but it can be people in the rest of the world also if you have written up about your solutions. You could get a publication in your name because there are conferences and journals on educational technology research and your skill in applying scientific method will improve if you do the first part and perhaps it would improve in other areas of research also because you get more practice in applying the scientific method. So as ET practitioners it is really worthwhile for each one of you to actually think about doing ET research based on the problems and solutions coming up from your classroom. The goal is worthwhile but now we have to think about the means of achieving that goal which like any other scientific discipline has a structure and has rules and so on. So in the rest of the stock what we will do is look at some of those rules and some of those methods which will help you go from an ET practitioner to an ET researcher. So one I will just spend one slide on the theoretical method that is behind all of this and that is called action research and what is action research? It is conducting research on one's own practice it does not have to be limited to classrooms but organizations and corporations use it a lot it is used in many other disciplines where the practitioner wants to do research on their own practice. So it is a discipline process of inquiry already you know the first sentence tells you that it is a systematic scientific process it is conducted by and for those taking the action. So it is by the teacher and it is for the teacher. When you do domain research the results are applicable by the scientific community. So the results are not only it is not specifically for the teaching practice here whoever is doing the research is also you directly using the results of those research. The primary reason for engaging in action research is to assist the actor. So here the actor is again the practitioner and a researcher in improving and refining his or her action. So the goal of action research is to improve practice and that is why this this paradigm is very suitable for us to do research on our own teaching. And if you look up what is action research you will see a diagram you may see a diagram like this that it is a cycle where you identify a problem plan research collect data. This looks like a very standard scientific cycle of research you may also see you may also see steps of action research given like this that you have to select a focus clarify theory identify questions collect data analyze data report results again if you simply go down this list it will look familiar to you because you may have exposure in research in your own domain. So in this session we will not focus in all of it usually we do a week long workshop for all of these but we will focus a little bit I mean we will focus a lot on this particular aspect on how to identify suitable research questions based on your teaching when you do education technology research and we will talk a little bit about collection of data. So what research questions should you be investigating and how to do the data collection and further analysis of this data that is going to be the focus of this session. So formally the learning of objectives of this session are that at the end of the session you will be able to write some questions write some research questions for innovative teaching learning ideas and then you will be able to state metrics for evaluation of these ideas that why these ideas will work and you will be able to identify instruments now an instrument is something which does a measurement whether it is an engineering or whether it is an education. So these instruments are data gathering tools so you will be able to identify what sort of data collection tools are required to evaluate your idea that you wrote in the beginning those are the goals of this session. So take a moment to read these because now we will dive into each of them one by one. So next comes an activity. So what I like all of you to do is in case you were distracted just get alert because the next three slides tells you what exactly you have to do. This activity has three phases it is called a think-pair-share activity some of you may be familiar with it. So what think-pair-share means is that in the first phase you will be working on the topic individually the what you should be doing will be told just in a second. In the second phase you will turn to your neighbor and you will do something and in the third phase you will actually share your findings with the rest of the participants in your center and the center coordinator will actually tell us and tell what is happening and you can use the chat window for that. So the RCs coordinators and the center coordinators also have a role to play in this activity. The goal of this activity is actually to come up with an innovative teaching learning idea that you implement in class and the key word is innovative. So think of something which you have done which is different, interesting, novel and so on. So let us actually jump into this activity and you have about three minutes for this activity. The idea is that in your notebooks individually write one innovative teaching learning idea that you have implemented in your class preferably using a technology tool. What is not allowed is giving a lecture or showing PowerPoint slides is something that is not innovative that is something very standard and routine. So go beyond simply showing PowerPoint slides and go beyond only the use of moodle because using moodle to upload assignments again is fairly routine. So think of something interesting, something which you did in order to for a goal in class. So what you should write here is in your innovative idea what does the teacher do, what do students specifically do and what you expect to improve by this idea. So take three minutes and write down this idea in your notebooks. After you write this down then I will come to the next phase. So if you have written some interesting idea that you have implemented in class or it could also be something that you may choose to implement later. Now what you can do is turn to your neighbor, anybody left or right neighbor and first share your answer with your neighbor and listen to your neighbor's answer and then this second point is where you have to do some work. Come along with your neighbor, come up with what you could measure to indicate that your idea is successful. So you have implemented an idea or you may want to implement a new teaching idea. So what should you be measuring which will help you decide if the idea is successful. So write statements like I think my idea is successful if I find that my students are doing blank or if I find that my students are feeling blank. So first thing is what you should be measuring to indicate the success of your idea and secondly select any one of these statements and try to determine how you will measure the success. Here it is just what you will measure. So you may say that I think my idea is successful if my students are doing this behavior. So how will you actually measure that? So this is a pair activity because it is not very straightforward to do that and I think you will require 3 or 4 minutes but at this point first share your idea with your neighbor, listen to your neighbor's idea and for each idea try to come up with what you can measure for success and how you will measure it. Start talking to your neighbor. This is not the time to be looking at the slide please share your answers with your neighbor and start talking. So if you have finished doing this with your neighbor what you can do is in your own center share your answers with your colleagues who are there in your own center. Coordinator just wait for a minute because listen to many answers. So share your answers with your colleagues either I mean if you are in Amal Jyothi or Samal Jyothi or some other college and mention both what is to be measured and how it is to be measured. The what and how both are important it is not just the what. Mention both of these and once the coordinator hears a few answers what you can do is share the 3 most common answers over chat actually let us make it since there are many centers you can share the most common answer. So here it is not just the teaching learning idea that is important but what is to be measured and how to measure it that is what is important. If you think your idea is working on what basis are you making the claim. So anybody who has metrics and evaluation ideas please tell your coordinator and coordinator please let us know via chat. One idea here is that from Purnima University Jaipur is that the innovative idea was to get students to prepare objective questions for a test. So students are not just reading the they are not just taking the test they actually have to prepare questions for the test. So it is an interesting idea and the next part of the idea was that students in teams of three find out the best questions. So this is a very good idea. So now what you have to think about is how will you evaluate if this idea achieved what you wanted it to achieve. What will you measure to check if the idea was successful. Some other people have given ideas like an activity of crossword puzzle was given best answer and so on. So what it looks like right now is that many of you have good ideas teaching learning ideas and many of you are implementing it but it does not look like you are actually thinking in terms of how to evaluate success of this idea. So at least right now it does not seem to be clear to you how to actually evaluate how to measure if the idea is good. So let us move on and hopefully by the end of the session you will get a better handle on how to do this. So let us go to how to identify research questions. So you have your ideas. The ideas are good. Learn via role play is something which somebody has said. Somebody else has said that you will get students to do a virtual lab. You have said you will use a mobile application like WhatsApp for teaching something. The ideas are good but hold on to your ideas for now. We will come back to the ideas later. So the goal now is to see how to take your idea and turn this change this into a scientific study. So the first step of any scientific study is how to identify good research questions. So what is a good research question in education? A research study contains research questions. It has to have answers and it has to have evidence. So let us so please stop sending the things on chat. Let us look at features of a research question and why something is or is not a research question. So a research question should express a relation between two or more variables in a context. So if you ask a question as to are animations effective? That is not a good research question because it is not clear what is the two variables what relationship is being expressed. So are animations effective compared to what? That is not clear in this question. What does effective mean is not very clear in this question? What is the context is not very clear in this question? So one feature of a research question should be that it expresses a clear relation between two variables in a very specific context. So here is not this is an example of something which is not a research question. One more example is that a good research question has to be stated as a question not as a statement. So if you say something like the purpose of my study is to gather data to support my idea of showing animations this is not a good research question. Because what exactly is it is not stated in the form of a question. The 1, 2, 3 that you will see that three conditions these are the criteria of good research questions. And then we will actually see an example of a good one. The third criteria is that the research question should imply possibilities of testing. This is a very important idea that the research question should be written in such a form that you can actually do an experiment based on the question where you can gather data and gather evidence. So if you say something like should one use animations in primary school classrooms it is hard to do an experiment that answers the word should because the word should means there is some moral judgment involved or there is some rules involved. Research questions have no moral judgments. A research question should allow for empirical testing. So a good research question which has all these three features is what is given in the next slide. If you ask a question which says are animations more effective than still visuals like diagrams for conceptual understanding of electromagnetic fields. This question has all three conditions met because it expresses relationship between variables. The variables here are effectiveness is a variable and the mode of teaching animation versus still visual that is another variable and the question is asking is one more effective than the other. There is a specific context, the context is understanding of electromagnetic fields. The question is in question form it is not a statement and most importantly it is possible to empirically test this. So you can set up an experiment where some students learn with animations, other students learn with visuals in the context of electromagnetic fields and you measure their conceptual understanding through a test or some other instrument. So a very important or three very important criteria or conditions are these three here. Let us look at a few other examples. So this was a study in which ThinkPairShare the activity that you just did was implemented in a first year programming course. And what the researchers asked are these three research questions. How much student engagement occurs in the ThinkPairShare activity? How does the amount of engagement change as the activity progresses? And does do TPS activities lead to increased conceptual understanding and application of programming concepts? So I will leave this on for a few minutes, for a minute or so. Check if each of these three criteria are met. Is there a relationship between variables? Is it stated in question form and can you actually think of this? Can you think of performing educational experiments in your class which actually meets these? The next example is an idea where teacher, so this is an idea of using an animation. Normally when we use animations we just show the animation. But here what the teacher did is showed part of the animation, only the first half of it paused, asked students to make a prediction of what will happen in the next step. And then compared the prediction by showing the rest of the visualization. So the students, the teacher did not simply show the animation, that would be a routine. But the teacher stopped the visualization in between, asked students to make a prediction of what happens next. Then resumed to check whether the prediction was correct or not and then did a discussion. So the research questions should be, does prediction activity with visualization lead to higher levels of learning than simply showing the animation? The two conditions were simply show the animation or actually pause it and do the prediction and compare the learning outcomes. What are student perceptions about learning from visualization with this strategy? And you can also ask questions like what are differences in learning and perception between high achieving and low achieving students in your class. So the last question actually helps you understand for whom this strategy works better. So these are ways to write research questions. I will just show you one more example. This is an engineering drawing, this happened both in an engineering drawing and a computer graphics course. So students used 3D animation software like Blender in a lab. And the research questions were did this Blender training improves students mental rotation ability. So in engineering drawing and graphics, students need to actually be able to rotate 3D objects and try to visualize what happens behind the object and so on. It is very hard to do in pen and paper. So they used the software to do this. So in all of these studies what I want you to focus on is the way the question is written. That the question implies how the possibility of what to measure and how to measure. So if this is clear, what comes next is an activity of your own. Remember the idea that you wrote some time ago? So what you will do now again in think-pair-share format is revisit your answer to the think-pair-share activity the previous one and write one research question for your study. This should be a question, it should not be a statement and it should imply a possibility of doing an experiment. In the pair phase, check your neighbor's answer and confirm that the research question is actually a question. And in the share phase what you will do is convey your argues to your colleagues and coordinator please share over chat one question, single research question, the best question from your center. Again now we are no longer looking at ideas, we are only looking at research questions of this format. Does something work better than something else or what are students perceptions if this is done? So from each center please send us the best question. Firstly what you have to do is spend about three minutes to write to do the think phase and then move on to the pair and share phases. If you are sending something by chat, please do not send us the idea. Send us only the research question that you will see this is a session on how to do research on your teaching. It is not just about the ideas you implement. So you have to now start thinking in terms of scientific research. So what is the question you will ask to check if your method is actually successful? The question is about has to be about students learning or students engagement because this is not research in a domain. It is research on the teaching. What I am going to do is put back the slide of the criteria of three research questions. Your research question should be of this nature. Okay so here we have a question, a good question I think. Does the idea is doing role play with students? So does student role play activity to teach a technical topic more efficient compared to a traditional lecture? This is something you can actually test and measure except that you have to be very specific as to which technical topic you are considering. But this is definitely one question which has possibilities of empirical testing. Another question is does group discussion after completing a topic improve conceptual understanding? Well this is firstly the idea is something very standard and you have to think of how you will measure. Somebody else has asked can providing PowerPoint slides of the lectures one day before will it lead to improvement in the learning process? Perhaps you can think of measuring this that might be possible. Somebody else has said does this novel activity, you have not said what the novel activity is, but does the novel activity implemented improve problem solving skills? That is something you can definitely measure. So don't, here is an example of a question you should not ask. Do not ask questions whether PowerPoint is improved compared to lecturing on Blackboard. That is really not a very useful question to measure. Okay so you have seen examples of questions which are not valid and we also went through examples of three questions, questions of three examples which are valid research questions. And you did make an attempt in your own study in your own idea to write these questions. So what we will do next is focus a little bit on what you can measure specifically. So typically a research study in educational technology has different goals. The question is why are you implementing your new idea? And you can focus on one or two of these goals. There are four possible goals. Do not try to attempt all four goals or all four metrics in the same study, but you can attempt one or two. So the first goal is called effectiveness. And effectiveness is improving students' learning of concepts or skills like problem solving skills. This is something which we will be most interested in. The second is what we call as attractiveness which is enhancing students' engagement, their motivation, their confidence and so on. The third is related to scaling a solution to larger contexts which is maybe you have implemented an idea in a small group and you want to scale it to a larger group and you can use technology to do that. And the fourth goal is to save time or save effort and so on. We will perhaps not look into this too much at all. So in the rest of the session we look at effectiveness and attractiveness as metrics or goals for why you are doing your teaching learning idea. So this is what to measure. Remember in the very first activity we had a what to measure and how to measure. This is the what are possibilities to measure. So in how to measure, here are some possible ways to measure. So tests are very commonly used instruments for measuring effectiveness and learning. So there are some do's and don'ts of tests. These are usually not the same as final exams. These tests have to have some specific conditions again. You can use survey questionnaires to measure students' engagement, their motivation, their confidence and so on. And you can also do things which we won't discuss in this session. But if you are more interested you can look them up, focus group interviews, observations and other such tools. So if you plan to use tests to measure effectiveness of learning, what evidence you need is improvement of a very specific concept. What data you should collect is performance on a test related to the concept before and after the treatment. And preferably between a group that did the treatment is the intervention, the idea that you implemented. Preferably you get data on students who worked under your idea and students to whom your idea was not given. And what you should use as the test preferably is a standardized test which has conceptual reasoning questions related to this concept. So you should not use a final exam because that may not have specific conceptual questions related to what you want to measure. If you want to use a test as your measurement instrument then you should create these specific questions which target the concept or skill. So if you want to measure problem solving skill you should have questions, problems in your test which actually measure the specific problem solving skill. If you want to measure students' debugging skills for example programming skill or a debugging skill, you should ask students to write a program to solve a problem. If you want to measure students' debugging skills you should give them an erroneous program and ask them to debug the code. What you should not do is ask questions like what is a variable. So you are right that tests actually can help you measure what you want to measure but the test must be very specific to the goal. So if the goal is debugging skill then what you should do is give them this erroneous program and analyze how many errors there are in the program and classify them as syntax and other kind of errors. If you want to measure students' engagement with the topic you should collect data of students' perception of their interest in the course. You can also measure things like attendance or participation rates or how much time students spend on task. And you can measure these using questionnaires, you have to actually find these questionnaires or construct these questionnaires. So this is all how to measure student engagement. If you are constructing your own survey questionnaires you should not simply ask do you like it or do you not like it. Instead you should ask specific questions on whether students you know on what you want to measure like students' perception. Ask a lot of specific questions. Ask about students' interest. Give them a rating scale like do they agree that the course was interesting? Do they agree that the method helped them learn? Do they agree that group discussions improved their motivation? Don't ask a single like dislike question. So what we will do now is look at two or three examples and I would like you to vote just a yes no vote initially on whether this is an ET research study or not. So there will be a scenario given and based on what we discussed in the last 15 minutes or so, vote whether this is a valid scientific research study. So in your centre just put up your hand and say one for yes and two for no and the centre coordinator can convey by the chat the majority vote in your centre. So here is the first scenario. Your colleague says that I will prepare interactive multimedia content and animated videos. Using modal student can access the videos and student is more interested in the study. So this is what your colleague says. Is this a valid scientific ET research study yes or no? Vote in your classroom, in your centre and coordinators please convey the majority vote. And based on what we learnt so far on what is a good research question, how to measure and so on. Okay we got a number of yeses but the answer is no this is not a research study. This is not a valid scientific research study it may be a good idea but it is not a research study because simply developing material and putting it up on modal is not scientific research. So if you go back here what the person did has prepared interactive content, has put it up on modal and thinks that students may be interested. It is not clear from here what the student what the teacher will measure. It is not clear from here how the student how the teacher will measure. The teacher only thinks that videos will help students learn concepts. So this is not an ET research study. So mere development of instructional material or strategy is not an ET research study even if the material is based on an innovative idea. So in order to be considered as a research study what you need to show is how it resulted in student learning. You need to have research questions, you need to have instruments to measure data. So to make this idea into a research study you need to have evidence. For example you can give a quiz to test students understanding on the concept before and after. You can prepare a questionnaire that ask students to about their preference of using multimedia content or traditional content. But the example you saw here no the answer is no this is not a research study. The referee if you write a paper like this the referee won't accept it. And the reason the referee won't accept it is because of these problems. Okay let's do one more example. Your colleague says that the purpose of the study is to use modal in an engineering course and study the motivation behind use of this participant. So what the person is doing is using activities such as presenting information, managing course material and evaluation through modal. Is this a research study? We are getting mostly S's and a few no's. Again most of you are thinking yes. However, so okay let me tell you the answer. The answer here again is no. It is slightly better than the previous one because the person the colleague is actually trying to measure something. Because the colleague says instructors were asked the benefits and barriers to using modal. But the problem in this study is that the reason it's not a research study is that the educational technology tool which is modal is used in a very routine manner. What does routine manner mean? Modal has been used to present information to manage course material and evaluate student work. This is very standard so when you are trying to do ET research you want to go one step beyond this. What you want to do is something like use modal to create a game. Use modal to check how much collaboration is happening. So go beyond simply uploading assignments and getting students to download them. The good things about this example is that at least the researcher is trying to do some measurements so that part is okay. But the idea here is to routine. That's the problem here. So let's summarize what is and what is not good ET research. That not ET research is when you only develop instructional material and don't think about measurement that is not ET research. You have to think about measuring either learning or engagement. If you use ET tool in a routine manner that's not ET research again. Even if you do measurements but your ideas varies standard and straightforward it's not good research. And if you simply report how you did an ET, how you implemented a strategy that is also not good ET research. So good ET research should have research questions. It should use one of those technology enhanced learning metrics like effectiveness or learning attractiveness or engagement. It should gather data. It should collect data to provide evidence for these metrics using tests or surveys. And you should go one step beyond routine application. So you might be thinking at this point okay how do I go beyond the routine I can only think of using modal for uploading assignments. For this you have to read start reading ET research papers. Just like in your domain scientific research in your engineering or science domain you read research papers. You have to read ET research papers also. So here is one example of a good study. And this is an abstract in from a SIGC conference and take a minute read this. I'll highlight what is good about this and then we'll move on. The goal here was to compare peer instruction. So peer instruction is a method that's used with clickers. The teacher poses some multiple choice questions and students vote on the answer with clickers. Then they do a discussion and they vote again. So if you analyze this example you will see that there's a very precise problem description that we look at the impact on student learning of the pedagogical approach. So the problem which is being targeted is very clear. The pedagogical approach is their strategy. So they want to measure student learning using their strategy. The solution approach is novel. They use peer instruction which is not a routine method. Their procedure is very sound. They compare two sections of programming. One is taught via peer instruction and one is taught using some other method. And what they measure is student learning and they evaluate it. They find that the peer the section which was taught using peer instruction strategy scores 5.7 percent higher than the standard lecture class. So a research study has to have all these elements. Mainly start with a good research question. Then think of what to measure and how to measure and then go and do the measurement. So if you get started at this level at least you have a that's a good start to do research on your own teaching practice. So what next? And you may be wondering okay where will I get these ideas? Where will I see more examples? I will give you one or two examples here. One is that in India we have a conference called Technology for Education. And this happens every year in December. This year's conference is in Amrita University in Colom from December 18th to 21st. So please come for this conference. Please register and there are some scholarships and fellowships for engineering for college teachers and school teachers to register. A lot of your registration fee may be reimbursed. So you will be able to learn about a lot more such research studies using technology and education. Once you get ideas from this conference and meet other people who are doing this work you can actually consider executing your own idea for next year's conference. And if you want to do it for next year where this year's the deadlines are over since the conference is just three months from now. But please do come for the conference. It will give you a lot of exposure and good workshops on how to do ET research. And next year you can actually plan an idea, conduct an experiment and write up and submit. And we have some resources which are templates on how to plan, conduct and execute your idea at this URL. If you go here you will be able to find lots of resources. So let me in the official session here. What we can do now is about 10 to 15 minutes of question and answer if there are any questions from the centres. Purnima University, Jaipur. Good afternoon ma'am. Hello. My question is in a one hour lecture. Is it possible to conduct think-pair share activity? Good question. Is it possible to conduct think-pair share in a one hour lecture? So the answer is very much yes. The entire think-pair share activity usually requires about 15 minutes. 10 to 15 minutes maybe it depends on how much discussion you do in the share phase. So what we do usually is we do not implement think-pair share in every single lecture. But in some lectures let us say there is a problem that needs to be solved or there is a discussion that normally you would do. We convert the problem or discussion into a think-pair share activity. So the recommended time for one entire think-pair share is about 15 minutes. And if you go to that URL that I put up on the last slide there is an entire video and slides and lots more resources on how to do think-pair share in your class. Question from SVITS Indore. Ma'am my question is that we have suggested for the mobile technologies. So what are the disadvantages regarding and how can we overcome it? Like using the mobile, we have the battery side effects, the time consumption and other effects that create problems. So how can we use? So there are many ways of using mobile technologies in the classroom. And the thing is the idea that you implement, the teaching idea that you implement should be very specific to the topic and the goal. So it is not that people are on their cell phones and tablets all the time which anyway they are outside of the classroom. So you have to have a specific question or activity for students to do which needs the mobile technology. So for example let us say you are using a tablet. Let us say all the students had Akash tablets. You can ask them to interact with the simulation on the tablet and then answer the question that you have already posed. So it is not simply go and play with the animation. It is interact with the animation, answer the question that I have already posed on the board and in two minutes come back and answer the question. So you need to have very specific tailored activities for students. That way the technology will not become a distraction. If you completely leave it then I mean it is still possible. There are people who implement mobile technologies continuously also. But the distraction might be a little hard for you to control in the beginning. So for effective learning think of coming up with a very specific activity that includes or needs the mobile technology. FAMT Ratnagiri. My question is that many of the times if the research work is signed to the best impact factor journals. So it takes more time, more than six months and seven months to publish the work. So it is so long time. One year also it will be text. Yes that is standard in scientific research. You can think of publishing in conferences if you would like. What is the procedure to publish the papers in best time? See that is the standard time for scientific journals. There is nothing you can do about it. That is the way the system works. What you should do instead is try to find a relevant conference or journal. The time for conferences is usually shorter. So if you are interested in speedy publications you can think of reputed. Do not go for any conference but go for reputed conferences. But a better way to do it is to find a good which means a reputed, peer reviewed conference or journal which has a good fit with your work and use that. The time is not something you can control. Thank you. MGMC and Nandit. My question is when we write abstract then whether it is required to write on which data set it is performed that experiment is performed it is required to do that or not. It depends on the domain. Are you asking about educational technology research or something else? Generalised abstract. Generalised abstract there is no general answer. You should actually ask an expert in the field. So if you are doing research in computer science, ask a computer science expert what is the best thing to do for that journal? For education technology is it some data bit present for education technology? Data is required, data is required in papers. Yes, data is always required in education technology papers. Even if you are doing tool development you should at least pilot it with some users and try to get usability and usefulness perception data. So in the meantime let's take one more question from SDES. Madam my question is what should be the ideal number of samples that we should include in the research work? Number of samples. Again you are asking about, are you asking about education research? Yes, suppose if you are going for any case study or something like that. See again it depends. So what would be the ideal number of samples? So it really depends on the question that you are asking. If you are doing quantitative two group experiments. So let's say you have one group learning in one method and another group learning in another method and you want to do statistical analysis between these two groups. It turns out that most statistical tests are valid or have their power only when the number of participants or number of data points in each set is 30 or 40 or more. If you have 10 and you are trying to do statistical analysis like T test the errors are too huge. So some statistics are not even applicable for small data. On the other hand if you are doing detailed interviews and are using methods like grounded theory to code the interviews then you can go for a smaller sample. So there is no single answer here it depends on what type of analysis you intend to do. One more thing madam. Say suppose if we are having some existing technology so we should carry our research with the existing technology or we should call something which is not yet there existing. Okay so that's an interesting question that if there is an existing technology can we do research with it. See one aspect of research which whether it's education research or engineering research which is highly sought after is novelty and novelty means whatever research you do has to contribute to the community in a way that nobody else has done before. One way to achieve novelty is by actually inventing a new tool for a given problem and doing it that way but that's not the only way. So if you have an existing tool you can think of using the existing tool in a novel way then that would still be valid as a research study. So for example think about the Moodle example. So Moodle is an existing tool lots of people are using Moodle but the primary use of Moodle is to upload assignments or get students to download those assignments. Some people give quizzes on Moodle, you make announcements on Moodle these are all routine uses of Moodle. But now if you say that okay I'm going to create a game so some new versions of Moodle actually come with a game module so you get students to play a game and chat with each other on the game which is related to the topic now you found a novel use of Moodle or use Moodle for some collaborative projects of students that may be a novel use. So if you use existing technologies you should think of novel ways of using them. If you use existing technologies in a routine manner it won't count as research. Okay so this is the previous question from COE Belgam. The question is what is the impact of doing research for getting PhD in UG level engineering colleges on the quality of lecturing is this kind of research helpful? So I have two parts to this answer. One is if you do research in engineering domain let's say you do a PhD it will definitely help your teaching also it's not only for promotions it will help your teaching because what you're bringing to your class and to your students are the most current up to date knowledge of your field. You will improve your thinking will improve and so your students thinking will also improve. So doing research in an engineering domain even if you're teaching UG level is actually very useful. But if you're now thinking of doing research in educational technology that is also useful and in fact in our department Tiety Bombay we do have a PhD program which for doing research in educational technology and the participants the students in this program the research scholars are some of them are college teachers from science and engineering colleges also. There is some problem of getting it recognized in their colleges so that is something which we have to live with right now but there are we do have college teachers also doing educational technology research. So whether you do engineering research or educational technology research that's a call you have to take. But the goal of this session was that since you're all teachers and you're anyway doing things to improve your students learning you should think of going that one extra step and doing it more systematically. You could get a publication in the T4E conference that I mentioned earlier is actually in IEEE International Conference. So it does carry a lot of weight. Okay let's take two more questions before we end this session. I shouldn't eat into your lunch time. Purnima University, Jaipur. Ma'am my question is how can we perform the effective data collection relevant to our research? Well if you ask a very broad question like that I'll have to give a very broad answer and say read some research methods textbooks but what I really mean is this in today's session I gave you some instances of what type of data can be collected that if you want to measure learning of a particular topic you can do pre and post concept tests. If you want to measure student engagements you can do like a scale questionnaires. If you have a more specific question or you want a broader answer you really have to read some research methods textbooks once your research question is defined. There is no one single answer that what is the best way of data collection. So data collection methods include test scores, questionnaire, you can do interviews, you can do in-class observations, focus group interviews, there are many many different possible instruments and you have to determine which one is the most appropriate for your research question and the topic that you are doing research in. Shree LR Tiwari College of Engineering please go ahead. Is this a good idea to teach practical subjects like you know accounts and statistics through PPD. So I would like to know from you. Is this a good idea to teach? Again when you say through PPD, PPD is just the tool. The blackboard and chalk is just the tool. What is really important is what you do with the tool. So what teaching learning strategies are you implementing? So for example, I used a PPD today to do a think-pair share. So PPD itself is not the strategy. If you want students to be able to do something hands-on, you have to give them hands-on activities. So in practical subjects, it makes a lot of sense to give some activities for students to actually do something hands-on. So in statistics course, give them a small statistics problem. It can be even 5 minutes long or it can be even 2 minutes long. But let them do it along with their pair, along with a neighbor in the class. So even when you are thinking about what is the best method, I would encourage you not to think only in terms of the tool. Don't ask, should I use blackboard and chalk or should I use PPD? What matters is what you do with the blackboard and chalk, what students do in that class, and essentially what activities they are doing. Okay, I think there seems to be one more question. Let's take that question. Dronacharya Gurgaon, please go ahead. Good afternoon, madam. Good afternoon, madam. Hello, please go ahead. Myself, professor, Dr. Archie Sharma from Dronacharya College of Engineering, Gurgaon. Madam, my question is, we sometimes do research on questionaries based. And if we change the number of respondents, then our results will change. In such cases, how it is going to come to a conclusion which one is correct and which one is not? Because the entire research depends on the number of respondents. Right, so this is actually a very important question. It's a good question that how do you decide the numbers and very likely in questionnaire based research, the number might change the results. So if you read good research methods textbooks, there will be a whole chapter on what's called sampling. And sampling says, actually what sampling tells us is how to choose the participants for our research and also tells us that before you begin the research, before you actually go and implement it, you have to have made decisions on who these participants are and how many you will pick along with justifications of why. So for your research question, you need to have clear justification and arguments saying that these are the number of participants I'll pick and these are from, these are who I'll pick. And the rule there is that your participant, the total set of participants, in the end you should be able to generalize to the larger population. So based on the future criteria of generalization, you have to choose your sample. Now once you choose your sample and once you have defended that this is going to be my set of participants, you're not allowed to leave or usually you're not supposed to say or you're supposed to include all of them in your data analysis unless some peculiar things happen. So what might happen here is that some participants may not fill your questionnaire. Some may have initially said that they'll do it or they'll do part one and not do part two. So what do you do with incomplete questionnaires? What do you do with people who don't show up? And again study by study you have to make decisions of these because it may be possible that the participants who did only half the questionnaires are the ones who are not interested and they dropped out. So suppose your study is about finding people who are interested or not interested, then it's very important to see how many dropped out. On the other hand, if you can make an argument that the dropouts are actually not important for your study, then and if you include the dropouts, then your results will be skewed. If you're able to make concrete arguments, then you're allowed to drop them. But all these arguments have to be given before you see the results. And these are called threats to validity, threats coming from sampling and a research methods textbook will actually tell you what are these possible threats, how to look out for them and how to minimize them. Having said all this, usually if you have larger numbers and I'll come in a moment to what is meant by larger, your threats are the results won't change too much if you drop a few or not drop a few. So if you have a set of 20 and you drop 6, your results might change a lot because 6 out of 20 is not very large. But if you have 150 participants and you have to drop 5, your results may not change much. So again there, this topic, this point might be important. Okay, let's go to one last question from Nandheed. My question is while doing the research in biomedical field, there is a problem with analysis of results. Means whether we are getting a correct result or not. For that one, we want to take the help of doctors while analyzing the results. So is there any other solution for this one? You really have to ask somebody who's doing research till engineering because I'm not an expert in that at all and I really don't have an answer for that. Sorry. This is a specific question related to my research. No, I don't know. I'm asking for the solution. I don't know. I'm not an expert. Okay, so thanks for a lively session and hope the rest of your workshop goes well. So we'll end the session here.