 So, it is a 401 and it is time to get started with the last or last, but one session. So, in this session we are going to look at how to plan the study, what to do I mean we have talked about the various studies and then what the next steps are ok. So, once again let us look at our journey at a glance. In the pre workshop you started with your idea of a teaching learning problem. On Feb 2nd you evaluated your idea for strong paper features. In the last week you filled up and submitted the idea proposal form. Today you further learnt about study planning, what you will be doing later will be submitting the study planning proposal and then onwards to submitting a paper draft. So, that is the process that is our journey so far and we are somewhere here in the journey. So, once again what has happened is that two aspects of novelty and positioning have been addressed on second Feb and the other aspects of soundness and evidence to be generated have been addressed today. So, in order to plan the details of your study we have what is called a study planning template which will be uploaded on Moodle. So, initially I thought that we can take a look at this study planning template together so that you get an idea of what is there, but then now I feel that maybe you have already heard a lot about various methods and it is been a lot for you to absorb lot of new information you have had to absorb. So, we will not go through this template right away because again I doubt if any of you is going to start filling it out tomorrow. So, what we will do is most of the slides in that template are self-explanatory and there are examples to help you fill out the template and you can post queries if you do not understand anything about the template in on Moodle which we will answer. So, just to give you an idea this template is about 40 PowerPoint slides of which 10 of them are instructions and examples and the remaining 30 you have to fill out. So, that is the kind of scale of work that you are looking at in order to complete this post workshop assignment ok. So, another announcement is that there will also be a post test and questionnaire which will be uploaded on Monday. So, by now you know what is the meaning of the word post test. It is basically to see how much of this jargon that you have acquired and how much understanding you have got about the various research designs that we have talked about today. So, is this a single group post test only? Just think for yourself for a minute. What are we doing? I mean would it have made sense for us to conduct a pretest, would it have made sense for us to have a control group? So, just think for a minute. Anyway, we are going to do a post test which will be up on Monday ok. So, now once you fill out this study planning template what are we going to do with the submission? It is the same reviewer funnel is here. So, your submitted study plan is going to go through the standard review process. So the moment you make a submission your workshop participation certificate will be ready ok. So, we were initially thinking about putting some minimum marks for the submission and then we decided that we will just go ahead and give the participation certificate to everyone. So, there is a chance that you know you can get away with not writing much in the study plan, but then that is then it is your paper which you are working on. So, we expect that if you have been with us till now you will be with us seriously through this process. So, then our TAs will act as referees and your submission has to pass these four filters in order to reach this point where you will get further mentoring for executing your study and for T4E paper writing. So, another point that will happen is at this point some of those who have submitted really stellar study plans. So, we have already seen some of them in the idea proposal template itself. Some of you have done exceptionally well in the idea proposal template and those of the those of you will also become mentors and referees. We will kind of co-opt you as TAs for our work. So, it is not just our work it is your work also because we are all trying to together raise the standard of the country. So, now let us do this activity it is a 10 minute activity you have 10 minutes. You want to reflect and individually write down. So, this is not just think it is not just keep in the mind it is actually write. So, write is the keyword here. So, write down one point that you learnt from this workshop and write down one point that is still not clear to you. So, take about 3-4 minutes to reflect about what all you learnt and maybe you can write two points that is all right. So, do this writing before you go on to the next step. So, I am seeing people are sending responses through the chat even though we have not asked for it. This is actually very good. It shows that you are very proactive about carrying out the activities. So, and some of the ideas are very good we like the idea about plagiarism and how to avoid it, how to publish a paper in a standard format, first point of effective research especially strength and weaknesses, then what is research and how the report should be, what is novelty. We learnt about plagiarism, difference between citation and references, how to analyze strong and weak paper, how to apply teaching to make innovations among students. Some queries which I do intend to address later are coming back again. So, just hold on to your queries if we have not addressed it till the end then we can post it. Effective research. So, there is one comment here while you are thinking about this, it is useful to think about it in terms of sentences. Now, I understand that since chat is a very short medium you have to write words like paraphrase or effective research, but in your own mind when you write down on your paper you should kind of be able to write sentences like I learnt to be able to do. So, what is it that you have learnt to do? So, that is the main thing of reflection and now you can actually go on to this other stage of briefly saying what you learnt and hear the others view with your partner or with your group. You can do this in a informal setting also as the entire centre and simply go quickly around and say what you learnt and hear what the other people have learnt. You can also mention the point that is not clear to you. So, that if somebody else in your centre knows the answer, the point can be cleared right there. So, we can stop sending the chat messages now. So, let us come back and hopefully many of your queries have got cleared in the session itself and some of them we will be taking up now. The key reason for doing this activity, so this activity is again a standard activity at the end of any teaching learning process. So, this reflection is part of any teaching learning process. So, unless we do this activity we do not realise what is it that we have learnt. So, that is the whole idea and here again the technical term is metacognition. So, that is the purpose of this activity. So, once you know that you have learnt something your knowledge of that becomes a little stronger. So, that is the whole purpose of doing this activity. So, you could in fact go back and revisit the slides over tomorrow, over Sunday and get to see what is it that you have acquired in this workshop. So, what next after this, so I think we can stop to the chats for a few minutes and what next is after your study planning assignment is done you should be ready to execute your study. So, at this point you want to recall the soundness of the procedure and the items related to ethics regarding the experiment subjects. After that what are you going to do? After your study is executed you are ready to write your paper and once again here the things to recall are bring out the evidence and the ethics related to reporting prior work. So, later on once you have submitted the study planning template and gone ahead with executing the study. We will share a paper planning template which will actually help you to take the work that you have done and put it in the form of a paper that is expected in a typical academic forum. So, the forum that we are considering is T4E, T4E stands for Technology for Education T4E 2013 is the conference that we are targeting, we are hoping that many of you will submit papers to this conference and we do hope to see a lot of you at the conference after having got your papers accepted. So, the conference is in IIT Kharagpur in December of this year. So, for those who pass all the filters we will provide feedback on your paper draft towards T4E submission. There is a link for the call for papers you can simply Google T4E 2013 it will take you to the correct link. So, most of you will have the focus area as innovative content development and classroom use and most of the type of your submissions based on what we have seen in today's this week's workshop has will be in the category of experience reports. So, what are the T4E timelines? So, the T4E submission deadline is 30th June which is quite a bit away it appears to be so that we can start a bit later, but it turns out that we do not have all that much time because if you work backwards you need to polish your paper during most of June. So, which means that you need to finish all the work and update your draft by 30th May by end of May which means that you should have your draft written by end of April. So, which means that your study experiments have to happen during March to April. So, 2 months is actually a good time to 2 and half months is what you have to actually take your idea plan your study and execute your study and you know for that to happen you need to finish planning your study by 28th of Feb and as Meenal had mentioned earlier the key thing is spending enough time on this planning phase. So, if the study is well planned then all the other phases will fall in place on the other hand if the study is poorly planned we will spend a lot of energy in execution lot more energy in trying to figure out or make sense of the data that has been collected and it can lead to a lot of frustration. So, planning the study is the key step so the next assignment is due on 20th Feb which is the study planning template that we have not yet shown you ok. So, let us also talk about collaborations there were some queries on collaborations. So, collaborations is a good idea finding a partner who is working on the same or similar problem to collaborate with is desirable. So, we can have study planning templates submitted jointly well maybe both of you can make the same submission ok. So, maybe both of you can or one of you submitted through one of your IDs and the other person also makes a submission saying that is already submitted put both your names on the submission if you are collaborating. So, again here there is a point that is worth keeping in mind and that is called sleeping partners ok. So, it may happen so often when you would have seen it in your own classes. So, when you give group assignments there will be one person who is doing all the hard work and three others who will just add their names to the work at the end saying that I also did this submission. So, this notion of sleeping partners has to be discouraged because it is there it does not really add value to either the one who is working or the one who is sleeping right. So, make sure that you find a partner who is interested in working with you on that problem ok. Recall the benefits of peer review both of you can plan and discuss the details together and this is again a very important thing that you can gain because of this setup where the benefits of triangulation. Now, let us say there are there is somebody in the north of India who has an idea and there is another person in the south of India who has a very similar idea and both of you happen to be registered for this workshop. So, this the Moodle forums that have been set up are an excellent medium for you to discover each other and for you to start collaborating even though you may not meet each other till the actual conference happens. So, this will actually make your study very strong because each of you can implement the study in your class and together report both the sets of the findings. So, this is one of the benefits of triangulation that you saw in one of the earlier sessions ok. So, once again this idea of peer mentors those who do well in the study planning assignment will act as peer mentors for the others and keep in mind the ethics of peer review because now you are going to be sharing ideas with each other ok. So, now coming to addressing the your queries for today. So, we can see that there are lot of queries and we do not really want to go into inflicting more heavy content on you at this point. So, we will try and address most of the queries. Many of these queries are from the Moodle postings and from the chat sessions over last week as well as today morning ok. Now, I see a lot of thank yous it is not over it simply means that there is still a lot of queries to be addressed. So, please do not go away many of your queries are going to be addressed now and there is really no need to thank us at this point. We will get our thanks when we meet you at T4E where your paper is accepted that is the real thanks that we are looking forward to ok. So, now coming back to this query on domain specific ideas and studies the query says can domain specific topics the significance of these topics and difficulties in learning them be used for my idea. So, the answer is yes you can use it for your idea and, but you need to pay extra attention to positioning of your idea ok. So, see today's first session on what exactly is novelty and what exactly is positioning and also other people may have done this and even after Mirinal's session there was a commentary on why it is important to position it well and how to pay attention to that. So, there were many queries dealing with justifying novelty and positioning. Now, how to decide that our idea is novel what is the criteria. So, the criteria to justify novelty will emerge basically from the gap analysis of prior work it is not that there is some hard and fast criteria to justify that. So, you have to do a careful gap analysis and whatever is the gap that is where you will be able to say that there is novelty. There is another question which says what if there is no prior work for my idea. So, many of you have asked this question and the answer to that is that this is extremely unlikely basically what you need to do is look harder you need to look for related topics and synonyms you need to search more you need to broaden the scope of your topic a little and once you have found some references then narrow it back to recent years or top journals and conferences. So, you need to look in different sources because of the moment we say that you should look in different sources the next automatic question that will arise in anybody's mind is that what are these sources where are they. So, where should you look for these papers there are lot of databases and indexes which are useful for academic publishing. So, some of these are sites here right now you do not really need to note down all these things in a hurry these slides will be posted up all that you need to keep in mind is that we should look in databases. So, some of you may have thought so far that Google is the end all of search that is not true when it comes to research papers Google scholar is more appropriate than simple Google search, but using searching in databases is much better and also you can search in journal or conference web pages what is the next obvious question what if I do not find these papers what if I cannot get access to the paper now this is likely to be a fairly common problem. So, the first thing you want to do is check with your institution's library many institutions are part of what is called the Indist Consortium. This is I do not remember the full form of this some Indian consortium for science and technology which basically subscribes as a group as a country wide group to many large databases and journals and many large repositories. So, if your institution is part of this Indist Consortium automatically you will have access to the contents of the journals which are subscribed for. So, in case your institution is not part of this consortium one thing you could do is you could try to talk to the people and become part of the consortium and if that is looking like it is going to take time or not going to work out what you can do is you can search for the authors web page. So, once you know the author name and the title name of a paper you can find the authors web page and many authors are going to put up their papers on their website. So, you can get the paper from their website directly and if it so happens that the authors have not updated their website it is usually sufficient to send an email to the authors. So, most authors are extremely thrilled when somebody asked them for a copy of their paper. So, they will be very happy to get more readers and get more citations. So, it is extremely unlikely that there will you will encounter authors who will not share their work with you. So, this part is actually implemented by everyone. So, the question of what if I cannot get access is addressed in this way. So, the next question which many people asked was when should I do the literature review. So, for this there was already a video that was posted on model last week. So, that clearly has this particular slide that I have just grabbed one slide from that video. So, there are three times at least when you need to do your literature review one is at the beginning when you start looking for the problem one why are we doing this in the beginning the reason is to see whether there is novelty in the problem itself and once in the middle once your work is under way. So, the reason now here is to see whether there is novelty in your solution and once towards the end when you have your results to see how you stand with respect to other solutions. So, that is the comparison. So, the key point is this is not a standalone step it is an iterative step you have to keep on going back to the literature. The other question that we got was how to read a research paper what is the way to reading a research paper again this slide summarizes it is from this video that is posted on model with the same title of how to read a research paper. So, the main thing to do in reading a research paper is not to start reading the entire paper thoroughly in detail in the beginning. So, if you jump into stage 2 the chances are that you will get bored and you will stop reading the paper. So, the way to get way to read a research paper is to actually do two passes before getting to the details. So, first you need to scan through the paper to get a feel for what is it that these people are working on what kind of methods they have used and so on. And then once you have got the overall picture then you go on to looking at what is the big picture what is the exact problem how have they solved the problem is there novelty in their work where have they explained novelty. So, all of this getting the big picture was what we addressed on Feb second where there were call outs and where you had to circle where are these pieces in a strong paper and only then you go to the details. And one important point to remember here is that just because the paper is published it does not mean that it is valid or it is entirely correct. So, what you want to do is you want to evaluate the details once again for yourself ok. So, this again is something which we saw in the first session on Feb second when we looked at strong paper versus weak paper and then finally, you synthesize the details of the paper ok. So, then this is again another common query which many of you have posted which is the difference between a journal versus a conference. So, it is actually sufficient for you to read this slide instead of my reading it out to you. So, related to this there was a query on can we submit our paper to a conference and a journal ok. So, the answer to that is typically no, but yes only if we fall in this category ok this category where you have done significant extensions of your conference work ok. So, significant means there should be at least 30 percent new work that you are reporting or new results that you are reporting in the paper ok. So, there was also a related query about what happens if the paper is rejected in a conference or a journal can I resubmit it to another conference or journal. So, the answer to that is yes and the catch there is that you need to work on the refree comments that you got from the previous submission from the conference or the journal to improve your paper before you resubmit it otherwise the chances are that you will get the same comments again. So, you are free to revise and resubmit your work either to the same conference or the same journal or different conferences or different journals. You are not allowed to submit the same work once it is published to another forum. There are now so just after I have finished answering this question there is a query which says can we send a research paper to more than one journal first time ok. So, the answer to that is that it is not ethical practice to send your paper to more than one journal the first time itself. The problem being if you are basically wasting the reviewer time of two journals and the problem is that if it so happens that your paper is accepted in both the journals you will be caught in this trap of which one to finally accept. So, it is not correct to submit your work to more than one journal at the same time. So, you can submit it to a journal get back the refree comments and in case it is a reject you can work on it based on the refree comments and then resubmit it to another journal that is acceptable ok. Once again there is a query on is a presented conference paper can it be submitted to a journal. So, I will refer you to this slide look at this point if there is a significant extension then you can submit it to a journal ok. So, now moving on there are many queries which came about ok we saw so many college teachers coming in making presentations here and we would also like to join for a Ph.D. So, how can we go about doing that? So, the answer to that is as follows what to expect if you apply to IIT Bombay is that there will be a written test followed by an interview and the criteria for that are given at this link. For those of you who pass the written test we do expect to see your study planning template. So, we do have your names and when you or if you show up for the interview we will know whether you completed this assignment or not ideally we would like to see your paper, but if not that we should at least be able to see your study planning template ok. What is not good enough is a strong desire to become an ET researcher without any supporting evidence ok. So, many of us get lots of queries on this saying that I am a teacher I am bothered about my students and I want to do some research and can you guide me. So, if something if we get a query at that level there really is nothing we can do about it because you have not even taken the first step. So, you need to go through this process of taking your idea refining it progressively till you reach this study planning template that is what is evidence for us that you are actually serious about this ok. And then there are queries on this very interesting point of whether a PhD in ET will be recognized or not ok. So, the answer to that is I do not know. So, you will have to check with your own university or college whether a PhD in ET will be accepted on power with PhD in your discipline. Some colleges will say if you are in electrical engineering your PhD has to be in electrical engineering for you to qualify for promotion. Some other colleges may say that it is ok to have a PhD in any discipline it is still fine and there is a query which says ok what about AICTE I have no idea and this is something that you need to check on your own. We are not making any guarantees these are the there are some standard disclaimers about doing a PhD in ET. So, you will need to find out what we are doing is we are trying to generate larger and larger number of awareness among college management on the value of doing PhD in ET. So that most colleges should accept it ok. There is a query which says if you pay then most journals accept your paper how can we identify good journals. I think that is a good query how to identify good journals once again there is there are journal ranking. So once you get your study done what you can do is you can just look for journal rankings or we will also post some top few journals of education technology. So that you can pick journals from those rankings. So typically if a journal is giving you a guarantee that it will publish your paper if you pay a certain amount what do you know about that journal. So you know that it is not something that you want to send your paper to they are in the business of generating publications not in the business of publishing research. So most journals will not require you to pay most good journals will require you to pay only for extra pages if they say that ok the page limit is 10 and you want to submit 12 pages they may say that you have to pay for the two extra pages. But most journals will go on to accept your work without any payment ok. There is also another good query which talks about give us a website to check plagiarism. So this is again something which conferences and journals have. So they are called plagiarism detectors most of the websites through which you submit the conference and journal submissions they will automatically do plagiarism detection and I do not know if you need a link to an online software. So you need to check plagiarism only in the assignments that you give to your students ok you do not need to check plagiarism in your own work. So you know whether you have paraphrased properly or not. So I think it is not really required for us to know which are these websites there are a few but then I am not going to go into that now ok. Let us go on to some queries from today's chats. So there are many queries we have actually attempted to shift out some of the good ones. The first one is how many post tests are required to come to some suitable conclusion. So it might be possible that in one test students perform poorly and in the very next test students perform better. So which post test are you going to choose? It is a very good question and the answer actually depends on what is the exact nature of your study. So if it is possible for you to conduct multiple such studies then it is desirable for you to report the results of all these studies and in some way show the average of these post tests. So these can be actually if they are parallel studies with different groups of students then it can be used to show that your idea is generalizable ok. If they are if the post tests are with the same group of students over a period of time once again it can be used to show that the study is longitudinal that whatever they have learnt they are able to retain ok. But then there are other factors that you also need to take care of to ensure that the result in the second post test is not better due to some other reasons. The validity of your test also needs to be checked. If you find that students are performing poorly in one test and a different group of students are performing better in another test then perhaps you should revisit whether your post test is valid or not ok. So that is the answer to this question. Can continuous assessment be one test strategy? In some cases yes. So now these are good questions because they are difficult to answer in one or two lines. So there is no single yes. So if I say yes many of you will think that it is true for all cases which is not so and if I say no many of you will think that it is no which is also not true ok. So the answer is it depends for some studies continuous assessment can be a strategy depends upon what is your research question and what is your setting. So for example let me give you the example of let us say we are conducting a study of this own workshop this workshop that we are doing. So now we want to know is this workshop this way of treatment that we have done this is also research that we are doing the way of treatment that we have done is it effective ok. So we do not have a pre test we did not take any pre test from you because we expected that many of you would not know anything. So and a single post test we know is not a good design. So what we can do is in this study we will be doing some kind of continuous assessment of your assignments and so on to know whether how much improvement is there. So there are cases where it is a good strategy. Moving on to the next question if the pre test itself has some ingredients of the strategy then it will be difficult to attribute the results of the post test to the strategy alone. How do we ensure that pre test does not influence the post test? This is actually a very hard question and often as teachers we also want to ensure that even when we are doing two groups we do sometimes wind up giving very similar treatments to both the groups because you want both the groups to learn. So some of these answers are somewhat lengthy. So we will post them on Moodle about how to guard against these things. Some other questions again these are good questions. So if I am doing the experiment involving students belonging to only one organization how can I say that it is universally valid? So this is the whole point it is not universally valid unless we carry out some generalizability study. So this is again a place where collaboration is very useful that you do the experiment in your college and somebody else does the experiment in a different college and together you can show at least minimal generalizability of the study. This is also another reason why it is valid to take the same domain and do some sort of replication studies because one does not know whether the results will be the same in all the studies. The other question is that what about if one strategy is working good for teaching one topic is it true that it will be so for all the topics? So the answer here is no because the strategy has to be aligned with your research goal or with your teaching learning goal. So for each topic it is not necessary that you will have the same goal for your topic. So if those of you who flash back to Mrenal's presentation she had mentioned that it is necessary for this alignment of the topic and the use of the interactive visualization. So if you choose a topic for which visualization is a good strategy and then you choose a topic for which visualization is not required and you carry out the same study naturally you are not going to find expected results. So this matching between strategy the topic and the research question is very important is another question on is it valid strategy if I select single group and teach different topics T1 to Tn with different treatment for each Ti can I compare learning effectiveness. So again what is happening in this question is that there are too many variables. What you want to do in your study is to show that the treatment that you gave is the cause of the result. So what you want to do is attempt to vary only one variable at a time. You do not want to vary multiple variables simultaneously and then get confused and not be able to establish any causality. So this causality is something that you will encounter this term in your study planning template about how you are going to show causality. So the point here is that it is possible to select single group and teach different topics and all that but this comparison of learning effectiveness 99 percent chance is there that it will not pass the soundness filter. So it will be very difficult to show causality. So the correct way of going about this is to do many small studies each in which only one variable is being changed at a given time. Going to one more set of questions if the strength of the class is 11 for example PG classes what to do with group construction. So here if you want two groups you can still go for a two group study but it is not advisable. So what you can do is modify the type of study that you are doing. So it is not advisable to go for what is called a quantitative analysis technique. So many of the studies that we have talked about are statistical in nature and the statistical studies hold only for larger and larger numbers. So when the strength of the class is 11 statistical studies are not that useful. What you want to do instead so what can you do instead does it mean that you cannot do a study that is also not true what you can do is a qualitative study. So instead of looking at simply exam marks and averages you can interview the students and talk to them about what they learnt and you can look carefully into the answer papers to look for evidence that your technique has actually resulted in the learning. So those are the ways in which you can handle small classes. What exactly is expected in educational theory while writing a paper this is a question which I am sure many of you have but again it is not something that we can explain in two minutes. So we will post some of these answers on Moodle. Some questions on how to validate what are some validation techniques. So once again we will post these but to quickly recap in Madhuri's session in the morning there was a lot of questions about why should I bother or how will I show this. So all the answers that were there in her slides are the validation techniques. So they go by technical terms like content validity, construct validity and so on. But in general what you want to do is at a very top level one validation technique is to show your solution to your peer. Another validation technique is to get a bunch of experts to look at what you have done. A third validation technique is to execute the study for different groups and see whether you get the same results. So there are many such techniques some of which we will post on Moodle. The last question is how to ensure the process of randomization. This is something which you do not have to worry about so much. So once you have two groups you have taken both the groups from your same sample. So there is a difference between randomization of assignment and random sampling. So by random sampling what we mean is anybody in the population has an equal probability of being selected and that is very difficult to ensure. So for most of our studies because we are we have only a captive population of students we do what is called convenient sampling where it is convenient for us to conduct the study with those group of students. So the sampling technique is usually convenient sampling and ensuring the process of randomization is you do not need to be very rigorous about this randomization you can simply divide the class into two groups as long as there is no bias in your division like for example dividing the class all the low performers into one group and all the high performers into another group is not a valid process of randomization. So we will post some techniques about randomization. So for more Q and A so what we can do is we will now go around to the various centres and like the previous time we will transfer the floor to you however there may not be enough time for us to visit all the locations. So this is what we would like you to do please see the slide for clarification questions please ask your peers in your location. If many have the same doubt then ask or post on Moodle. For off topic questions please do not ask these type of questions when we transfer the floor. So what is off topic? Anything that is not directly related to the contents of the bookshop like for example queries on impact factor and H index. So I had said in the previous session itself that this is off topic nevertheless I have seen a lot of these queries scrolling on the chat window so these questions are not going to be addressed and also please remember that there are thousands of participants it is a waste of everyone's time if we transfer the video to you and then we have to take it back because the question is off topic.