 So, welcome back. So, by now you must have all done some work on evaluating your own idea about from the perspective of it becoming a strong paper. So, one of the things is while we do this activity I would like the coordinators to take a poll and see how many people have finally evaluated their idea as having potential to go for a paper and in the meantime we can go on with this session of now that you have evaluated your idea you find that you have something to say what is the referee looking for in your paper. So, this is useful for the TAs also here. So, what do referees look for in your paper? It is a little hard for you to see, but let me read out and you will also be able to see this later on in your when you see the slides. The first thing that any referee looks for is novelty. So, what novelty means is that you want to analyze your prior work work done before you buy other people to show that your idea is unique. So, often what happens is we get carried away by our idea and we run away with implementing the idea. So, many of our papers fail at this point itself because we are not able to establish the novelty of the work. So, even though so then we get disappointed because we have done a lot of work and we find that oh nobody is accepting my paper and people start getting disappointed. So, this novelty is the primary criteria that referees look for. The next one is positioning. So, how is positioning different from novelty? It is in the sense that there has to be a more careful analysis in the case of positioning to be able to say how your work actually advances the state of the art. What is meant by state of the art is what is known up till now. So, once you have done these two then comes the question of soundness of the procedure. So, now this often we get this right as long as we follow the specific research methodology a lot of which you will be learning in the next session on the next Saturday. We will go into a lot of details of how to establish soundness of procedure, how to do analysis of data which shows that it supports the claims. But these are the next two things that need to be there in a good paper. The soundness of the procedure and evidence to support that the solution works as you have claimed. And finally, what is required is overall coherence or consistency between the parts of the paper. So, often as referees we have to referee papers for many conferences and journals and what we find is that the paper starts out saying that I am going to do X finally, it does some Y and reports some results for Z. So, that such a paper is a clear reject and the effort put in by the authors in that case goes a waste. So, it is important to pay attention to these points. So, the way we are going to address it is by revisiting the sections that we have already discussed today. So, for your own idea when we talked about is this an acceptable paper features of published papers that is where you will find the hint for looking at whether there is novelty in your idea. Soundness again in the analysis of strong paper where the previous session is where you will find hints for how to analyze your paper for strength and weaknesses. Similarly, the previous two sessions have talked about features of published papers and the analysis of strong and weak paper activity. So, by now we hope that you will be able to go at least a couple of more steps from where you started at the pre-workshop assignment. So, by now you should be able to say that these are some features which must be there in a strong paper. You should be able to identify what is a weak paper and you should be able to also state what all is required in your own idea for it to become a strong paper. So, moving on how do you know if your paper is positioned well. So, the point here is that you want to analyze the related prior work. So, often what we think is that it is sufficient to put a section in the paper called related work. So, many people make that mistake of saying that every paper should have these sections introduction literature survey method results and analysis and they think that I have got these sections. So, therefore my paper is a good paper that is not true it is not enough to simply say that these are the things that are already be done. What is important is to clearly bring out the gaps. So, the first thing is to be able to refer to other papers that have addressed the problem similar to yours. We will see an example shortly and have you referred to other papers that have a solution approach similar to yours. So, this again is a very important thing which referees look for and which often is missed out by you know enthusiastic researchers who get carried away with doing the work and doing the implementation and not keeping in mind that am I reinventing something or am I addressing a gap. The next thing you want to look at is does your solution actually address this gap. So, it is one thing to identify the gap and it is another thing to address the gap and finally you want to see whether your solution is based on application of appropriate educational theory. So, you learn some of these how to do this in the next Saturday session, but at this point you want to keep this in mind that you want to base your solution on some theory and not simply that I felt like doing this. So, I did this. So, papers of the type. So, they are actually called I did this I saw that type of papers. These are typically they invite a referee to reject them. So, here are the examples from the same paper A that you have read in the previous session. So, have you referred to other papers that have addressed the problem similar to yours. So, if you see that these this paper does a very good job of bringing out saying that ok here is there are several simulations have been formulated. First of all they start with saying this is not a new problem many people have worked on this problem and then they also bring out saying what is it that the other people are doing that the other people are doing embedding of video in virtual classroom others are doing circuit capture and simulation system and not just doing a brief description along with that they are also giving a brief description of the results. So, this is an important point to note when you write the literature survey or the related work section of your paper and similarly you also want to talk about papers that have a solution approach similar to yours. So, in this in the case of this paper A it happens to be the same two papers. So, they are what they are doing at this point is identifying the drawbacks of the other two papers. So, they are bringing out clearly that the paper one continues to use lectures to teach theory while paper two has not done this detailed coaching which we have done. So, this is a very crucial part of any paper that you write and often people have this section week which causes referees to hesitate to accept even good quality work. So, finally the paper also brings out clearly that this is the gap. So, there is a gap the efficacy of the simulations compared with that of physical equipment labs is not well understood and this is the lab this is the gap that we are addressing. So, that is the mechanism of bringing out what is the gap in the related work and then at the end of it they also say that the study is reported here investigate the extent to which laboratory simulations add to realistic graphic representations and may replace some physical electronics laboratories. So, they not only identify the gap they also show how the authors address the gap. So, these are the main aspects of positioning your work and the last one which we have not really dealt with in this session, but which is the basis of all the work is it based on some appropriate educational theory. So, now for this you will have to do some reading what we will be doing is putting up some references for you to look at which will give the broad descriptions of many of these theories and the other way to pick up this language of what does this active learning mean or what about these other type of theories is by looking at papers that are already published which are similar to your work. So, in those papers you will find references saying that we have based our work on this theory and that is the time you want to go back look at their references and read up on those theories. So, to recap about positioning your paper the first thing is you want to have good research to write about. So, which basically comes back to saying is your idea worth telling to the rest of the world and then you also want to say have I done my work thoroughly. So, do my claims match the results are the results a significant improvement over known results and are these results verified in multiple ways. So, often this is a very important thing that we find in good papers that they do not simply ask a few students did you like what I did in class and report it as students like what I did in class therefore, my paper is a good paper they do not do that. So, what they do is they not only ask whether students liked it they also go and look at how students have performed in exams they also conduct some other in depth interviews of may be the teaching assistants or the instructors and generate data which show that these results are confirmed from many different ways. So, that is the first thing that you want to keep in mind that have good research to write about and the second point which often again many beginners make a mistake in is they try to tell everything in one shot. So, that is not a very good idea you may have worked for an entire year and done several things, but the important thing is to remember that a conference paper has room for only one good idea and a bunch of supporting sub ideas. So, you need to focus on presenting one key idea what precisely was your research question what exactly is your new result and this sentence is a fairly important one. So, I will give you a moment to look at it. So, it basically reads or for those who cannot see it it reads as use verbs that show results and achievement not just effort and activity. So, any sentences which are of the type we did this we did that and then we did this. So, those type of sentences are typically what you will find in a report they are not the kind of sentences you want to find in a research paper. In a research paper you would have sentences of the type that we did this and we found that. So, the and we found that are the ones which show results and achievement. So, the findings are the central emphasis of a research paper not just your effort and the procedure that you have used. So, this again is a very good slide and that is the reason why I am repeating this point. So, this point is actually been repeated three times since the morning today in the hope that you have if you have missed it the first time and yeah. So, there is some question I will take that in a bit later. So, in the hope that if you have missed the importance of placing your work in context the first time you will get it at least in one of these times. So, the basic point is that this this is from a paper by Mary Shaw on writing good software engineering research papers. This paper is already uploaded on Moodle in the resources directory. So, I urge all of you to read this paper even if you are not from software engineering background. The ideas in that are pretty much generic and applicable to all areas of work. So, let us look at this in a little more detail. So, the example that she is showing is that here is a way of writing about the problem. So, the galloping problem has attracted much attention. So, it is some problem and if you simply state that in your paper that is the worst way of explaining your paper in terms of related work. Equally bad is when you say A and B have worked on my problem slightly better, but still poor is saying that A did this while B did that. What is a good way of addressing is by saying that A's approach to solving this problem had this result while B's approach achieved greater result by doing something else. But B's approach has the limitation that it works only for some specific cases. So, that is the basic point that is being made here and finally, you will be able to get through to the referee very easily if you are able to state this saying that this is what has been done so far and this is the way in which I am extending their work. So, we say that we modified the approach to use some other technique and achieved even better results while getting rid of whatever this restriction was there. So, we have got rid of restrictions in the previous work. We have improved the results of the previous work is something that we want to bring out very clearly. So, this section on explain the relation to other work clearly is a very critical section. So, I think this is also a good time to tell you that this positioning of your paper is so critical that the referee usually makes up his or her mind within the first two pages of your paper. So, usually that is the critical time and the mandate of a referee whether it be a conference referee or a journal paper referee. The referee's mandate is to find a reason not to accept your paper, but to reject your paper. So, that is something you must be very clear about. The referee's job is not to find what is this great thing that you have done. The referee's job is to find out what are the reasons by which I can throw this paper away and that is the chance that you do not want to give to the referee, which is why this positioning, this explaining the relation to the other work and bringing out the soundness of your own work. These two tables that you see as part of these slides, you should keep on revisiting them so that you ensure that you do not fall in these very standard traps of simply describing your work. So, the next thing that we want to talk about is the idea proposal form and let us walk through the assignment. So, while I am waiting for it to upload, if you have any questions, you can type it in the chat window. So, there is one question that I see which says what is the difference between a journal and conference. So, journal and conference at this point, we do not really want to worry about that. So, typically a conference paper addresses one idea and a conference paper has limitations in terms of number of pages. So, usually and a conference paper also looks for greater novelty in the work. On the other hand, a journal paper typically does not have this limitation of number of pages. It is not so much concerned with novelty as much as it is concerned with soundness. So, once you have done the work in a complete fashion, that is when you target a journal. So, usually when you have an idea and if you feel that it is a new idea and you want to put it in front of other people quickly. So, that is the point of the difference between journals and conferences. There is another question which says who will decide the recap. So, the word recap that I had used in the slide actually means that I am recapping. Recap basically means I am summarizing it again for you. So, it is not the author or the referee who is going to decide the recap. It is going to be, it is just a summary on the slide. I mean, there is no section in the paper called recap. Do you mean gap means the unfolded or unexplored portion of the related work? Yes. So, now over as it asked this question has understood the meaning of gap perfectly. So, it means the unfolded or unexplored portion of the related work and the portion which you want to address. So, there are other questions of the type. Can we publish conference paper in journal and all that? Those can wait for a while. I mean, we will take them at the end last session because they are not really dealing with the core content of the papers. So, and questions like what is the impact factor of the journal and all, we are not even going to address. So, if you are going to worry about those factors at the beginning of writing papers, it is going to be very difficult to get anywhere because your focus is on the wrong aspect of the research. So, all those who are worrying about impact factor, please remove that from your mind. First focus on doing your research in a thorough manner. And this assignment which is there on the slide about the idea proposal for T4E paper, this is what you are going to be doing. This is not visible at all and let me do a zoom. So, this is actually uploaded. It is just to quickly walk you through about what is there in the assignment. So, now that the day is coming sort of to an end and you have to go away and do this assignment. Let us just make sure that how you are going to do this. So, many of these questions that are coming up about how do we know that any prior work has been worked on, all of these will be addressed when you do this assignment. So, this if there is no related work yet, what is there to write down in the research paper. So, it is very unlikely that there is no related work. Often students come back to us and say that there is no related work and it takes about 10 minutes to find related work after that. So, it is basically a matter of searching thoroughly. We will tell you about all that. We will upload these ideas on Moodle as to what are the sources that you should look for for looking for related work. So, going on this is the assignment that you are going to do. So, what you have done so far is you have become familiar with different categories of ET papers. You analyze the properties and what you want to do now is to detail out the idea that you worked on. So, so far you have you have done taken your idea and worked on it twice already. So, in the first time what you did was you simply thought about your idea and answered some very brief questions about do you think it is a good idea, how do you know that the idea works and that was your pre-workshop assignment. Then in the previous activity just before the 4 o'clock from the 3.30 to 4 o'clock you took your idea and you evaluated whether the idea has any potential for becoming a strong paper. So, that was the second thing that you did. Now this is the third thing that you are going to do on your own idea. Also remember that you have the flexibility to change your idea even at this point. Even let us say tomorrow you feel that the idea that you have picked up for submitting your pre-workshop assignment is you are not so happy with it and you are able to find a better idea. You are free to submit this workshop the idea planning assignment using your new idea. There is no problem with doing that. So, and then after that in the next session or that is the next Saturday you will further work on this idea and the assignment that we give you after Saturday is also going to be about the same idea. So, the hope is that you will be able to go from saying that I am going to do this in my class to actually coming up with a draft of your paper as the final submission of the workshop. So, this is your third step on the idea. The first thing that you want to do is you want to select the category. Remember that we talked about technology for education technology of education. Now once again here we are talking about are you going to create something are you going to design implement or evaluate instructional material or are you going to simply take existing instructional material and attempt to improve student learning. So, that is the first thing that you are going to do which is basically going to be selection. Then you are going to select the domain which is an easy thing to do which is very straightforward and then the third aspect of the workshop of the assignment is to say what is your idea that you propose to develop into research study. So, this is pretty much the same thing as the previous activity that you just did except that now you are going to do it in much greater detail and address the specific questions like what is the issue related to teaching learning, what will you do during the execution of the idea, what will your students do during the execution of the idea and what is the evidence that you will collect. So, answering these questions will actually help you to come up with a plan for conducting your study. So, otherwise the study is likely to remain as a very ad hoc I did this I found that type of a study. So, having this plan will help you to conduct the study in a very thorough manner. So, going on what you want to do is find and list three papers related to the problem that your ideas attempting to address. So, I have already seen questions which says how do we find these papers we will talk about how to find those papers using the Moodle forum. We will put up a list of it is fairly straightforward there is a list of journals and conferences which are considered to be the top journals and conferences which is what you want to go and search through to see if there are any other papers which are published which are similar to your idea. So, you will spend time finding these three papers. So, this is going to take you some time it is not something that you will be able to do in one hour. So, you will probably have to find about 30 papers and then say that these are the three papers that are related to my problem. So, you may find 30 papers which are vaguely related and then you may find three papers which are actually related to your idea. So, this exercise would probably take you a few hours if you want to do it diligently and once again let me stress that this is an important exercise because this is what will help you to position your work. And then you want to do these A, B, C, D which I am not going to read out right now except that I will just go to the product of what you have to do after reading doing all those activities is that you have to come up with a comparison paper comparison table wherein you say what is done in paper one for what is the problem what is done in paper two and what is done in paper three. And finally you will also say what are you going to say as the idea for my work what am I going to do as the idea for my work. So, the next dimension is what is the instructor doing after that what is the student doing finally what is the educational theory and what are the what is the evidence that are collected. So, educational theories is already stated in the assignment sheet when you read it you will get to know how to look for these educational theories. So, after you do that what you want to do is now you want to describe finally you want to describe the your idea in a very systematic way. So, you are going to say this is the procedure for my study you are going to describe the setup how you plan to do the group assignment how are you going to give the details. So, that your study can be replicated by a colleague and also you will list down what are the properties that you will measure are you going to measure the marks in the exam are you going to measure the student satisfaction and are those metrics matching with your research goals these are the things that you will talk about. And what we will find is we will also post some checklists in the moodle and also in these presentations which tell you what are the referees looking for. So, what you need to do is to just follow these checklists and make sure that you have all these things and finally you can get your peers to review your answer from a strong paper perspective. So, there is a forum which we have created on moodle which is about finding your partner. So, you can find another person who is working on a similar idea you both can become partners it does not have to be from the same institute and then you can review each others and thereby strengthen each others work. So, this is the basic outline of the assignment the deliverable will be this part where you describe your educational study. So, one of these examples I will just quickly show you. So, this is again I am just I know that you cannot read this, but nevertheless the reason for showing it is to give you an example of how the template has to be filled out how the table has to be filled out. So, now this person's idea is on computer based training for improvement of some ability mental rotation ability. We saw this in the morning example this is the same paper that you saw in the morning's example and this is what the person has filled out for each of these dimensions. So, what I am going to do is I am going to read out one of the dimensions. So, what does the instructor do in this paper one which is by this Jianping you they have provided conventional and computer based training and they have used this special visualization test. In the other paper again this person has noted that they are giving computer based mental training for 5 weeks with 1.5 hours weekly and the third paper is going to do give students CBT for 8 weeks with 2 hours weekly and then the person says that in their idea they are going to talk about the duration of the training which they want to say is much less than the original training duration which is mentioned in the other papers. So, here while you describe what is the idea for your paper you can also describe what is the difference that you are going to bring out from between the other 3 papers. So, clearly in this case you find that some of these other papers are talking about training in the matter of weeks whereas, this paper is talking about training in the matter of a few hours. So, similarly the paper also is going to talk about what are the students doing. So, here once again they are saying that they are going to talk about how the students operate in the various papers. So, I am not going to spend too much time on going through these details because it is going to be a little hard for you to absorb everything right now. So, what I will do is just so that you get an idea I have shown you this and you can download this from the activities folder on Moodle. So, that it becomes an example for you to look at while you do your assignment. So, let me also show you one more of a different category. So, here is another example. So, there are 2 examples that we are putting up of what you have to do in your next assignment. So, here again if you talk about let us look at the same row of what does the instructor do. So, in one case they are saying instructor is showing a video tape lecture in the other case they are saying that the instructor is going to create assignment tasks based on animation and in the third case they are saying that the instructor is going to give a lecture in class and then distribute the lecture notes along with the video. And what is this person's idea? This person's idea is that they are going to have 2 groups. So, this is what is called a control group experiment and for one of the groups the instructor is going to teach it on the blackboard and then show some animation. Whereas for the other group the instructor is going to give a short introduction followed by animation and then is also going to give some assignment sheets. So, as you can see that this table is very useful for you to clearly bring out what is the difference of your idea from existing ideas. So, this is your idea and you are clearly bringing out that my idea is different from this because this guy is only going to do a lecture whereas I am going to do a lecture and then I am going to show the animation and in the other group I am going to not only do a lecture but also show the animation and give them assignment sheets and that is addressing this problem of whether student learning is more effective when they watch assignment sheets, animations with assignment sheets rather than just watch the animations. So, that is the example of how these assignment 2 or this assignment next week's assignment at the table that you have to fill out. So, let me go back to the, so this is the same set of slides that we have already seen just quickly going to the point. So, to summarize these are the four homeworks that you have to do we are coming to the close of our today's session. So, what you want to do is go through all the slides posted for today's session because the slides have lot of pointers to what you want to look for and then there are two videos that we will be posting from a previous session like this one of which will be on how to read a paper. So, you have already carried out this activity to some extent when you did your analysis of the paper but still you could watch that video and the other video will actually tell you how to do a literature survey. So, many of these questions about where to find the paper, how to find the related work all of that will be addressed in this literature survey. Then you will find three papers related to your idea. So, this is probably going to take you some time. So, what I would suggest is you block a couple of hours every day to work on this assignment and that is the only way in which you can get to submitting it in a you know satisfactory form. Like I said you should remember that you want to challenge yourself to go beyond the obvious. So, you want to get to the stage where the referee is able to immediately spot that your paper contains a worthwhile idea and you have done a thorough job of its evaluation. And then finally you fill out the idea proposal form. This is the form that we will be looking at for orienting the next session and our TAs will be available for guiding you through these forms. So, whatever queries you have I think there are two forums which are created on Moodle. One for each of the two categories that you are going to be working on and we have the TAs whom you saw in the morning they will be able to do it. So, there is an important query here which says is it an individual assignment or we have to do it in a group. So, it is an individual assignment the only way where it could be done is a group is if both of you have exactly the same idea. In which case both of you can collaborate and one of you can try the experiment in your class the other of you can try the experiment in another class and then you can together write a paper. But then you must remember that each of you still needs to do all these activities in order to learn something out of this workshop. So, moving on so what is next? So, in the next hour or 45 minutes or so we will do some reflection on what you have learnt today. I think we have continuously repeating this. So, this process that we are following is called a spiral methodology. So, now that you are familiar with some of these education technology terms the two theories that we are using are one is called spiral curriculum in which the same idea we keep revisiting again and again at greater level of depth as we go along. So, we will be revisiting these ideas again in the next week, but at much greater depth. So, the idea is so that you get slowly used to these concepts and these thoughts instead of bombarding you with them all at one shot at a very great level of detail. So, this is spiral curriculum and the second idea that we have implemented from theory for this workshop is called active learning where instead of us doing all the work and you sitting there and listening you are doing all the work and we are sitting here and occasionally summarizing. So, these are the two well known theories for which are known to be very effective for learning and those are the ones that we are implementing in this workshop. So, by next week you will complete the idea proposal and think of how to apply what you learn today to go from being an E.T. practitioner to becoming an E.T. researcher. So, next Saturday we will talk about how to set up the study. So, by that time you would have thought about saying that I need to do a study on something and you may be wondering how to go about that study. So, that is why we are we will set up that research we will talk about how to set up research study in that next week and the methods to evaluate our solution and as expected this will not be simply telling sessions where we come here in lecture you will do activities in order to get a grip on these methods. So, now what I will do is I will quickly scroll through the chat window and find a few questions which are recurring. So, one of the questions that is recurring is how to say that the idea is novel is there any basis related to the domain topics. So, this is the hardest problem and what it is really there is no clear solution for that other than exhaustively surveying the literature and identifying. So, one thing you can assume is if it is the first idea that it that has struck you you can more or less be sure that it has struck other people also. So, you can more or less be sure that it is a fairly obvious idea. So, make yourself go deeper than that. So, say that assume that this somebody else has thought about and then what is more that you can do on that then you are likely to get upon hit upon deeper ideas which are beyond the obvious. So, lot of questions are dealing with novelty which are the factors that make novelty all of these we have talked about. If you go back to look at the slides you will find clear definitions of where the novelty is being brought about. So, especially this slide where you are saying explain the relation to other work clearly. So, here is where the novelty of the work is being brought about. So, where you say that x, y, z did a, b, c and I am doing it differently in this way. What I will do is I am only going to take those queries which are worth addressing in a face to face session because we do not have much time. Many of these can be addressed in Moodle. Is the research paper, can the research paper be domain specific? Now, this is a good question. Yes, your research paper can be domain specific. So, if you are working in let us say mechanical engineering on a particular subject you will definitely need to include some details of the concepts involved in that subject and how you applied your research. However, you must note that simply describing that this was a topic and this is the way I taught it is not going to cut it. What you also need to do is to keep on focusing on what is the novelty of your work. Can we make the idea that is similar to previous work, but having the idea for different domain? Now, this again is a good question. The answer is yes. You can make the idea that is similar to previous work, but you applied in different domain. The difficulty here is that you have to clearly say that this is an idea which is there and you have to bring out the reason for why you are applying it in a different. So, what is the novelty that is going to be brought out by your applying in a domain? There is brilliant person asking you should I study. So, this is something that we have already talked about. If you want to write a paper then you do this study. If you do not want to write a paper then you can simply ignore it.