 All right, so let's, not wait, we already lost 8 minutes of a 55 minute class. Have all of you been able to view Sāna-Murti's lecture which partly I had played in the class on how to read research papers? Anybody who has not been able to read, just raise your hand. All right. How many of you could quickly peruse the paper which I had uploaded on the model because it was sent very late yesterday evening on? Okay, so most people have not read the paper which is okay because the exercise today is about reading that paper. Now, all of you should have a copy of the paper. This paper was published by Professor Kannan Nautgaly recently based on his experiments on flipped classroom that he has been carrying on for quite a few years. Our ambition today is to apply the approach that Professor Sāna-Murti has outlined in her talk on how to read a research paper and try to implement that approach in practice by using this paper as an example. Now, all of you have viewed that lecture but what I have done is I have captured the gist of activities that need to be undertaken. If you recall, Sāna had suggested a three-stage approach. The first one is to get the big picture that is when you peruse the paper and jot down certain important things. Then you capture the details and then you ask some critical questions. What I have done is I have captured those screen shots from a lecture which outline the activities. So, here is the first one. So, the stage 0 is to get the feel for the paper. So, read the title, see how long the paper is, where is the paper published, look at the figures and read the section, subsection headings. You can do that very quickly by flipping through the paper. So, please do that. This is a one-minute exercise and in the process this is the mini-activity that you will have to perform. So, locate these parts in the paper that has been given to you. The title, abstract, introduction – very easy, straightforward because most of them have headings very specifically. At this stage, please understand that you are not reading the paper. You are just identifying segments of the paper pertaining to different components. So, identify the title, the abstract, the introduction, the background motivation. Where do you see the contribution of the paper? Because the contribution might be spread across multiple sections. Where do you see related work being described? More importantly, where do you see problem definition? Where do you see the scope, assumptions and limitations? Where do you see the solutions approach and then details of solution? Again, where are they mentioned in the paper? You are not reading the details. You are just identifying components. Where are the findings described? Where is the evaluation done? If there is any take-away from the paper, clearly identified, mark that and check where are the references. So, just mark on that paper very quickly. Where do you find this piece of information? Just so that you don't lose sight of the paper on which you are working. On the top of the first page, write your roll number in the copy of the paper so that when you exchange papers, you will know whose paper contains what. Just write the roll number. I will take about 20 seconds on the first page of the paper. Now quickly, peruse the paper. You do very quickly, glance through it. That's an R. What happens is human mind tends to read sentences. The trick is, to save time, very quickly look at just the headings and try to identify from the headings where you think these components exist in the paper. So, for example, the contribution of the paper may be spread across multiple sections. Problem definition would usually be located at one place. Unfortunately, people don't necessarily write problem definition as the section heading. So, you will have to identify where the section is. I presume that you have identified things. Now exchange the paper with your neighbors and let your neighbors comment on A, whether you have successfully circled or identified all sections or not. And you do the same thing for your neighbor. This is to ensure that one's weakness is covered by somebody else's strength. Of course, if both of you have forgotten to mark the common point, then both of you will miss it, will come out in the subsequent discussion. So, we'll have a group discussion at the end, but right now just exchange it with your neighbor. If you are exactly three people in the group, swap the paper across the three. I can see that some people are still identifying things. See, it requires some mental training to quickly identify these components. As I said, we all tend to read the paper even to identify these components. Not correct. You have to train your mind to very quickly look at the headings, peruse the paper very quickly and circle things saying this is this, this is this, this is this. Please note that in the literature survey, you will have to peruse papers like this in the numbers exceeding 50, 60, 70. So, if you have to take half an hour to do this with every paper, you'll lose 20 to 30 hours of your life just perusing papers. Literature survey is still for this. We can't afford that like this. You have to do this quickly. I don't see any discussion happening. People are still busy jotting down their own components on the paper. Learning happens when you apply your mind. Learning happens more forcefully when you apply your mind on your neighbor's work. As you can see, we have spent much more than said two minutes which were required to peruse a paper and much more than one minute in looking at your neighbor's paper. Everybody is still flipping through pages. All right. Those of you who feel that your neighbors have not identified all these components on his or her paper, please raise your hand. Neighbors who are not convinced that their counterparts have found all the sections, please raise your hand. One, two, three, four, five, six. Partially hands are coming up. Why half-hearted hands? I mean, you're not very sure or you're not seen your neighbor's paper yet? Okay. Okay. Now spend another 30 seconds in pointing this out to your neighbor, which part the neighbor has missed and convince the neighbor that the neighbor has actually missed that part. There's something that you should have seen. If not, I'm reminding you. I hope you have followed this process, but if you have not, there's to re-emphasize that to get the picture, you have to read the title, abstract, introduction and conclusion, less than two pages. And then you have to go through section and subsection headings and you have to look at figures if there are any. So that should be able to give you a big picture. I do hope that most of you have done that. If you have not done that, your neighbor will catch you on the laps unless of course the neighbor also has made the same mistake. So now, stage one is further detailing of getting the big picture. So collect back your own paper from your neighbor. Now this is slightly harder work. So look at these details which are written here. What are you looking for? What research area, subtopic, does the paper fall under? It will be there in title and abstract. What problem does the paper attempt to solve? This will be perhaps partly indicated by the title, certainly by the abstract introduction and problem definition if there is one. What is related work and why is it not sufficient? What are the gaps? This should generally be found in introduction. What key contribution does the paper claim? Again title, abstract, introduction and conclusion. And broadly how does the paper solve the problem? So again introduction and figures. How do the authors defend the solution? Introduction and figures. And what category of paper is this? Introduction and headings. This is how you are supposed to get a big picture. You tried that by circling things on your paper and so on. Now you do the following. You take a piece of paper of your own, a notebook and write down answers to these questions. Let it take 3-4 minutes, doesn't matter. It appears to me that all of you who claim to have perused the paper to get the big picture have identified these components, have shared it with your neighbour and yet when I ask you to write down specific short answers to these questions you are again reading the paper. Am I right? This is another thing that we must realize. First time when we do it, our mind must be alert to capture these essence in the mind. You should not have to refer to that paper later. You should be able to recapitulate. Otherwise you will be revisiting the same thing again and again and again, wasteful of time. All of you know how chess is played. You may not be chess players but you know how chess is played. So people have to analyze the position and think about what happens if he moves this and what happens if the opponent moves that and so on. That is how you think. The difference between a mature chess player and a professional chess player is as follows. An immature chess player imagines that if he moves the knight or bishop somewhere then this fellow may do this, fellow may do that, reaches some hazy conclusion, discuss that, starts looking at some other possibility, discuss that, starts looking at some other possibility and after 2 or 3 minutes, revisits the first possibility again. That is because the player has forgotten what was the conclusion drawn while he or she analyzed that first possibility. So an immature mind wavers and keeps revisiting the same alternatives again and again. A professional chess player never ever does that. Once he or she analyzes a particular move and its possible consequences he or she reaches a conclusion in the mind and that conclusion is remembered. That line of thought is never revisited again. That is how you save time. So when you examine 2, 3, 4, 5, 6 different alternatives, when you are examining the sixth alternative you are very clear whether the sixth alternative is better than which one of the earlier five and worse than which one of the earlier. You are very clear. You don't have to revisit the earlier. Why I elaborate this example is this is what normally happens to all of us. In reading research papers in some sense we are immatures. In spite of the fact that we have read a whole of a lot, we are not trained of our mind to read in this fashion. Now this is an exercise in that trend. So just to recapitulate, you glimpse through that paper, you presumably identified these components, shared what you have identified with your neighbor, you never commented on it and yet now when you have to write these things on a separate sheet of paper you are forced to revisit that paper again. You have to glimpse through that paper again. It's okay. Happens with all of us. Now this is where you have to take away the learning from this exercise. It's that first time when you do it, do it to its complete. In a sense, you made a mistake by just circling or identifying things on that paper itself. If you had a piece of paper with you, while you were doing that you could have jotted down these answers right up front and in fact that would have ensured that you don't miss out circling any part on the paper because you will be writing down and circling simultaneously. Does it make sense? Great. Now have you completed identifying these things? So complete that. Another two minutes. I think some people are still doing that mistake of reading the entire paper or reading several sections in details. In the first stage when you are getting the big picture that is not really required you need to read the headings but you need to read abstract introduction and conclusion thoroughly. By the way, I have seen a lot of masters and PhD students getting confused over how to write abstract and how to write conclusions and they often reproduce one as another, just give a different title. They are different objectives for both and the amount of time that needs to be spent on writing your abstract and writing your conclusions is actually more than the amount of time that you spend on writing any one large chapter of your report. Has it occurred to you that it takes that much time to write conclusions or write an abstract? It has to be short, it cannot be the entire thesis. So precy writing is not our normal strength and that is what is required when you write that. If there is a well written introduction and well written conclusion then any reader's task to identify the big picture becomes easier. If the introduction and conclusions are hazily written people will not be able to make out what you are trying to say. In this particular case I think Prof. Kannan has done a reasonably good job so abstract introduction and conclusions has everybody read these three components at least? So let's go to the next stage. This is details how you do it. So when I asked you to write down things on a separate piece of paper you were again writing down from what you had identified against those questions earlier. This is a more comprehensive list of what you should write down. Those of you who have seen Saana Murthy's lecture could have seen this slide. These are all snapshots from Saana's lectures. So the answers to the following questions that you need to write are what research areas of topic does the paper fall under. So let me ask this question loudly. Anybody would like to answer that? Sorry? Educational technology, padagaji, teaching methodology, distributed classes. Why do you say distributed because they are talking about a single classroom here? So I will not agree entirely with the interpretation that is educational technology. There is hardly any technology usage that is indicated except as a supplement to what the main theme is. Padagaji and teaching methodology are more in line with what is being attempted in my opinion. Any other observations? The paper is all about students' learning rather than teaching methodology. So it brings out another important point. Are these two related or not related? So the counter question that I am asking you is is teaching methodology related to students' learning or is it not related? Because if it is not related all teachers in the world are wasting their time. Actually it is true partly. There is a theorem which says students learn not independent of the teachers but sometimes in spite of them. But that is taking it to extent. So as a matter of fact the best teaching methodology is one which maximizes the students' learning and in that sense what our friend said there that the paper is about teaching methodology is correct because you see the whole experiment and the whole idea is planned by the teacher, executed by the teacher where the teacher now wants to find out whether the students benefited or not. So what you are saying and what he is saying are not at loggerheads. They are actually connected concepts. It is about teaching methodology and it is also perhaps because of that it is also about maximizing students' learning. Both should go hand in hand. That is the objective of any teaching method. Why? What problem does the paper attempt to solve? To increase the effectiveness of learning. So the problem is that learning is not effective in normal classes. Now there is a difference between convenience and effectiveness. Things which are very convenient are not necessarily effective. He is bringing another angle. So that is the problem of lack of teachers. But tell me if there is a lack of teachers then how would such an experiment be conducted because such an approach is conducted with teacher moderating the whole thing. So teacher is required. The flip method may use the videos of some other teachers. So you are elaborating on his observation that because there is a lack of teachers the flip method can utilize the expertise of some teachers in giving good lectures and use this approach to enhance the learning. In this particular example or the experiment which is not done once by the way he has been doing it for five years he actually measured the controlled group and non-controlled group for the first time the performance. But the point here is all the lectures which the students viewed were recorded by the same teacher. So it is not really lack of teachers. So let me mention two points in this context. When he said lack of teachers and when he corroborated it both of them implicitly assumed that the main function of a teacher is to give lectures and that is what all of us assume. For when I am a student for me learning means attending classes. Attending classes means listening to a lecture. It generally does not mean anything else. So we presume that teaching means people will give lectures and we presume that teaching means the teachers will set papers and assess us. These are the two important things that we associate with teaching as students. Sadly teachers also associate the same two things with teaching. Oh I have a class that means I have to vomit out something sensibly for one hour give a lecture. Oh I have a mid-save I have to set a paper conduct the exam as teaching that you consider as teaching. In neither of these learning is emphasized anywhere. It is supposed to happen almost by accident or inadvertently. Now that is being questioned for last 40 years and these questions have come to the fore only in the last 5 years when technology is available now to remodel this entire teaching learning process and that is what is happening here. So I will not agree with you that there is a lack of teachers because if there is a lack of teachers in the sense of requiring them to enhance the students learning then there is a generic lag there are more teachers which are required but more teachers who are required to do actual enhancement of teaching learning process and not delivering it. So the observation that I can depend upon some of the best lectures is a correct observation but that is not the main stage the main stage is that teaching learning process should not involve long time lecturing by the teachers teaching learning process should involve activities interaction application of mind and there may be some explanations in it agree? I think what I just described is the motivation because the current teaching learning methods do not necessarily enhance learning like sermons as people say you know this whole classroom model and lecturing comes from the old concept of guru giving a talk to a large assembly or a priest saying things in a church whatever there is one person speaks the model is being questioned why is this paper needed? what is the related work and why is that not sufficient this and such papers are needed because they give substantial and concrete numerical evidence researched over fairly large numbers making the observation statistically effective otherwise many of the policies many of the methods that people follow are opinion based I have I have seen teachers teaching means giving lectures and setting papers so my opinion is that is how teaching is done you have seen teaching being done like this so you believe learning is effective only if the teaching continues like this this is an opinion based thing you also based on solid statistics what is that solid statistics every teacher that is known to you does the same thing and every student known to you does exactly same thing listen to the lectures take notes solve problems it what this paper seeks is not just identifying alternatives this is not a new thing classroom has been identified as an effective major quite some time but it experiments and it experiments very methodically it takes a control group a group which is using the flip classroom another group which is being taught the same subject in a conventional manner and then compares at every stage the learning effectiveness how do you measure learning effectiveness the only known measure is performance of the students in some assessment but there is also a measure of engagement I remember I have mentioned to you the engagement level and there is another research paper by Sridharayari students and Sanna Murthy and how the engagement in a flip classroom goes up this again has been buttressed by several observations across the world what are the what key contribution does the paper claim and broadly how does the paper solve the problem the identification of that experiment conduct of that experiment conduct of various quizzes conduct of various types etc how do the authors defend the solution their solution is flip classroom is a better method now how do they defend it have you been able to identify that section in the paper which is that section which average that's right so it's not the marks alone but the performance same questions are being asked to two different groups but the ability to answer questions by one group is seen to be much better in particular if you notice a very specific that you will come to know when you read the paper in detail that certain questions related to either root locus or some such question which 90% students in the past batches had failed to answer correctly by majority of the students participating in the flip classroom whereas with the other group the performance for this answer to this ability to answer this particular question was that that means it probably buttresses the assumption that certain key concepts this methodology is able to drill into practically every individual's mind not just few students understanding everybody understand the physical requirement of any teaching learning process what category of paper is this so what is meant by category anybody who has written any category for this paper experiment comparative analysis of two different situations it's an experiment comparative analysis it is certainly not a literature survey by the way I disagree with you because literature survey is only a component of a paper now comes the main task so as I mentioned for the literature survey will briefly discuss the literature survey thing in the next class but the idea is that to begin with you would have to collect search and identify and collect some 50, 60, 70, 100 papers could be more could be less sometime which you believe are relevant to the topic that you are working on how to find those papers it's a different thing but when you lay your hands on a paper soft copy, hard copy whatever you have to be able to get the big picture you may not be able to do it in one minute you may require to spend five minutes on it but five to seven minutes you can afford to spend on every paper if you spend less time than that then after sometime you will be revisiting that paper because you are not jotted down in that paper to decide later whether it is useful to read it further in details or not to arrive at that decision you have to jot down a minimum thing that is the reason why I had circulated a sample sheet containing what you must jot down from every paper that you lay your hands on so authors, titles the exact publication space etc and abstract because that often gives you the big picture but you may want to jot down some small notes of your own so that means in your literature survey and this is a practical suggestion because you will be doing literature survey as part of your seminar report do locate fifty, seventy papers and do identify such things as I have outlined with your own additional notes if you want very quickly for each paper so that you don't have to revisit again physically reading it you can just read your short notes and you should be able to figure out whether that paper is relevant to you later or not now assume that out of these fifty or seventy papers that you have perused and you have identified the high level components the big picture now you will how to apply your mind saying okay I am working on this problem and for this problem these five or seven papers or ten papers appear to be most relevant from amongst whatever fifty I have seen there could be only three there could be eleven whatever now comes the major task you have to now revisit those papers so you see already filtered from fifty to seventy papers to just three or five or seven papers and that task you have accomplished seven minutes on each paper now that you have identified these seven papers on each paper you should spend at least one to two hours and that is where you try to get these details so what is the precise research question address this is an introduction and problem definition why is it believed that solution works better than previous so this is the solution approach and figures what are assumptions and scope problem definition solution approach what are the details of proposed solution argument, proof, implementation experiment the gist of the paper so you will have to read the solution the system details experiment, methodology, figures what evidence is provided generally in terms of some results or figures and what is the take away message from this paper that is the overall country but not in the partly indicated in the conclusion but you have to make your own judgment on the whole at this stage three these are the questions you should answer I don't think when you viewed Sahana Murthy's lecture it occurred to you to look at the problem in these stages in this fashion that is the reason why I thought it would be useful to actually give you a paper go through some of the steps and indicate how you should go through other remaining steps now if you look at this stage three this is the criticality with which you will have to read those four, five or seven research papers which are relevant for your work as your mind gets trained believe me in five or ten years time if you keep doing these things you will be able to find the details not by spending two hours but within half an hour by just first reading of the paper itself you will be able to truly peruse the paper or flip through it in one minute capturing all the essential things that's all a question of practice that practice is what needs to be done and I believe that the seminar that you do the literature survey that you will do for your seminar although it will help you identify your MTech project or learn much more about a topic please use this opportunity to also learn the technique of papery and discipline your mind on how to read quickly appreciate what is written in terms of the essential things and absorb after you do that there is a stage three plus this three plus stage is important because that is where you want to further your activity you have read those four or five papers you are working on a problem these five papers are critical in terms of what other people have done for solving that problem but now you have to synthesize your own thing and this actually is not a one hour two hour kind of thing it's not done in a single duration so you have to contemplate you have to think you have to synthesize so look at what points are being made here for helping us to synthesize we have to ask creative questions by the way this is another problem with majority of people in the world and more so with the Indian society we have not been trained to ask questions in fact from our childhood we have been told to keep shut and not ask questions obey, follow so we are like this everybody says this you are doing and therefore our mind needs to be retrained in asking questions look at you you are all educated people smartest definitely because you are sitting in IIT Bombay and not only that you are already dependent on computer science so you look at the gates course like the J.E. ranks I think the gates course terminated some absurdly high point that means all of you have done fantastic things in terms of academic achievement and yet how many questions we ask look at the discussion forum on the moodle it has not been used even once by anyone the only news posts are by Farrakh Firuza or Nagit there is exactly one news post there were seven responses you read those responses and you find that all those responses were irrelevant as far as the subject matter is concerned they were on extremely mutant thing which are irrelevant this is another syndrome of mind which I would like all of you at least to get out of this has to happen with all educated population of the country education is not complete unless we learn and practice to ask questions because when we don't ask questions to others you know what it means we don't ask questions to ourselves also and when you stop asking questions to yourself your curiosity dies and creativity stops if you feel a bit shy to ask questions generically in discussion forum openly at least ask questions to yourself and write them down how many of you have recently written down three questions that occur to you in a week by spending five minutes in that week writing down those questions not a single person you see because doing that is not in the syllabus and there are no marks or grades for that but this is a fundamental human quality of being curious asking questions and look at the firm if you ask questions nobody will shoot you you absolutely stand no danger of being punished for asking questions in fact I am trying to motivate you to ask questions what would you what would motivate you properly I will tell you I am doing two things one is I am asking Piroza to and our TS to set up the discussion forum by putting some pinned topics will announce the opening of the discussion forum formally where every person will be required to post at least one question in every topic over the remainder of the period and the only incentive I can think of is you are passing or failing the course will not depend on attendance as I have already declared but it will depend on the post if questions are not posted then you will fail does that work as a good incentive no he is not happy with it so why you are not happy can you elaborate sorry undue tension my god I am amazed I never thought that asking questions is a matter of tension my dear friend I must meet you privately to ask you many questions I am curious now but believe me I appreciate what you are saying there might be some others who are not as osipherous as you are you are at least honestly saying undue tension so let me elaborate further the tension is not because you have to ask questions the tension is because if you don't ask questions you will fail am I right so passing or failing a course getting a good grade or not getting a good grade is a matter of tension it requires not the competency gain or not gain is that fair to us we are of course here to get good grades and pass courses and so on but if that becomes a matter of paramount importance and results either in tension or non tension but everything else which is more relevant to us as our growth as a human being does not cause any tension it is a very good thing I should be very tense if I am not acquiring new skills and competences so please re-examine your theory of tension alright so please take sorry no no you see when you start asking questions for the sake of asking questions without your knowing so you will suddenly start asking decent questions that's the whole idea because the human mind is still creative let that creativity loose that's all please take the paper and try to go through that for that detail