 So, good morning once again. So, as you know, Dr. Sameer Sahasrabudde is going to engage with you for several sessions from next session onwards. Talking to you about visual communication, particularly about the skills in setting up your graphics and also how to organize your concepts so that you can present them properly. But because the main theme of this entire course is improving our ability to articulate ourselves properly while composing technical literature, if you recall we have taken a single task for the entire course as the most important task and that is to prepare a report, technical report on your own. And again to reiterate, this report will primarily be based on your own interests in research pertaining to the seminars that you have registered. Preparing a seminar report of the best quality and presenting it to the best of your abilities is the target of this course. Now to begin with, any research begins with understanding how other people have looked at the problem that you are trying to solve and that is mostly done through a process called literature survey. All of you are familiar with literature survey. So anybody who has conducted a literature survey earlier before coming to IIT, how come when I said are you familiar with literature survey, everybody sort of nodded, but when I said have you conducted a literature survey ever, only one hand partially was raised. So what is it? Let us speak loudly. Have you conducted a literature survey earlier before coming to IIT? One yes and few yes, alright. Have you heard the term literature survey? Can you guess what literature survey could mean? Surveying an existing topic from what point? Because a topic has multiple facets. So surveying a topic from the point of view of technical literature that has been written to solve problems associated with that topic. You do not read technical literature as a storybook. You read it to enhance your own knowledge and understanding of what other people have done about that topic. Once you define a problem, you have to actually describe some aspects of that problem to yourself. What are you going to solve? What problem are you solving? And then you start searching for literature or papers presented typically in conferences or journals to find out which of those papers, and there are millions of papers for various topics, which of those papers pertain to a, the topic that you are working on, the problem that you are solving and subsequently having collected 50, 60, 100 such references, you then say, alright, out of these 100 references, we seem to be generically indicating some work related to this problem. The specific problem that I am interested in is perhaps dealt with by only these 5 or 7 papers. Then you read those 5 or 7 papers more carefully and present your finding and understanding in the form of a report. That is essentially a literature survey. So you survey the available literature. Please note that you might miss out on some important contribution made somewhere just because you have not searched well enough. So the first task in the literature survey is to search the world globally. In old days, traditionally, the only access to information was through journals and conference proceedings. I remember in my intake time as well as subsequently PhD time, we used to go to library and jot down relevant references. Any time a new journal issue comes in or a new proceedings come in, you will go there IEEE or computer society, whatever, whatever, ACM, and you will scan through them. This is one paper which could be useful. That is one paper which could be useful. How do you organize that information? Do you just remember in IEEE volume published in last February 2015, I saw that paper. For two days, you may remember that. Two days later, I saw some paper. Ten days later, I think I saw something. Don't remember. So do you write down? Forget the literature survey. Whenever you read a book or whenever you listen to a lecture, do you take notes? You do. Note taking itself is an art by the way. It's an art of communication. So listening to a lecture and taking notes is an important point. I believe that is being covered or has been covered in the issued course. But coming back to our literature survey, one of the problems that is constantly faced by all our M.Tech and PhD students is that whenever they submit a project report, whether it is an MTP or seminar report or first stage of annual PhD research or whatever, they will invariably have a list of references. Everybody has read at least one research paper in life or not yet? Anybody who has not read any research paper from any journal or conference? Oh, not a single Brave Hulk. Anyway. So all of you are familiar with the way the papers are written, right? At the end of each paper, you will find citations, references which are cited in the text. You are familiar with that also? How is a reference written? So in a seminar report or an MTP or a PID, thesis or a progress report, invariably at the end of the report, you will have references. How are you expected to write them? There is a standardization. Forget rest of the world. Department of Computer Science and Engineering issues a standardization. How many of you are familiar with that standard? None of you. So that means you go to the department wiki just to see a particular notice relevant to you, saying tomorrow is a holiday or some such thing. Please explore the department wiki. Please explore the material that is available in your own department, on your own department website and find out how the references are read. Why I am emphasizing this point is that people in their list of references typically will write the names of the authors. Often a single name, initials missing. They will write the title, which will always be approximately accurate, not exact. They will occasionally write the journal in which it was published or the conference in which it was present. They will invariably forget to give the reference of the year in which it was published, the volume in which it was published, the page numbers which was published and if there is an ISBN number. Are you familiar with ISBN numbers associated with books and proceedings? Do you really go to physical library ever or don't? You do. So you know ISBN. So can you explain what is ISBN? So what she is saying is that every book is assigned a unique number. It is not an arbitrary sequential number. It's actually a coded number. But those of you who do not know about it should find out what is ISBN, what is that ISBN number and so on. Please note that the method and the discipline of writing percolates to proper organization of information as well. And book is probably the most organized form of presentation. We'll talk more about book writing and book reading. But the second problem is even if your reference is correct, you will invariably rush to library or rush to your resource at the end of writing your report saying, what is the exact reference? Because in the first instance, when you visited that paper, you forgot to jot down the important things. Now I'll tell you the discipline that we were taught when there were no computers. We're all used to be given index cards, small postcard size thing with the lines. And our gurus will tell us that when you go to the library, you see a good paper. You glance through it all right. But the minimum information that you must jot down on this index card is the title of the paper, the journal or conference in which it appeared, the names of the authors and the abstract. Why? Because abstract is supposed to give you the gist of that paper. When you first time look at it, you don't even know whether that paper is going to be useful to you or not. You don't know. So you write down the abstract topic. Today, when you get access to such papers on net, you invariably will search through glance on the screen and forget about it. You might jot down just the name of the author or something like that. That is the beginning of a huge nuisance for you because you will end up spending time later in compiling the same information again and again. And if you don't, you will get at least one downgrade in your grade because your references are not properly written. You will get another downgrade in your grade because the reference that you are citing in your text and the sentences accompanying that citation do not indicate any connection with what was discussed in that paper because you have forgotten. So first principle, whenever you look at a paper, you must actually prepare a growing document like this. So for example, roll number 051013 Deepak Phatak, topic teaching and learning programming. That's the problem I am looking at. So for teaching and learning programming, I write list of references and abstracts. This is a soft copy I am preparing. I have not written a single line here except the top, my name, etc. All others are extracted from the search that I made on the web, but notice the way I have jotted it down. First, A.C. Latinin, Christy Almukta, Hanumati Jhervinan, these are the authors. A study of the difficulties of novice programmers. Proceedings of ITIC is 05. Proceedings of the 10th annual 6CAC Conference of Innovation and Technology in Compressorized Education. Pages 14 to 80, ACM, New York, NY, USA. Copyright 2005, ISB and this. Many of you might be seeing all these details carefully for the first time. But these are the minimum details that must be there in every reference that you cite anywhere. If you don't do that, it's not a question of losing grade. You will be considered sloppy. If a paper that you present which contains your brilliant technical work and if that paper is presented with such sloppy references, people will simply ignore that try-down. If you want to make an impact, you have to be perfect. And this is the perfection, first step. So is that clear? That means when you start the literature survey, you'll be scanning, as I said, 50, 70, 100, 30 papers across the globe. You know how to search papers? How do you search the literature? Digital. One way, of course, traditionally go to library and look at the reference. But how do you search them? Google search. And Google scholar. Google scholar, those of you who do not know this, please write down the name. Google scholar and Google search. Google search, of course, everybody knows. In fact, there is a verb called Google the date, which means search the name. And Google scholar will give you a lot of them. And there are digital assets. Our library subscribes to a large number of online journals and conference proceedings. I do not know whether you are aware of that or not. But for your own sake, please find out which are the journals, which are the conferences for which IIT Bombay Library digitally subscribes to. Find out how do you get an authenticated user ID for use? It's a shared usage. But you can find out this. Now what is most important is when you search something, whether it's a physical journal, in which case you methodically type down the name abstract and all these details. If it's a digital journal, you extract that information like. So this is just a sample sheet I have put up on the net. By the way, that reminds me, how many of you actually read the two documents which I had sent a mail to you to read before coming to the class? One, very sporadic. You're making a mockery of the flipped classroom. The whole concept hinges on the fact that people will actually study some basic material at home so that we can have more lively discussions. From the blank looks on your faces, I could figure out that most of you have not read this. Please don't do that next time. Proceedings. Whatever is the proceedings are the reference paper. You see pages 32 to 36 are pages of that publication for which the ISBN number is given, or for which the ACMC SC bulletin archive, volume 39. So this is actually a reference matter. Means there exists some printed copy somewhere guarantee. It brings me to another question. You might find several references which are actually give a very detailed account of some solution for your problem but they're not published in a proceeding or something. This is a web paper. How would you refer to that? Any idea? Yeah, sorry. URL and excess date. URL so that other people can refer to URL. Why is excess date important? Content might change. You see that's a fundamental difference between a printed material and a web material. A printed material does not change. A new edition may subsequently come but once printed it remains permanent imprint. If I have a web page in which I publish a good paper today and then I suddenly say, oh, I have actually written something wrong. It's my web page so I change it. Now you go and cite that reference with the original writing and the person who is examining you say, but this is not what Fatak said. He said, no, no, sir, this is what Fatak said. Now the teacher goes and looks at the Fatak's page, finds out it's completely different than what you have done. So the excess date is important but is it adequate? Supposing he says, no, no, no, I have excess Fatak website in December 2015. Now the my examiner or your examiner will say, how do I know that this was the content in 2015? Because what I see is this. Let us go ask Fatak what was the published material. Can you do that? No. Therefore, every paper that you cite from the web, not only you must give excess don, but you must download that paper and keep it with you as a soft asset. Must. No choice. If you don't do that, you may even fail a course. You may fail a seminar for wrongly representing, for no fault of yours. Am I adequately being emphatic? No choice in that. Either you give a proper reference to a printed or a published thing. If it's a web page, not only say excess on this date, but also download that copy and keep it with the file name of your choice, but the date on which the file was created must have the date on which you claim you have downloaded. You can't say excess on this date and two months later download it. Then there is a mismatch between your citation and the file creation. Of course people can do fancy stuff by changing the date and time of the computer and then creating something etc. But we believe we are all honest and decent people. We won't do that. So this is the first task. This list would be a growing list. And this initial survey which is almost a random head, random search, all that you are looking for certain keywords which pertain to your problem area. And it is not uncommon to create the first list of 50-60 references. By the way it would take roughly two days and two nights of searching to just get the first cut list of 50 or 60. Now do you read all 60 papers seriously? No, you can't unless you are doing a very detailed survey kind of paper itself. But you generally glance through those papers. So how do you read a research paper? Now some of you have read the paper which I had put up on the web. Not very many. Some. So how much time did you spend on reading that paper? It was an eight or nine page paper. Thirty minutes. That's a very long time. Anybody else who has read that? So you listen to Professor Sahana Murthy telling us on how to read a research paper. That's an art by the way. Since most of you are silent and you have no clue on how to properly read a research paper, I think it is worthwhile to look at what she says. Another session on how to read a research paper. By now you might have undergone several sessions in which you may have heard of the scientific process, the various... I will be uploading this and another lecture by her. It's unfortunately a full one-hour lecture. So you might have to listen to both these lectures in succession. I'll be putting both these on the web so you can download and watch yourself. But I'll be playing some parts of these contents because they are extremely relevant, particularly of the stages in which you ought to read any one research paper. The point is, if you spend, say, thirty-four, forty minutes, which is a good thing that you did, but imagine this, you have collected at least a hundred papers. Now, each paper you spend forty to fifty minutes to understand that paper completely and then realize that this paper does not add any value to the problem that you are solving. You have wasted fifty minutes of your time with some useless knowledge added to your memory. You can't afford to do that with hundred papers, fourteen to hundred minutes. How much is that? Four thousand minutes. How many hours? And you have other nice and interesting things to do in life than literature, sir. You can't afford to do that. You can't afford to do that for five or six papers. But which five? So that is how you do the staging. Let us look at this. She is Professor Sanamurthy, by the way, a faculty member in our intensive program on educational technology. She works very closely with Professor Seetha Raiyer of her department and has done some extraordinary research and finding which has helped us propagate across the country the use of flip classroom and active learning. Parts of the research process, the scientific method, the skills required while doing learning or science background, I agree, but the kind of guidelines we are talking about are not specific to any one discipline. They are mostly reasonable. Just to give you the context, there was a workshop that we conducted when we trained ten thousand teachers across the country for the first time. The title of the workshop was Research Methodology. This was coordinated by Professor Karmal Kaur of IIT Madras, whom we had specially called in. Professor Sachin from Chemical Engineering, a lot of people. Sunoj was there, several people, very good people. In that, as a part of that workshop, we are trying to tell the participating teachers because you see, our teachers in most of the institutions come from our own ill. Several of them have not even done their MTA, very few have done PID. So they have not been initiated into research and therefore this was organized and that is the reason why it appears generally. It does not say computer science because the lecture was not intended for computer scientists alone, but an important. So these are the guidelines that she is going to talk about. It will be applicable to computer science, electrical engineering, as opposed to read as a pre-homework to the session. It is called design and deployment of flickers in distance education. I am assuming that you have read it because a lot of examples that we will be doing today are based on that paper. And it would be good if you have a printout of the paper. No, I did not bring a printout because I presumed that you would have read that paper. But given that only two or three people or four people have read that paper, this whole exercise will flow tangentially to your head. I will still do it very quickly, but please understand that when you view this lecture, either keep a printout on your hand or keep that paper open in another window and in one window you play this video. Then you will be able to understand and correlate, but let me just tell you the basic steps. In front of you, maybe one for every page is to get the big picture on that paper. And it would be good if you have a printout of the paper in front of you, maybe one for every two or three participants. And if you do not, perhaps the coordinators can help at this point. When she says coordinators, whom is she referring to any idea? Remember, she is talking to 10,000 teachers across the country. These 10,000 teachers don't assemble at a single place. The mechanism that I have set up is we have more than 300 remote centers. In this particular workshop, about 200 of them participated. At each remote center, 30-40 teachers would assemble. And those 30-40 teachers, participating teachers, their activities will be coordinated by one workshop coordinator. She is referring to a coordinate. So like we have a classroom here and let's say Sana Murthy speaking from IIT Kharagpur. So I will be your coordinator asking you to do the same thing. There is almost a kind of a group discussion or a team discussion facilitated by a coordinator. But look at the three-plus stage approach for reading a research page. Please read this carefully. I will quickly play the portion that she talks about. Stage zero, get a feel. Stage one, get the big picture. Stage two, get the details. Stage three, evaluate the details. And you reach stage three plus only if you are convinced that that paper is actually relevant to what you are doing. But anyway, let's quickly go through this process so that you understand. So let's actually look at the approach of how to read a research paper. And we have called it the three-plus stage approach here. Now there is a lot of references and a lot of people have given good advice on how to read a research paper. So what we will talk about today is really, it has been borrowed from some things that people have already said. I'll tell you sometime later towards the end of this talk on where to find some of these good references. And most of them talk about the approach to reading a research paper in a process which looks like what you see on this slide here. The zero at stage, so this is really before you actually begin the first stage, is what I call get a feel for the paper. And each of these boxes we are going to expand in the next few minutes. Feel is something very quick. It's literally a feel. You may just pick up a paper like this and you may flip through it. First important stage is to get the big picture. Then one gets the details from the paper, one evaluates the details and to get a good idea of the paper one has to at least go through stage three. And if you are a research student or a researcher who is using the paper for your current work, you often have to go to stage three plus where you have to synthesize the details. Okay, so let's get a look at what you mean by get a feel for the paper and how to do it. By the way, each stage will require one reading of the paper. So when you read a research paper, it's a multi, you'll have to read it multiple times. In fact, that's one of the questions we explore later. So how do you get a feel? First thing is read the simplest thing, the largest thing on the paper and the largest font is usually the title. So read the title. Ponder upon it if you'd like for few seconds. And then as I mentioned earlier, pick up the paper and see how long it is. And papers vary in length a lot. The variation is really huge. It can be as small as four papers and sometimes you see reflection papers as small as two pages. And sometimes they might go 30 or 40 pages. So you need to get an idea of how large the paper is. And typically, the length of the paper also depends on the type or where it's been published. For example, conference papers for technical research conferences are usually between four and eight pages. Journal articles are longer and these can be small, like five or six pages, can actually even be four, but it can go up to 15 or 20 pages, typically. Review or survey papers are usually much longer. If you can get a review or survey paper on your topic, then that's a goldmine, because somebody has done exactly what you are attempting to do, a literature survey. But the entire paper is only about literature. So I'm not going to add any new value, but I'm for the benefit of all co-researchers in the world, I'm actually compiling information about 50, 100 research papers. Each one of them is methodically studied to stage three plus and then consolidated. I'm just presenting to you on a platter what you would take months to research or search, right? We are not fortunate to get this because it is unfortunately not considered the most important contribution that a researcher can make, and therefore very few researchers venture into writing such papers. Only when you have made a name for yourself and you are established as a dada in the field, then you can afford the luxury of doing such survey papers etc, but not till. Okay, the next thing you need to know when you are just getting a feel for the paper is where is this paper published? The bibliographic details as we call it. And one needs to know how to find this information and where to find this information. So bibliographic details, where was the paper published? Remember what I said, page number, proceedings, year, all of you invariably treat this information as dirt, useless, and all of you suffer the consequences. I have seen seminar presentations after seminar presentations where the examiner's team is berating the student not for lack of talent reflected in the work, but because of the useless manner in which the references have been compiled. And in fact, every examiner these days looks at your introduction, abstract, conclusion, and reference. And if the references are sloppily written, the examiner mentally decides to give you either a fail grade or just a pass grade, independent of the nature of you. That is the importance of organizing. So I cannot overstate that case. And in spite of my stating this every year, again and again I find that the references which are compiled and presented in your research papers are very shortly written. Please don't ever do that. Typically what happens is it's available, if you have downloaded the paper from the journal website or if you have photocopied it from a specific journal, you should look at the header or the footer of the page and you will find that information. If you have downloaded it from a website, from let's say somebody's personal page, see if they give bibliographic details. And what we mean by these details are the year, the title, the journal, where it's been published and so on. So before reading the research paper in details, if you follow my style of compiling the base material, you would have automatically done this. So getting the feel for a paper is what you will actually do for every paper that you do for hundreds of papers before going through the details of any paper. That is the suggested approach. And the feel you get not only by flipping through the pages and reading it, but even on a web, you do exactly it. But what I am adding is while you do that, independent of whether that paper is going to be useful to you or not, please extract this material. You just have to copy paste on a document and keep it. It doesn't take too long. You might be helping yourself, most important. If not, you might be helping others. Because what you are creating is actually some kind of a collection of survey papers. At least the names and titles and abstracts. Look at the figures. Just glance at them, see what you think and read the headings. So when we say get a feel for the paper, this is all you do. So look at the figures just like a storybook, graphs, figures. What do they make something? And read the section and subsection headings. If you read the section and subsection headings along with the title, you will get a feel for that paper. Do you agree? It's common sense, right? But we don't apply. Typically we start reading a paper completely. We should not do that actually. And this can take as little as one minute sometimes. We'll come to how long should I spend on a research paper. We'll come to this question little later. But this is really getting a quick feel. In fact, you'd be able to get a feel of a paper much quicker than it took me to go through this slide. So we will have to look at before we go on to getting the details of a paper is that the scientific research paper is actually a very peculiar piece of writing. It's not a story, but it's not a story. Let me digress here a bit and tell you about something else which is very relevant. If you look at what she has written, it's highly structured, almost predictable headings. Every item in paper exists for a reason, nothing merely for cosmetic reason. That is the difference between a story and a research paper. And each part connected with other parts, sentence, section one, sequence is important, and figure and text correlation. All this makes sense. In fact, books are written like that, chapters, etc. Now I pose a completely different problem related to how human beings understand world and anything in the world, and how human beings make a model in their mind of whatever knowledge they have learned. Anybody has read about this? How many of you are familiar with the term concept maps? Anybody has heard this? No. It's a new wine in the old bottle. It actually represents the way most human beings think and learn. So let us go back in time to our childhood. When we are growing up, we are not learning research or anything, we are not even going to school, but we are learning about the world. What are the typical things that we learn? Let's say we are admitted to kindergarten or first standard or something like that, just starting to go to school. First we learn a language. Through the language we try to identify things, okay, fixing the nomenclature in our mind and associate it with that nomenclature in number of other first sets which come to us. As a child, what would I say? I'll say mother is a concept. Father is another concept. Sibling could be third concept. Food is another concept. Okay. Now as a child, when you think of food, what do you associate the food with? Father, sibling, always mother. Okay, fight. When you grow up, you start fighting with your father and mother, but that is much later. You will find that there are umpteen such things and there are cross-correlated. You suddenly discover that while your mother takes care of food, father takes care of your study, but if you misbehave, both join hands together and bash you. So there is a relationship between. This is how we build. This is called a concept map. Now without going into further details, you can easily imagine that it is natural. Human brain is nothing but a neural net and therefore it is natural that it should be repaid. We have no way of figuring out how the brain does it. As somebody said, brain mapping, you cannot measure the neurons flowing through the connections to figure out how the brain thinks. That's a different thing altogether. But the fact of life is that the brain maps these concepts and the network in their mind. That is how we all learn. All of us will always have concepts and relations. Why I am saying this is now look at not just the research paper, but any book or any write-up that you do. Any write-up is essentially sequential, right? So you have paragraph, paragraph, paragraph, paragraph, paragraph. You had the most of chapter, paragraph, paragraph, paragraph. Of course when you want to represent, let us say multiple things related to a topic, you may have separate sections. These sections must appear sequentially because that is the nature of any writing, any script is sequential. However, you may be able to represent the interrelationship between concepts such by saying that this is chapter, this is section one, this is section two. But section three may be related to section two. If they are not necessarily independent sections, they may be sequenced also. Within section there will be paragraph. In short, any writing that is done is essentially sequential and at best it can represent a tree. Agreed? At best it can represent a tree. There is no way some section here can refer to this. In fact it is done which is called cross-referencing. But in a printed text when you go to page 56 and it says look up page 25. You have to physically go to page 25 and then where on page 25? Third life, fifth life? You'll get confused. Please understand therefore the importance, criticality and difficulty of both writing a book or a paper and reading and understanding. If I am writing a paper, I am actually trying to represent this concept map into a sequential structure. When you are reading a paper, you are trying to recreate a concept map in your own mind. I hope that that concept map is similar to this concept map which I have in mind. Will I always succeed? Not necessary. That is the reason why writing a good book is extremely difficult and only recently books have started coming with a concept map of the book. You may think concept map is very difficult to draw. Dr. Sastra Kutney is going to actually discuss this because it affinity maps and so on later on. But here is a concrete story. Umpteen cases, umpteen examples where children were told a story and they were asked to prepare a concept map. Children who can't even write properly. But you will be amazed to know that most children came up with almost similar concept maps. That is because when they heard a story which was interesting to them, they could correlate to their own mental image of the world and could say this, this, this relates to this. They might write poor English, poor Hindi, whatever, but they could do that. Have we lost the ability to do that? We don't know, we have never tested it. None of you would have ever drawn concept map for anything. So I stopped this digression here, but it is extremely important that you understand the notion that the model of the knowledge in your mind, whether it is a research topic or problems or whatever, whatever, is always in the form of some network. And its written articulation has to be necessarily sequential or at most esoteric. The ability to reconstruct the original concept map. In fact, I will not be surprised in about 10 years time if every research paper is not required or mandated to be accompanied by a concept map. So that people who can look at that concept map and then correlate different paragraphs of that research paper to each other so that they form a better model. This is a very important part of communication. The authors are trying to communicate something. They convert this into this, which you read, and you try to create something in your mind. How are you sure that that something is this same something? That is how there are misconceptions, that is how people interpret things differently because their model is alcohol. We have exhausted this class here, but I would like to remind you that this lecture, by the way, does not talk about concept maps. But this lecture talks about reading a paper. Let me just go through some sample thing. It will look a little hard to be able to locate it towards the end of the introduction or somewhere screen. But what I want you to do is look at the colours. I think you should be able to see at least that. Title, I am sure all of you found. It is what is being shown in yellow. Abstract. In fact, here it says it is the abstract. It is usually the very first paragraph. It is either there in different font or it will go back and forth. So she actually suggests that you circle the paragraph saying, this is the point this paragraph is talking about. Please note that you are actually trying to write down the blobs of a concept map effective. Although she never mentions that because that time concept maps were not to the fore of our own understanding. But you identify these important parts of that particular paper. So this is the after getting the feel. This is the next thing that you... Let me go directly to the section where she talks about timings. But you should also be able to find related work in the section on background on page one. So basically she is talking about different components, background, details, technicalities, etc. You are supposed to identify them in the paper. Again, not a very serious reading. But this is important that you identify components of that paper saying this is what that paper... What this really tells us is that what you as the reader of the paper have... So getting the big picture she has given a sort of suggestion that what research area subtopic that is title to abstract. What problem does the paper attempt to solve? What the related work is? What key contributions broadly? How does the paper solve the problem? These are the components of a research. I like this breakup because this is exactly how you should be presenting a paper or a seminar report of yours. Something very similar. Who has to identify. So it's exactly... You should go through the section and subsection headings and then look at the figures, diagrams, illustrations and so on. This is all you need to do at this pass and about what problem they have. I think that was much new in terms of... This is where she suggests that you should never read a paper alone. Should have a pair. So although that topic may not be of interest to you, but what that other person feels about it and what you feel about that. So if you read in a pair or a group, small group, it adds tremendous value to your own understanding. But you have to discuss. Don't just quietly sit in a class and listen to a lecture. You are actually talking, discussing and that is what is suggested. Because the teacher had some... You only want to get the big picture details. Paper that we've read so far. Write a two or three page review which contains a summary. We'll use a new presentation. It's of the paper and to summarize it. For example, let's say you're writing a literature, a lot of papers that you will use for it. And by the way, you may not know what it is. If it... No, I may say. Anyway, I'll close it because we are exceeding our time.