 Welcome to the ISD workshop on Introduction to Research Methodologies. This session is on tips on literature review. What we will do is pose a series of questions and means of some tips or guidelines on various aspects of conducting and writing and literature review. So far you have gone through the session on how to read a research paper. So you are now, if you were not earlier, you are by now familiar with how, what to do when you have to read a single research paper. But in the course of your research, you have to read multiple papers and then you have to do something with them. You have to put them together, write about it, evaluate it and what we will discuss in the session is how to take, how to read multiple papers, which papers to select, what to do once you read them and so on. What exactly is a research, is a literature review? And very briefly, it is both a process as well as a part of a research paper or a part of a thesis. And as a section or a part in a thesis or research paper, a literature review is a summary of related work. It is not a simple, just a direct listing. And it has some descriptions, but it also has some evaluative components. So there is both descriptive components of related work, what other related work is about and evaluative components, how they compare against each other, what works and what does not and so on. So there is a lot of analysis involved here. And even as a process, when a researcher reads multiple papers, gets ideas, tries to find out what is missing, what is common, what is good and all, the entire process is highly analytical in nature. Why do we even need to do literature review? That is, I am doing research, why do I need to look at what others have done? And you may laugh if I say that, but there are actually several reasons here. One of it is that one should give credit to others' work, because the kind of research one does is never in vacuum, it is always based on something maybe a few years ago or maybe a few hundred years ago that somebody else has done. So we need to give credit to others' work and in the academic community, it is equivalent to a law. So if a researcher writes a paper and does not give credit, people assume one of two things, does not give credit to other similar work, people assume one of two things. One is that the researcher is not aware of what others have done, others have done. That is not a good reputation to have as a researcher, it just comes across as if the researcher is naive or just does not have an idea of the field. The second option is even worse, the possibility is that the researcher knows that there is a related work but does not want to write about it for whatever reason. That in fact carries a very ugly name which we will talk about in the end. So essentially, it is equivalent to a law, one needs to give credit to others' work. So from a scientific perspective, that was from a practice or from an ethical perspective. From a scientific perspective, the literature review gives the background on which the piece of research that is being reported is based on. So it gives background to the work, it gives motivation to the work. So in a way it lays the foundation to the main piece of research that is being talked about. It situates the main work in the broader context, that is it is like think of a painting where there is a lot of background, there may be mountains and the sun and the clouds but the central thing is a house and a tree in front of it, without the background the house and the tree are not very meaningful. So it gives context, it gives meaning to the specific work that you as a researcher is doing. And finally, very importantly, science builds in a community as a series of steps. So the progress of scientific knowledge happens when new work is built on old work and that is exactly what all of us are doing. We take something that is existing and we either extend it a little bit in one direction or we broaden its scope or we compare it against some other work or we contrast it. It does not have to be our work does not simply have to be an extension of somebody's work, it can be the opposite of somebody's work. But we still need to know what the original work is or what the related work is in order to extend or contrast our work with it. So there is a lot of reasons why literature review and again this is mostly to set the stage. So you are a researcher and you are embarking on a new research project. When should you do the literature review? At the beginning when you start looking for a problem. Why? Because this is related work may be a place where the researcher gets ideas for the research problem. You may in fact get an idea as to whether you should extend the piece of work or you should try to argue against it. But that is not all. You need to do literature review in the middle of your research process once your work is underway. And this kind of research, this kind of literature review may be slightly different from the first kind. In the beginning typically the review is very broad. One is sampling from various areas, one tries to cover, look in different places. The visual image I will give is one does that or one does that. But once your work is underway your problem is a little more focused and you know what you are looking for. So at this point your literature review might become more focused. You may have chosen to work in a problem in this area and you want to find out more deeply what is happening in that area. So that is when you need a more focused literature review. It is not as open or wide as the previous one. But you also need this process towards the end when you have let us say you do an experiment and you have found some results and you have used your results to explain the solution to your problem. But this needs to be situated in the context of the broader work, the broader problem. So do your results agree with the previous results or related results or do they contrast it? Do your results, are your results very surprising because related work said something else? All these questions you will have once you have your results somewhat towards the end of your research process, the research project even at that point you would need some amount of literature review. Again there it is very focused. So from here what you see is that the literature review is not a distinctive standalone step that you do once and that is it. It is iterative, it is a feedback loop based on what you find in your literature review at the beginning you may choose a research problem. You may make some progress there and then read a little bit more literature and realize that that is not the right path to go and you may go back. So it is an iterative feedback loop, you do it multiple times, it is not one step but it is a part of the entire research process. The first question that we ask as researchers is what type of papers should I look for? And this is something we discussed a little bit in the session on how to read a research paper. A lot of it overlaps with what we discussed there that we are looking at engineering and scientific research papers. We are looking at published research papers in established journals and peer reviewed conferences and I have put established in quotes because this question always comes up how do I know a journal is established that is a question there are many answers to that. Sometimes in a community of researchers a certain journal has or is regarded as a sound journal or a reputed journal perhaps because it is been there for a long time perhaps because high quality papers get published in it and so on. Once you get more experience in a research area you will be able to identify these easily. If you are a novice you can ask your guide or you can if there is a paper you read and you feel that it is of high quality that it is a sound paper see where it is published that might be might turn out to be an established journal. You should definitely look at survey or review articles where which talk not about one single research problem and how it was solved but a survey article is a literature review of a problem. So in a way if you find a survey article your work is very easy because somebody else has done a lot of the work that you may have needed to do by looking at several papers. Sometimes you will find it in a book chapter we again discuss this in one of the earlier sessions that several papers on a research topic may be put together in a book. So you can look at papers in all these places what I would say is that you avoid papers downloaded from websites which contain no source information I am not saying avoid the internet. Two points I wanted to make first of all what do we mean by established journals and what do we mean by peer reviewed conferences because they are gradations and gradations. One of the standard thumb rules that we use is what is the acceptance rate of either a conference or a journal. So if 100 papers are submitted and 101 are accepted in a conference that extra one being the last minute paper authored by let us say director of that colleges. While there may be some very good papers in that conference in general the confidence level is not very on the other hand where the acceptance rate is 5 percent 10 percent and if well known researchers are submitting to that conference then that conference becomes a prominent. In my own field for example VLDB conference the very large database conference has a higher reputation than several journals in DL because that top notch people are KDD for example. So there are some conferences which have acquired a stature of much higher level. Now this is a practice which in IIT practically every department does I do not know whether you also follow that practice or not. Routinely every six months a group of faculty colleagues will do a review and rewrite the order of these journals and conferences rating them as A, B, C, B plus, A plus, etc. This is done for two purposes. This list is by the way routinely available to the entire academic community here. So this is to encourage our research scholars and MTech students that look you should pitch for publication in one of these journals or conferences. It does not mean you must but that should be a priority. Definitely then the researchers automatically look at the papers published in those journals and conferences, understand the style, understand the quality and try to emulate that which is a good idea. I doubt very much whether the institutions from which our 10,000 teachers will be coming whether there is any such practice is very doubtful. In fact I do not know whether in all your institutes you do that routinely and rigorously because it is important. It is important to send a message. This list is also used while we shortlist candidates who are faculty applicants for IIT Bombay. So we ask two questions one give copies of the three publications of yours which you consider the best and second give a complete list of your papers. From the second list for the faculty applicants we identify how many publications in A plus or A category journals or conferences from our list. From the first list that is from the first submission we rigorously evaluate all the three papers. It does not matter which public which conference has been published. So it might be published in a local conference in Chhattisgarh somewhere does not matter but that paper since the author claims there is one of the best papers it is very rigorously evaluated. So this is to ensure that a fair evaluation is done for the case. So the pedigree is important but individual claim of the best paper is also equally important. Now that is when your papers are being evaluated for a possible employment. There is no reason why we should not share this approach which many institutions would be taking including yours with all the ten thousand teachers. That be sure of the work that you do be always ready with identifying the papers which you think are best and additionally always endeavor to publish in high level conferences. So it is from that point of view that the literature survey shapes the mind of a young researcher that if one does survey from only local conference papers and based on it then there is not much. Why I am saying this is that while we talk about introduction to research and introduction to research methodology and while we tell them how to do literature survey and so on we should also emphasize the aspect that each one of the participating teachers should over the next two or three years by consistent work in a focused manner should try and get a qualitative improvement on the contributions that he or she himself will be making in the coming. I hope you agree with this. It is a double objective that yes impact factor is again used to characterize a journal or a conference and it has the same level of what should I say appreciation as the categorization as A A plus in fact almost all A and A plus journals or conferences categorized by various the best of the journals and conferences have occasionally have very poor quality papers they see and sometimes unheard of small conferences come up with one or two extremely good paper it is it is very difficult to generalize but by and large the impact factor has a mean unfortunately what happens is when you make that impact factor a part of any evaluation process like promotion or like employment then there is a rat race to get papers published in impact factor general high impact factor journals and another rat race to get your own conference or journal to be established as a high impact factor very funny things happen when you when you attach any kind of a personal remuneration or personal advancement to such thing the academic excellence sometimes takes a back seat but you are very right high impact factor journals and conferences have had traditionally a value system in the minds of people and that continues you see if I am going to evaluate a faculty member it is like you you start a game of hockey and you do not define where the goal post is or you keep shifting the goal post. In the last workshop on writing effective conference papers Dr. Murthy had lot of contribution so whatever the assignments presentations were kept on a moodle side so at our remote center I had downloaded all the information our college has a intranet server so on that server in a folder giving whose title was the conference by IIT Bombay on writing effective conference paper all the material was downloaded and it is still there and it is accessed by lot many faculty and students and our college has a post graduate courses recently started so many of the students published papers may be in a local conferences but they won the prizes in those conferences due to this guidance so I just wanted to put this on a record. In fact that is the reason why I told Sanna I mean Sanna Murthy to take portions of that workshop and share some of her observations here you can see you can relate to whatever lectures she is giving because this time the number of participants will be much larger the idea has to reach out to people thank you for that observation unfortunately our own website has not come up on the higher end servers yet otherwise all that material will be there incidentally one of the aspects of creating collaborative forums is that if anyone else is having a local site with such information including papers published by the local people will be glad to replicate all of them on the IIT website and provide links to your website if your link your website is exposed to public will be very glad to put a link but let us say you find a paper on a website you do not know if it is published or not you do not know which conference it was presented at you do not know when it was presented you may not even know who wrote it it it just is someone's or a bunch of opinions on some topic which is there on a website typically try to avoid those at least till you are sure you are able to evaluate the quality another way to answer what type of papers usually it is recommended that you read recent papers for let us say the past few years 5 years or so. There are some papers which are important even if they are old they are the classic papers or the seminal papers which have key results that are applicable even today or there is a classic technique that was used then and it is explained very well there and everybody uses the same technique today. So, there are some papers in every field which are classic which everybody refers to so even if they are old those are papers which one should put in one survey where should I look for these papers I know now what type I am looking for but where should I look for them and here there is a there are number of things one can do there are things called databases or indexes these are not a single research paper but you can search in the database what it contains is links to collection to collections of several papers several hundreds or several thousands of papers some of these may be familiar to you for example, IEEE explore is an index or it contains you will be able to find all the papers published by IEEE by most of the IEEE journals within in the past several years. So, you can either browse by topic or you can search like that for science and engineering the ones on the slide are very commonly used inspect one of them all the ones posted here are commonly used databases and indexes for engineering and science and if you look at the notes in italics there it says check your field because if you are a social science professor trying to do research these indexes may not help you there are alternate ones there. So, it is a good idea to check your field as to which databases and indexes you should be searching in whereas, one can find papers are websites of the actual journals. So, if you know that you want to look at the IEEE transactions on something go to the journal and if your institution has access to it has obtained access to it you might be able to see the entire article, but even if not most journal websites at least allow people any anybody to read the title and the abstract. So, journal websites are a good way to at least see what exists and then one can think about how to in fact obtain the particular article and of course, now these days there are several journals which give open access that is another movement which is catching on quite soon. You can go to conference web pages and see the top papers belonging to it. Conference proceedings sometimes are a little harder to find, but at least you will as you mentioned earlier you will be able to get the title and the abstract and one thing you can do is suppose you know that there is a certain paper in a which was presented in a conference in January of this year by such and such author, but you cannot find the actual paper copy look at go try to find the authors web page and see if it is uploaded on the authors web page that is another way another guideline to try to look at the paper. What I have put at the end is what the senior members amongst you would be very familiar with because that was all there was let us say 15 years ago. The library a good library a good academic library in a university would have bound volumes of journals where you can go and photocopy read the papers and photocopy them and so on. The first several are all electronic sources. One question which gets asked at this point is what about Google? Can't I simply put a Google search for I am going to use the same example that we used in the previous session. Suppose I put a Google search on use of clickers in distance education what would happen? So, the answer is you may get something, but it is like it would be slightly better if you use Google scholar that is more appropriate than a simple Google search which is extremely broad. You can also try sites here which is another focused search engine mainly for academic papers. So, when you are trying to locate papers you want a large number of papers, but you want them to be relevant. If you go to plain Google it is going to be extremely broad may not be very relevant Google scholar is better focused. So, is this one more focused are the ones we talked about earlier, but they may not contain all the papers that exist and so on. So, it is a balance there. Let us look at the next question. Once I have the paper what information of the paper should I record? Especially when you start reading more and more papers and let us say you have read 50 papers for your thesis. It is daunting to make sense of the 50 papers. In a few minutes we will see what information within the paper as in what to do about the content of the paper, but this is even before reading the paper itself. This is called bibliographic information. It is the identifiers of the paper. So, we need to record a spreadsheet is usually a good way to record these for any other way of keeping track. You need to record the title of the paper, names of the authors, year of publication, very importantly the source that is where was it published with journal or which conference proceedings or which book? Who is the publisher? If it is a book, what are the page numbers? So, all these are identifying information that one needs to do and just record it and keep it. I am mentioning it here it is a little boring to say or even to do, but it is invaluable if you have 100 papers and then you want to try to make sense of it. Secondly, sometimes if we go back to our literature review after a few months, we may have forgotten what papers we have already seen. So, the identifying information is useful in such a case. How to select which papers to read? How to select the papers? Again here are these are all tips, they are not, it is not the entire process that I am talking about. There is really no single process, but these are useful tips that any experienced person would tell you. I am starting with ask your guide and the reason is if you are a novice researcher, let us say an M. Tech student beginning on your first research project. Usually your guide is a first person who suggests to you the paper that you should start reading and then it is up to you to find the next paper to read and the next paper and so on. So, you can always start with an experienced colleague or a guide. As I mentioned earlier, a survey or a review article is a very good starting point because it has been written in such a way that a large number of papers pertaining to a given research problem have already been read, listed, analyzed, summarized, synthesized and so on. So, it gives you a very good idea of what the field is about. Sometimes survey articles do not exist for the topic that you are looking for and at that point your guide might ask you to write one yourself, but that is a different topic. Okay, next point here, how to select papers? Every field has certain key papers or seminal papers. Our job is to try to identify them. Once we have identified let us say 2 or 3 or 5 key papers in a field, then their papers we must read and their papers we will go back to over and over again. Some ways of trying to identify these key papers is that a lot of other papers would be referring to this one paper. So, if you see a name of a paper or a particular author or a research group being mentioned several times in many different papers, you can assume that this seems to be a key paper in this field because everybody seems to be going back to it. Sometimes it is the first paper that does some groundbreaking work in the area, but sometimes it is not necessarily so. Once you find the key paper, what you can do is look at some follow-up work from the same paper. So, find the key paper and see what the same authors have done after. Searching for the authors in one of the databases that we talked about or by going to the authors web pages and seeing if their research groups have done any further work. So, what you are doing here is really building up the collection of papers you are reading rather than going about it in a very random fashion. You look at let us say one survey article, two or three key papers, some related work and there is one more point I did not put here, but it was there on a previous slide that look at some very recent papers say five recent papers in the field that would give you a very good starting point to begin your literature survey. Then when you get more into the process what you can do is let us say you read an interesting paper. You like the paper for whatever reason it has a good technique or its results are interesting. Follow references from within the paper. So, see what papers are present in the references of the paper you like and backtrack through them that is another way to select which papers to read. One does not find too many sources there are very few papers one finds what to do then. So, there are a few things one can do we may have to change tracks a little. So, I would go back to discuss with some experts see if the expert is able to suggest some paper within some article within in that problem area. One more thing you can do is look for related topics or synonyms. So, for example, let us say you try to search for clickers in distance education and you do not come up with anything a very few. What you can do there is realize that the word clicker is sometimes also called student response systems make half the world calls it a student response system. So, maybe just look for a synonym or look for some related topics. So, instead of distance education you can think of online education which is not exactly the same but related. One more thing you can do is this is important you can broaden your scope a little bit. So, instead of looking at clickers in distance education look for benefits of clickers that that becomes very broad and you will get a lot of articles on clickers in face-to-face education also. But then narrow back sift through those and narrow back to clickers in distance education that is one try to if you have too few sources you can try to increase them. But what experts would recommend is look very hard try very hard before you conclude that no work has been done in this topic. If not this topic some work would have been done in a related topic and you have to be able to find it. I am not saying it is impossible but it is very unlikely probability wise that the problem I am working on is such that nobody ever has thought of anything related to it. If that is the case then the person who does that might indeed be the next genius around them which is a good thing but most probability wise most of us are not do not come up with ideas of that nature. What if you have the opposite problem which is more common actually well both may be common. What if you find too many sources what to do then? That means you find so many high quality good articles in established reputed journals that your guide approves of and recommends in the topic that you want to research. At that point you have to find a way to scale down your literature review to scope your literature review. So, there are some tips that you can choose to in the beginning look at only the recent papers. So, you can restrict by here you can restrict by source. So, for example, you will you can say that I look at only this these top two journals in the field I will restrict it only to those two journals or you can restrict it by topic that is you can further narrow down or scope the topic. To give you an example we will go back to clickers in distance education. So, get too many papers in that you can look for distributed architecture for clickers in distance education or you can look for clicker use in computer science in distance education. So, you can find words to scope your search further sometimes you use all of these together. So, you may look at only last year's top conference in a narrow field that is also possible. Again here an expert's help is useful in trying to sift through or scale down your search. So, now let us look at until now we were looking at how to conduct the review, how to find the papers, what to read and so on. Some manageable number of papers that you have read what to do then and how to report it. So, what I am going to start off with is an example that we see all novices many novices do. Let us say you have read three papers on the use of clickers I am just taking clickers because you have read this paper. So, we keep the same topic no other and it is also a general enough topic which is useful that can be understood by people, faculty members and students of all different domains. So, here is a typical novice way of reporting their literature review. Take a moment to read this. You see that there are three sentences. The first one talks about what paper one did, the second one talks about what paper two did and the third one is about paper three. But moreover the first one another way of looking at it is that the first line talks about student motivation and attendance. The second talks about student learning and also talks about how many students there were and the first one does not say anything about how many students there were. The third, the second and third papers both talk about the topic of the class. The second one was in a CS class and the third one is in a health science class. The first one does not say anything. This is not the way to write in the literature review section. It is the easiest thing to do when you have papers. There is room or there is a place, there you will have opportunity to list papers one by one. There is value in doing it. But when you write a literature review section in a thesis or a paper, an article you are writing, this is not the way to write it. And the reason is a literature review or a literature survey or the related work section goes by different means is not a listing of papers. It is not only a listing of papers. It is not even a descriptive listing of papers. It is not a summary of one paper after another, even if it is done well. So far we have talked about what a literature review section should not, how it should not be written. So, I am sure all of you are wondering, so what should I do? So, let us look at some tips there again. The main idea here is one has to be able to analyze and synthesize. And what this means is, what analysis means is there is something large and complex, just like an engineering system, which all of you are very familiar with. Analysis is breaking it down into parts and finding out the relationship between the parts. So, when you read research articles, what are the little pieces or the parts of a research articles? It could be the themes, it could be the variables in the research article, it could be the problems faced. So, there are many themes, issues, factors and variables, which are let us say the pieces or the sub pieces of a research article, those need to be identified. And then they need to be organized. So, one way to do it, I will just give you one example that, which might be useful in organizing these various themes and factors. So, I am going to show you a table and this is, these are papers on clickers. Draw is a new paper, this is paper 1, paper 2, paper 3 and so on. So, the list is long. What has been done is themes have been identified and that is where the skill of researcher comes in. What are these themes? Some are very obvious such as the paper title and year and maybe even the authors must go in this table. So, the broad topic here was use of clickers. So, the researcher, in fact, this was a PhD student in her first year. What she said is, okay, first I am going to try to see which subject each paper or what is the subject in which the clicker use, clickers have been implemented. The first one was physics, the second one was some numeric modules, the third one was physics again. So, even just by looking at three papers, you might get an idea that, okay, perhaps we can start looking for trends or patterns. And similarly, we want to develop more and more categories or themes and try to classify each paper according to those, according to these categories. Let us look at one more example. This category here talks about the methodology used in the paper. One uses surveys, the second uses a qualitative questionnaire and the third uses something. In terms of what is measured, the first one measured student learning and the researcher has also written how student learning is measured. For example, they say that she says that there is a concept test performance which is measured. Second paper claims to measure perception and learning and so on. As you see this table is evolving, you would not be able to write all these headings after writing only the first paper. So, your number of columns in the table goes on increasing, so do the number of rows. After reading two papers, the researcher, student researcher realized that she should write down what are the main results. So, she wrote down results and then she wrote down a column called comments and ideas for future. So, as she was reading the paper, she was getting some ideas, what she could do and she was jotting it down here. So, let us go back to our main slides. So, one of the first things you have to do is identify these things and find a way to organize them. This is just one way to organize the literature review section and organization is important mainly to identify the trends and the patterns. Then what one does, the table will become very handy now is analyze these papers further analyze on the basis of categories which we have already done to some extent, but on the basis of strengths and weaknesses. So, for example, using this table, one can look at the results section and see which papers report positive results and which report neutral or negative results. One can read the notes one has made under methodology and comments and decide whether what are the strengths and weaknesses of the paper. Perhaps after you read a few papers, you may want to come up with a new column called strengths. So, the analysis has to be done on the basis of strengths and weaknesses and here is an example again based on the clicker paper. The categories that the researcher decided. So, this is a researcher reading the paper that you read the distance clicker use and distance education. It is some student who read the paper and what he decided to do is he came up with mainly two categories. One is one category I will call as effectiveness and his summary was that clickers in face-to-face classrooms are highly effective for the following reasons. So, this has been written after reading several papers. Another way of analyzing was on the basis of benefits and drawbacks. So, if you read the paper you will see the since you have read the paper you may be able to recognize these that the authors write about both the benefits and the drawbacks. So, they are analyzing their literature review section on the basis of benefits and drawbacks of clickers and of related work to improve interactivity. What do you know what can you do next put all the little pieces you have found out together you have all the little pieces that you have analyzed you have to put them together or you have to synthesize and create something new. For example, you can find a pattern you can say that I will first look at the graph of the graph. This was also a student what she decided to do is plot a histogram a bar chart actually of the number of papers she read and the subject in which it belonged and how she did it was based on counting categorizing and counting the entries under this topic of subject. This immediately gives us insights you will see that number of research papers on whatever topic is very high in health professions and psychology not so high in others. So, synthesis gives us deeper insights. How does one do synthesis what you can do is try to look for anything that is common or something that is different. So, analyzing and synthesizing on the basis of commonalities and differences is a very often used technique and then you can use representations that you use in engineering charts, graphs, tables to represent the analysis. Once you have these graphs in all you can write your literature review section in terms of maybe three paragraphs you get existing solutions for your problem on related work. What are the common things in all of them and how do they compare? So, each of these questions can become one paragraph in the literature review section. Another important purpose one has to identify the gaps in the existing work. In fact, that is one of the key reasons why one does literature review. The example this example in blue has been taken from the Clickers and Distance Education paper. Just read it and the last sentence here clearly identifies the gap in the existing work. This paragraph comes from the paper itself it comes somewhere towards the end of the somewhere in the introduction section where they are talking about existing work. So, the authors have concluded here is that no one has identified Clickers and Distance Education so far. The gap is clear. This is important because this leads to the clear need for the research or the pieces or the paper that comes right. So, now let us look at some minor points to note two or three of them. As I might have mentioned earlier at a broad general level all this does hold in engineering and science maybe even in other fields. But in spite of that you need to be aware of the conventions in your discipline because in certain disciplines it may be possible that the literature review section has to be written in a very specific way. If that is the case please follow it. In the absence of strict guidelines like that you can use the broad guidelines that we discussed right now. This is one minor point. The second point and this is really the last point I want to make here. There is a tricky issue it is a conflict in a way that you have read about all these papers you have read about or you have read all the existing related work. So, ideas already exist and they have already been written by other people. But you have to write about something that already exists but at the same time you have to write it in an original way you have to say something original. What this means is we will do an entire session on this. You cannot copy a sentence from another paper and put it in your literature review section. You cannot copy a graph and put it in your literature review section. You have to say it say something original. This is tricky it comes with experience I know that seems to be a cure for everything. But if you follow some of these guidelines for example, how do you provide a synthesis? How do you do analysis? How do you identify themes and so on? These are guidelines that help a researcher come up with something original based on something that has been already said. What one should not do precisely that there will be most likely there will be a session on how to avoid plagiarism. This is the ugly word that I was mentioning earlier and I do not want to talk about it here. Let us end here because what this session was supposed to be is some tips and guidelines on conducting and reporting the literature review section of one's research and I hope you will be able to apply some of this to your own work. Thank you.