 Hello. Good morning, everybody. Buenos dias a todos. Bon dia to Tom. We'll begin this conference and I begin this opening words by giving the floor, handing over to our vice rector for research at the UPF at Universitat Pompeu Fabra, Enrique Vallduby. Welcome. Good morning, everyone, and welcome to Barcelona, Universitat Pompeu Fabra and this Poblano campus, devoted to communication language and information and communications technologies, and I'm here, it's a pleasure to be able to welcome you here today on behalf of rector Jaume Casals, who's committed elsewhere out of town, so I'm here in his stead. And I don't know, I basically, my job here is to wish you a very fruitful, enjoyable conference, the second international conference on academic communication journals. As vice rector of research, I'm aware of the relevance and the essential character of dissemination. I mean research is at the core of our raison d'etre as academics, but you know, research doesn't get disseminated if it doesn't reach out, if it doesn't get to, not only to the wider academic community, but to society in general. There's basically no point to it. So, you know, we need you, we need everybody, researchers need communication professionals, communication professionals, journals are also essential to our, to our trade as researchers. I finished my PhD in 1990, so it's a long time ago, I'm a linguist, basically there were five journals, you know, I had to read when I was doing my dissertation research. Things have changed a lot, not only in quantity, but also in terms of resources, the means, how things are done. I think this campus is interesting in mixing, you know, communication and technologies, because this is the way things are working today, in all respects, but essentially also in terms of publishing. And as part of my, my job today, I mean, Vice Rector, I've been Vice Rector for a few weeks now only, but one of the first things that I was actually pointed to is, you know, the relevance of, of, of this issue and how our publications get spread out to the, to the community. And that's definitely an area in which a lot needs to be done, and in which cooperation, mutual knowledge is essential. So I hope this, this conference is fruitful in that respect and that, you know, that, that leads to good, deep collaboration between the academic community and the publishing community. So having said that, welcome again and have a great day or couple of days. Only one day. Day. If you stay in Barcelona for the weekend, enjoy that too. Okay. So thank you very much for being here and have a great time. Thank you very much. As most of you already know, UPF is a public urban university, as well as a young, a young one. We have only 26 years. We have, we are only 26 years old. However, it has achieved quite a, quite a reputation among academics due to the quality of our teaching and also the quality of an impact of our scientific productivity. Besides that, the UPF communication studies play an important role as we are one of only, we are only eight departments in this university and the communication department is one, one of them. And also our teaching and research has been positively assessed by the academic authorities in Catalonia. Today we are hosting this conference on academic communication journals. It's a, it's second edition, the first one took place two years ago. And I hope you're going to find it useful and full of insights for your future work and which I believe will be a great opportunity to learn, to debate and to share about the pitfalls, the challenges and opportunities that communication journals have to face in the future. Thanks to the vice rector for his happening words. Thanks to the organizing committee for the brilliant and hard work. And thank you all for being here. I wish you all a useful and pleasant conference. And for those of you coming from abroad, a wonderful stay in our university and our city. Thank you very much. And now it's time to begin our opening lectures. So I'll hand over to Professor Rafael Pedraza. We will introduce the first speakers, our opening lectures. Okay. So good morning, everybody. It is nice to see you all here. Thank you very much for coming and welcome to the second international conference on academic communication journals. My name is Rafael Pedraza. I'm an associate tenure track associate professor at Pompeo Fabra University. And I will be with you as chairman of this first session in the conference. Well, before we start with the opening lectures, I would like to explain some aspects of the conference, of the conference program. Today we have, as you can see in your programs, today we have three sessions, a morning session, this one, and two simultaneous afternoon sessions. We will make the time for lunch at 1 p.m. And we will back and restart the afternoon session at 3 o'clock. During the morning session, we will have four presentations, show us two opening lectures and two keynotes. At the end of the session, we have a room table. Approximately at half past 11, between the opening lectures and keynotes, we will have the coffee break. The opening lectures will be carried out by Dr. Emilio Delgado Lopez-Cotter, next to me in my immediate right, from the University of Granada, and Dr. Ismael Raffles, on my left, from the Polytechnic University of Valencia, and also from the University of Sussex. And the keynotes, after the coffee break, will be carried out by Mr. Steve Smith from Kudos, thank you very much for coming, and Dr. Karen Boyle from the University of Sturling, that I can't see here, but she's here. Each presentation will last approximately 20 minutes. After the opening lectures, if we are on time, all of you, attendees, can ask both speakers a question related to the presentation. But if we are not on time, I would be very grateful if you can ask at the end of the session in the roundtable, okay? Well, we will follow a similar structure in the keynotes. In this case, you can ask the keynotes speakers directly at the roundtable, okay? Well, the roundtable is planned approximately at 12 p.m. In relation to the roundtable, one of the main objectives of this conference is to meet together and to inter-communicate. So, please, you are invited to ask, to speak, and to participate in the roundtable, okay? Now, another question. In relation to the language used by the speakers, in this conference, this is an international multilingual conference, so the speakers will use Catalan, Spanish, or English, whichever they prefer, okay? On the other hand, in relation to the afternoon sessions, well, they are focused on communication journals. One is devoted to international journals and the other to national journals, but from the point of view of the nationality of the journal, not by its scope. I mean, all journals with presence in both sessions in the afternoon and represented by the editors have international reach, okay? The international communication journal session will take place in room 55309 in other building, okay? You can find it in your program. The national communication journal sessions will take place in room 55410, the volunteers at the door of this auditorium will help you to find this room later, okay? Well, now I think it's time to start the opening lectures. And well, our first conference, I'm going to present it in Castellano. He is Dr. Emilio Delgado López Cozar. Emilio is a professor of methodology of research at the Faculty of Communication and Documentation of the University of Granada. He is a creator of a great number, I would say, of a large number of systems and tools for scientific science, such as the H-Index Scholar, INREC, the Journal Scholar Metrics, the H-Index of Spanish magazines, according to Google Scholar Metrics, Scholar Mirrors, Goath or Index, Publishers Scholar Metrics, Res, Spanish magazines of human social sciences, CIRC, Integrated Classification of Scientific Books, the Ranking of Spanish Universities, or the DNA, the National Directorate of Authors, among other many. The production is incredible. Well, it has more than 200 publications and it is the founder of the Research Group, Science Evaluation and Scientific Communication. Its research lines are focused on the study of Google Scholar, in fact today we are going to be focused on Google Scholar, as a source of information and scientific evaluation, also in the evaluation of the scientific research and research institutions, in the evaluation of scientific magazines, in the analysis of evaluation systems of Spanish science and in the study of the situation of Spanish research in documentation. Today we are very lucky to have him for our opening conferences. His conference is titled Revolution of Google Scholar, discovering the box of academic pandora, and Emilio, when you want, you have to point. I think that although I am invited to talk fundamentally about Google Scholar, I have to tell you that I will talk about that, but I will also talk about the context in which Google Scholar is acting. I think that there is a revolutionary transformation in the field of scientific communication, since we are in a communication conference, of communication magazines, therefore communication about communication, I think I should be talking about which transformations are being produced, and you have to mark the appearance of Google Scholar within those transformations, but I am open that to talk about communication you have to talk about evaluation, because I mean the evaluation of the scientists, because the evaluation is radically conditioning the way in which we are communicating. Therefore, I will talk about three things, of the transformation of communication, of Google Scholar in that context, that paper plays, and finally, if I arrive, because I do not know if I will be able to transmit so many transparencies in such a short time, well, if I go a little fast and crazy, it is the attempt to send messages, in some cases provocative, so that we generate reflection. Well, a year ago, in 1921, I wrote an article where, without knowing it, I have to say that there was a change in communication processes caused by technology deformation. Really, the changes in communication have always been produced because there has been a change in technology. The company generated a revolution in the 15th century and in the 90s appeared a change generated by the new technologies that I think has radically changed and will change even more in the future the way in which we communicate to scientists. The bottle walks. Well, if I had to synthesize what the changes are, I would say that in the old model, in the Glacier Gutenberg, the Glacier Gutenberg Scientific, I mean the distinctive element is that the power to transmit the information was controlled by the editors. Well, publishing of editorials, well from magazines, they decided what was published and the author could not do anything. Obviously, also in the scientific field, there were two filters. The first filter, now I am talking about the magazine, fundamentally the magazines are in almost all fields, the main media of communication, of research results, there were two filters, the first filter of the editor with his reviewers to decide what is published or not published according to the conventions of the scientific method and those magazines later had to suffer another filter, which is the one that established the databases, among them, and especially the one that Osayan, that select club where everyone wants to be and where not everyone could be, okay? Well, therefore there is a double filter, a double control over what is produced. What is the change that the Glacier Gutenberg generates? Well, fundamentally, I think that the Glacier Gutenberg what it does is give the voice to those who did not have it and I also add a loud voice to those who already had it. This is an absolutely revolutionary change. Anyone can have their discovery without any control. In addition, it allowed to do what Max Lujan was talking about, that global aldea, that beautiful metaphor, global aldea and interconnects with the scientists. I, because I have canas, I come from the typographic home, I am from generation 0.1, from the digital one, from finger to finger, okay? And, hey, we could not connect easily with the scientists from anywhere in the world. Today we are interconnected. This is also absolutely revolutionary in science. But not only that, the author is able to appropriate the entire scientific communication system. That is, the author can edit, can publish and disseminate himself with tools that are, that are available to everyone. That also seems to me to be revolutionary. And even more revolutionary is what comes now. The author connects with the reader. I have put in the upper corner what happened in the past. The author, claiming in the desert, wrote for whom? Who were his readers? Obviously, there were readers who were not known. Today, today, the reader, sorry, the author knows in real time, and this is revolutionary, and I do not know what consequences of social character they will have. I think it is very interesting. In the future, he knows in real time who is reading them, who is following them, who is retweeting them. All. I do not know what consequences this will have. In principle, always, all technological changes, such as acquiring technology, have a part of a friendly face and a black face, right? The black side of the force, or the friendly side of the force. From the point of view of communication, the old glass that he made is that he multiplied the channels of scientific communication. We start with the first generation, which are the web pages. We continue later. They are not in the same order that came up, the authentic repository of scientific information storage, which can be an alternative to the traditional media of communication. The blogs, the videos, YouTube is transforming the way of communicating and teaching. I warn the navigators who we dedicate ourselves to teaching. Many students learn today on YouTube and not in our classrooms, and this will be reinforced. And anyone who has knowledge and capabilities to communicate something. The social networks, which I am going to say about them, and the social networks academic and scientific, which are challenging. This capacity, therefore, of new communication channels, increases the amount of documents, the mass of circulating information in the world. And therefore, it also allows, for the first time, that we have open access to knowledge. And this is also revolutionary. And with some magnificent effects it is supposed for the generation of new knowledge. More knowledge must provoke more knowledge. And this would be what we would call the virtual circle of communication, okay? Well, with this mass of scientific communication circulating throughout the seas of knowledge, with 100,000 ports where to attract, we needed a boat or someone to transport the information that you put in communication to those ports. Well, there appears the searchers, to my understanding, and specifically Google Scholar. It is that boat that allows us to provide information. And I have put this one. I know that there are many, there are many more, there are many more. I heard about it from time to time, I do not remember it. Not only is Google and Google Scholar, but people are obsessed with using it. What are we going to do? In any case, always remember that the imperials do not last forever. I do not know if it will be around here, Rodríguez Gairín. I do not know. Well, who remembers Altavista? He, of course, remembers it, because he did a seminal and precursor job of many jobs that have come later to measure the evaluation. Who remembers Altavista? Who remembers Warpfeffer? Who remembers NSK? We remember what we have. But they have disappeared literally. Who remembers Flores? Well, we still remember Flores, because Microsoft is behind it. But, well, I do not know if this will happen with Google, Google Scholar. It can happen, it can happen. The Kain Imperials. Look at the perspective, what is history. Well, at the moment, Google Scholar has become, of course, the main academic searcher. And why has it been? Well, just because of that. Why is it simple? Why is it simple? Because it is fast, fast. Look, always the obsession is to respond to the search in a very fast way, millisecond. It is easy to understand, it is easy to use. It is universal, really universal. That ship reaches all the ports of the world. The previous ships of the world of scientific communication only landed in London, in Boston, in Berkeley. And in more places, the periphery did not exist. For this searcher, all ports, in all ports, it is convenient to track. Essentially, one likes to provide more than one to the other. It is multilingual. And I say multilingual, they are all the languages, all the languages, all. All. And also, and this one, I think it is a key to success, it is free. It does not cost us. Of course, that helps a lot, really. I do not say it. We like empirical evidence. And we have mounted a blog, a newsletter and a Twitter account, where we are transmitting all the scientific work that is referred to Google Scholar. And the evidence is overwhelming. Google Scholar is today, 10, in almost all the fields of knowledge. There is always a difference in discipline. I am doing a generalization. But despite the small differences in discipline, it is the main scientific searcher in the world, the most used. He gave us to find out the size, because the size matters in other issues. And two years ago, we found out that Google Scholar had 170 million of documents. Look at the difference regarding the rest of the basis of the scientific cluster. And it does not matter. We have done a lot of studies and a lot of studies with different sources of Google Scholar. The first, in the upper left, a study on biotech and documentation, of magazines, magazines. Now we are talking about magazines and using Google Scholar. You can see the differences with the databases that have been used traditionally to find scientific information. If we go to Google Scholar Metrics to the right, and we see the case of Spain, you can see the difference between the magazine number indicated in Google Scholar Metrics, which is restrictive. Remember, 100 articles in five years and at least a quote. Therefore, not all of them are in Google Scholar. If we look at communication, and this is interesting for what we are seeing here, then the differences are also important. I am not saying because I am not going to give you time. It does not matter that we look at magazines. If we look at authors, in my case, you can see the difference in the number of documents and quotes that are able to cast Google Scholar. The size matters because it means that it is able to cast all the variety of the scientific impact, and not scientific, as I will say next. And look at there, Resi Argate, as it has already been glued with respect to the rest of the database. I put aside Agüeo Sayan, that select club in the ice, for all the magazines, because there is a hidden part. They see that there is like a veil. The veil has to be seen at the quotes that appear in the area called reference site, where we can find the double of the quotes, and that it had never been taken into account by the biometers, fundamentally because of a control problem and normalization, the same as having Google Scholar. Well, what am I going to say at the international level? Is it really Google Scholar International? Look, they simply have to see the transparency. See what Google Scholar is and what are the other databases, sorry, with an important blind angle. In the lower part, in the case of disease, in the case of communication, the comparisons are still better, because we can compare each one of the databases with the world. Look how Google Scholar looks like the world. It looks like the world. Enough. Much more varied. This in terms of countries, but also in terms of languages. And if you look again at the comparison, compared to what the world is, look at the first column of Google Scholar, the second, as it seems, and how the pass of data by Osayin and Skopu are blind to the world, Anglosajón, as it is normal for the development of science. But what is interesting about Google Scholar is that it is able to collect all kinds of documents, all kinds of documents. In the upper part we see quotes, in the lower part we see documents, in this case of Anwil Harzing. In the upper part of the quote, you can see that the preceding quotes by Osayin and Skopu are so exclusively limited to magazines, while in the other case there are documents of all kinds and conditions. Of all kinds and conditions, and I'm not going to read all of them, because there are many. Of course, what does this do? That the portrait that Google Scholar offers us about communication, science, is different. Do you have the list of which are the 10 most cited works of 1950-2000? Sorry, thank you. 1950-2013, sorry, 2016, we updated this a month ago, which are the most cited works in the world in Google Scholar? In English and in Spanish. In the case of English, only two papers, and I say papers. In the case of Spain, none. All of them are books, and we even see some document out there that can't be strange, but there it is. Why? Because it's cited by the researchers. Therefore, the portrait is different. The portrait is more or less incomprehensible, because my thesis, the hypothesis that we keep in our group, is that Google Scholar is capable of capturing other impacts. Even the scientific impact, like the way they do it in Weoside and in Skopje, can't give data, but they have to know that the correlation, when we measure this, is very high, very high in all fields, above 0.9, that is, we are measuring the same, and why are we measuring the same? Because Google Scholar has swallowed the other steps of data, the other steps of data live inside it, okay? But also Google Scholar has documents, and it is able to measure the impact that I call educational, the professional impact, not including the political impact, I don't think the social impact. And empirical demonstrations, there is always data. Look, this is one of the products that we have done, using a ranking, from Google Scholar of Spanish investors in communication, sorry, in the Milo de Economía de Documentación, and it is shaded in red, people who are academic colleagues, but who have a great professional impact. If we used Weosayan, which I think is the third column, and they see their H index and their data of reference, they wouldn't allow them to be there. If we used Weosayan and Scopu, they would be absolutely missing. They are not missing when we use the data, and their impact can be calibrated in the community, which I think is quite high. And well, the other day I did a little tour with a magazine that will say that it is very difficult to measure data, and well, the other day I did a little tour with a magazine that will say what the hell is this magazine? Well, yes, my library, you will know this one from here, from the library, and little more. A very good magazine, almost divulgative, because look, it has 728 documents, or 728, and I dedicated myself to analyze who quotes them, who quotes them from the point of view of the documents. Notice that the magazines are, but what magazine? No, they are not New England, they are not the Hasis, they are the Journal of Documentation. They are magazines of national character, of character, in some cases, local, of divulgative character. We have there, in second place, the seminars, the congresses, in third place, the thesis, well, they are thesis and degree. And well, and the panoply is incredible. There is even a memory of opposition quoting. Impressive. And from the point of view of the impact, Spanish, 94%. This means that there are places, territories, where the impact is not really measured, only a part of the impact is measured. Well, precisely because of all these data is why we decided to open the Pandora's box. And we have dedicated ourselves for nine years in an intensive, extensive and obsessive way. I admit that obsessive. In science, they don't do great things like that. Well, I'm not cataloging what great things to do. Of course, I do many things, as Rafa said, with my team. With whom? With these. These work code, we work code with code, not code. Enrique, Alberto, Cuama are the key of all the products that we have made opening that Pandora's box. They are many that are referred to magazines, especially at the Scholarmetrics. They are referred to the authors, the H-index Scholar that we released in 2012, a real provocation for the researchers of humanities and social sciences. I thought that they were going to throw us tomatoes. 50,000 researchers of humanities and social sciences ordered by number. The people of letters ordered by number is a real provocation and with the G, I think that the G has its thing, right? Nobody protested. Well, very little. If I teach you the list of emails that we have received, this is the indicator of the biometric contamination that we live in Spain. The biometric wave arrived in the United Kingdom in 2015. In Spain it arrived in 1989. For once, Spain got ahead and it was a precursor. I would have liked to invent a vaccine, but no. We invented the application to Tutti Plén, as we say here, of the biometric wave to measure it all, okay? Well, we did that product, we made products with editorial ranking and we have entered in the last, which I think is the contribution. For us, these are experiments. People think that we make final products. No, no, they are experiments to know what can be done and how it affects people. And people in Spain react in any way. Well, this is the journal School Armetry. I don't know how I've been for a while, I suppose that's bad. Well, then you see that I'm not going to make propaganda of my product. Well, I'm going to do something about propaganda. Apart from including all the magazines that are on Google School Armetry, we have been able to identify 10,000 among them 320 of communication, very interesting for this field. We have created new indicators and for the first time in the world, I like to say so, we have found the auto-citation of something on Google School Armetry. The auto-citation of something we have done it. We are at the disposal of doing it for other fields. What happens is that, well, we have to count resources. Man, I would like to count with the Nouragha Chariah, the inventor of the thing, but this is difficult. This is difficult. Well, I can't count all the products we do. Really, what we have done with Google School Armetry, I admit what we have been doing, but I have to tell you that we are always critical. Someone may think that we love it, it seems that Google School doesn't pay us, don't pay us, unfortunately. I would like not to pay. Surely I wouldn't be critical. Well, the main criticism that is made by Google School is one of the main mistakes. Well, we have indicated four years ago that we have to say that the error rate in Google School should be around 10%, and they will say, oh my God, 10%, this we will not tolerate in any area of research that error rate is higher. Of course, when we talk about big data an error rate of 10% is small. Well, what is happening with the results that have come up? We will soon have the result of a doctoral thesis where we will already give results in a global way. Well, the fact that we give the hypothesis is true. There are no errors above 10%. For this reason, in a big data tool like Google School, it is irrisory. As I say there, even dirty, it measures more and better. I am referring to Google School and Cascope, which now we have discovered that they have errors. Well, of course they have errors. It is normal that they have, but the surprising thing is that it does not commit any errors, a searcher who attacks in all the ports of the world, in all languages, making investors, collecting documents that publish investors as they want in any format, it seems surprising to me that an error rate is so low. I think that the big danger of Google School and all the metrics that come and that come has to do with manipulation. We decided to manipulate the index, the index that can be generated from a heliometric character with our products and we wanted to show that it could be done practically by digital alphabets and it can be done. I also say one thing and this was when he inquired saying that how his product could, he said that it was a question of spam. And he said, everything is seen. Well, that's true. If one does weird things, it is seen and it is seen. Man, we wanted it to be seen easier, and that's what we told Anurag. Man, why don't you do that certain auto-citation rates of people appear? And so we did it. This we did it in INREX. I have to say that INREX is a wonderful product from the point of view of design. We could find out exactly if a person was manipulating the index, as we said in Andalusia, it had the procession. They already know that Andalusian processions are supported by many orquillers, because there was one that carried the entire procession. All the magazine, all the magazine was supported. Well, he was down, but he was also up. It was an extraordinary case. Changseng, really was like that. Well, I think this is an important danger, but to some point because it is seen. I think that this is also another one, the dependence of the technology. Soon, if one day Google disappears and Google Scholar, the little chick falls, of course, they can also disappear. It seems that other things they do is mutate. Thompson, Reuters, Kalar, I don't know what and I don't know what will come in the future. Another investment fund, I don't know. Well, it is dangerous to have dependence on a single product. Of course, the lack of transparency must also be criticized. We do not know. We know that what is inside Google Scholar is very good. Why? Because they are indicating all the academic web, all the editorial bases, LSEB, Springer, Willey, everything and what is in the repositories. But of course, we don't know. A bibliometer cannot accept, as now I suppose, that you will tell us Ismael with Leiden's manifesto. Hey, because we have to know what the sources are. Of course, we have to know them and we can't find them. Sorry, if we can find them, I think we can find them lightly. What are the things that try all these products that I have shown? Especially in the area of social and human science, which was an unknown area and therefore the communication that lives in that academic territory. Well, we didn't know anything. With Google Scholar, we see something. Okay? Good. And I'm going to finish talking about I think that in a change, in the communication model, Google Scholar is part of that change. And that change has to do with the appearance of a green route. Today I also talked about it, and I wanted to talk about it through the blog. Today, any scientist can follow this green route that consists of anything that thinks, that discovers, can deposit it, can spread it through Google Scholar, three days, starting from three days when a document is deposited in the repository and it is accessible, and then you can use all these diffusion tools, as are the social networks, the academic social networks, and from there you can collaborate, share, share collective reading. Does anyone do it? I don't know. People ask, answer, yes, in Reserhey, people ask and answer. Therefore, the implementation is fomented. Well, the great question of the people of the Selector Club, of the scientific aristocracy is, and how do we control the quality? Because, of course, in the traditional system the excellent evaluation has been produced, that is, the evaluation where the editors, with their reviewers, decide to be published. Well, in the new postmodern model of communication, it is the people. Speak, people, speak, scientific people, I mean, but also the non-scientific. Listen, if I publish a work, my work is exposed and can be commented, criticized, reviewed, enter in Reserhey. I will soon practice it in a systematic way, also as an experiment. I am going to evaluate co-workers who are interested in analyzing it. And I will put in the comments so that people can see it. I am a Darwinist. Even within this mass, I think that the good things always come about. And I have said, with this I will finish, I will go fast, I think that the evaluation is key in the form of communication. We warned it in 2002, then in 2007, and to Great Britain came the wave or the biometric contamination in 2015. The evaluation condition. What is going on in the evaluation? We are going through a biometrics where it is evaluated little by little and for little to evaluate everything, for everything, in all places, all the time, anything we do is measured. The old biometrics is a biometrics whose epicenter is the magazines. And everything is mounted doing magazine rankings. And with an indicator that was the ImpaFactor. Now there is an Olympic sport that consists of shooting against the ImpaFactor. Not only the declaration DOA, DOA I have said well, but the rest of the manifesto. Well, it does not touch now, but when I am there, Maria likes to defend the role that the ImpaFactor has been able to have. And that all the system of evolution that I voted in the magazines. Today, the system of evolution is led to what I call the old matrix. The old, not the old. Everything can be measured because everything we do is on the network. Because we leave our mark. And today we have many expositors. I speak of expositors of my mistake, of expositors. And the expositors are in many places and there are new expositors. And we have just generated that experimental product to show all those mirrors and we have done it also with a provocation, clearly, we have entered the counting house. In the house of the accounts. Or in the house of the appointments. Sometimes I like depending on where I play with one thing with the other. I am a bibliometer, I belong to that house. Therefore, that is a self-criticism, the next analysis will not vote on the magazines. Today we are doing here a day. They will do it on the documents and to start with the authors. Formule this question that I will not proclaim because from now on I present the transparency and I shut up. One moment, I have not finished, there is still something more disturbing. The gamification attentive to the matter. To the gamification of the scientific and the activity. We are anthropometric, which are narcissists. We will have new diseases. We already have them. Before the disease arrives with its cruelty, the impact and we have already had it, but the egotitis will arrive and I do not know if other things. I do not know if I like the world of science that will come. I do not know if the new metrics can harm science. And the same we have to do a research. I do not know if it is from Leiden or from other sites. Now it is over. Thank you. Before the coffee and if we do not do it in the round table. Well, now we go to Ismael. I pass to the English. Now I would like to introduce the second speaker. He is Dr. Ismael from the University of Toshoku, Japan. He was postdoctoral researcher in nanotechnology at Cornell University. Currently, Dr. Ismael is a science policy analyst at Ingenio, from the University of Polytechnica de Valencia. He is also associate faculty at ESPRU, the Science Policy Research Unit, as the at the University of Success. Where he worked it from 2005 until 2012. Well, Ismael investigates how to make more plural use of science and technology indicators for informing evaluation, funding and research strategies. Currently he is working on research portfolios developing mapping methods to interrogate the research landscapes for grand challenges, such as build flu or obesity. So as to facilitate deliberation on research prioritization he is also exploring inclusive metrics trying to understand and correct for indicators biases and the effects on policies, for example in issues such as local knowledge. Now, he is going to present the Leiden Manifesto. So Ismael, whenever you are ready. Thank you. I don't belong to this field of communication and I was wondering how I should present the Leiden Manifesto in this context. What I realized is that there was a lot of talk of evaluation in a conference that was about communication and I wondered what that could be suggesting. My main area is research evaluation and today I feel comfortable. But if you see here these colleagues in Leiden this is Sarah the Riker we are wondering whether journals now have become more a place to put things for so as to be evaluated so a clearing house for career aspirations. Rather she is asking whether journals have become more a place where we put things so that they are evaluated so that people, organizations universities become evaluated rather than a place for communication. And the emphasis in this conference which is about communication journals on evaluation suggest this potential tension between journals as a place to gain attention, to gain reputation to gain excellence versus journals as a way of communicating either with the scientific community or with why the audiences. The classic view of academic publishing as illustrated in the sociology of Robert Merton is that journals where researchers would share their findings they would share these findings for free with a community a specific community and the incentive to do so was that the community would award them reputation but it was the community it was not through bibliometrics is what not people external to the community now in the last 10 25 years in some countries earlier than others in Spain as Emilia was mentioning has been an early adopter there's a reputation by peers has been supplanted by the quantified excellence in sociology of sciences took the change of reputation by quantified excellence for example Paradise talks about it and in some such policies talk of the the supplantation of the logic of the old science which is by the community the scientific community versus what they call the neoliberal economy and I'm not sure this is a term that I agree with but let me just throw in that the number of people talking about the neoliberal economy where publications citations become the currency there's an economy around publication now these types of effect the effects are that there's a potential narrowing of the type of scientific activities because people try to get the currency rather than producing or communicating and there is the potential task reduction and this relates to what Amelia was calling the gamification and I'm presenting this in very much in a questioning mode so I would like to hear with the opinion for example of Professor Boyle Doyle Boyle about in the case of the UK the use of four publications which are then evaluated and have a huge impact on how many economic resources a department gets what this is happening and potential detrimental effects so let me present this I feel a bit out of place so let me present this in a questioning mode in front of this potential problems that the bibliometrics was causing we came up with the idea in the bibliometric community to have a manifesto now the manifesto that acknowledges that we begin with the idea we are bibliometricians that metrics indicators are important and useful why is that well in the last 30 years there's been this huge growth in the science system 20 years ago we had half a million papers for a year in the web of science now we have one and a half million this is only in the web of science in Google Scholar the increase is possibly two or three times this one there's globalization who knows who understands which other researchers and universities and departments in China in Brazil only in Brazil they have to evaluate these problems with this growth it is helpful to have maps to have numbers that help to in order to navigate the complexity there's also the issue of competition both globally and locally and the issue of the increasing societal demands in a time in which demands for public money to show that it's producing something universities are under pressure to show that they are doing something useful for society and do it in an efficient way so yes there are very good reasons and many reasons why indicators bibliometric indicators other types of indicators are important and are useful they increase the sense of objectivity when making decisions they reduce the complexity they reduce the time and cost of making decisions so indicators are useful now the problem might be that we end up believing that these indicators represent the whole reality within science and this is something that I would we can have a discussion with Emilio and we would disagree that everything that is in the web reflects what is happening in the world there is limitation to how much you can capture and how much you can look at it so there are many things happening which cannot be captured now the dream of rationality the science of science policy might help us in taking decisions because we have numbers now the question is whether we are making the right positions there are a number of people people like Daniel Saravitz Andrea Saltelli who are worried about the type of science that is being produced even magazines or newspapers such as The Economist in recent years have run articles asking about the problems in science the potential increase in fraud the potential increase in misuse and reproducibility a lack of reproducibility of the results so there is a mismatch between the expectations of what science could deliver and the outcomes of research this happening in a variety of areas in energy, in the environment, in health the digital economies and there is a perception that one of the problems is the discrepancy between the evaluation criteria in general and in particular the evaluation criteria which are behind the indicators and the social and economic functions of science so something that could be very obvious sometimes is forgotten which is that more publications more citations do not necessarily mean better science or science that is better for societal outcomes and let me present a parable that's a parable of the Prussian scientific forestry in the Enlightenment the Newtonian view of the world was dominant so in the 18th century the same way that Newton came up just 100 years before with this beautiful law to describe the world they also had a variety of approaches to better manage the world one of these was scientific forestry this was in Prussia and in a moment in which Wood was the main fuel the enlightened policy makers tried to manage forest in a way that would be perfectly rational in the end the rationality was about a single number they tried to increase the revenue yield of timber the amount of wood that you would get from the forest to do so they started counting the trees in the forest counting the different types of trees but in a forest like the one on the right hand side in a wild forest as they used to be it was very difficult to have a good reproducible yield so in the end they started to look by counting in terms of only the amount of timber and they came up with a simple abstraction of simplification which was understanding the forest only in terms of the trees forgetting about the birds, forgetting about the bushes forgetting about the people hunting in the forest people getting medicinal herbs for the forest and the way they view the forest the one on the right which was complex, diverse led us to transform it and they started planting rational forest which is the one on the left the rational forest would have perfectly aligned trees there would be no bushes and it was extremely good in terms of increasing the yield of timber the problem was that the soil was rich because of the diversity of trees and animals because of a variety of insects doing a variety of environmental services so they planted when they planted the rational forest they had a success in terms of increasing timber for one generation the second generation was not so good but in the second generation they started having plagues and lots of problems so what they had to do afterwards was to introduce diversity insects, animals artificially so as to revert a little bit to the original wild forest now this is a parable I think for what might be happening in science our focus in single measures might be reducing the diversity of activities and we might end up with forests which are not sustainable with science systems that are focused only in things that we think important nowadays but might not be important in 20 years time which might be important 20 years time we believe now that they are the little that they are unimportant the bushes in the original forest this is not something that happens only in science when in the beginning of the financial crisis there was a committee of experts including Stiglitz Amartya Sen, Fitusi actually presided by nobody else but Nicolas Sarkozy which said that we had the same problem with GDP that the focus in GDP was detrimental to generating well-being in societies now the question then is to ask ourselves what is it that we value do we value more publications, do we value more citations per se or are we genuinely interested in intellectual exchange and intellectual exchange in diverse areas and Emilia was saying that was asking whether indicators could do harm I think at this moment we have plenty of evidence, anecdotal evidence we don't have quantitative evidence that indicators can do harm in front of this in 2013 we started series of the liberations in the bibliometric conferences and we didn't really know how to go about fostering a better use of indicators in a way you know we felt bibliometricians we felt like nuclear engineers working with radioactive material and knowing that some countries in the world are using nuclear materials for not very useful purposes in places such as North Korea and the question is can we do, are we responsible at all to dealing with these nuclear materials so the idea was to have a set of principles about good uses of bibliometrics in a way of fostering a responsible use these 10 principles were published in 2015 in nature and they have had a wide reading and what they try what we realized in the debates we had in 2013-14 in the community was that we could not come up with a tight standardization of how bibliometric should be used this is because bibliometrics is used in a variety of ways in a variety of contexts so we could only come up with general principles and these are the principles which are very related to the ideas which are behind the metric tight which was a report published also in 2015 by Hefke being the high education funding council for England and Wales it's a very interesting thick document about good practices in the use of metrics the 10 principles were meant I have to confess one of the goals was that individual researchers when pressured by the head of the department or by evaluators could go hey we've got this document which is in nature which tells us we should not focus on these we can also do these on that the first one is that you cannot use indicators in a mechanistic way and that in the end evaluation, assessment should depend on expert judgment indicators can't help in informing the decisions but they cannot supplant the decisions this is a principle which is very simple but in the Iberian peninsula runs against the bureaucratic tradition of pretence of objectivity because of course the choice is always made by somebody about which indicators you use and depending on indicators there is value judgment so this is a very important principle in countries with bureaucratic with strong bureaucratic tradition such as those in Latin America and Southern Europe and Eastern Europe by the way the second and third principle are about context indicators should reflect the goals and the missions of the research so if we are doing research on health and we have a number of university hospital consortia in Spain which do fantastic research these consortia should not be evaluated according to their impact factor there's number of citations they should be evaluated about the outcomes that they produce in terms of the health they should also protect excellence in locally relevant research this is important given the biases that main databases have as was clearly shown by Emilio given the focus on Anglo-Saxon journals of the dominant databases it is important to remember that a lot of research which is relevant might not be in the databases and that a lot of relevant research and excellent research is published in journals that are not necessarily seen as important by the wider international community the fourth and fifth principles are about transparency so that the evaluation process is seen as fair and robust and besides being seen that actually that it can be fair and robust and that people can scrutinize it and propose alternatives the sixth, seven and eight are about variety about diversity is the fact that science is not a single endeavor but that science research thrives in diversity means that very often you have communities that do not share the ideas of what is important if you put sociologists and economists they will clearly disagree on what is good research if you put biologists and physicists they will often disagree in order to allow these these agreements in perspectives which is one of the strengths of science of diversity you need to be able to acknowledge different types of value as given by each of the different communities and these are not necessarily commensurable and these differences in terms of variety of outputs are particularly important when evaluating individual researchers because some people focus on certain types of communities and certain types of activities much more engage with stakeholders some other researchers are more focused towards the international community and you cannot compare these activities with the same indicators and finally the 10 the 9 and 10 principles are about the assessment being a social system in which the actors are capable of predicting what is happening in the system and reacting to it is what we call reflexivity given that we researchers are capable of changing our publications behavior it is important that we think what are the effects in terms of the behavior of the researchers and that we change the evolution in a way that is sensitive to the potential effects these type of principles were summarized in the metric types in big principles such as humility, sensitivity to context, transparency, diversity robustness and reflexivity time wise I should 5 minutes so in terms of I just wanted to get to the to the manifesto and then just to finish let me finish with a reflection of what type of effects the current focus on narrow indicators might be having so if you think of a space of problems in the world and you think of a space of research that is addressing part of those problems the current set of indicators is only focusing on a part of the research which is covering only part of the problems now the problem that we have when we are doing evaluation is that we mistake where there is the light whatever with all that exists in the world is the problem of the drunkard looking for the keys of his car below the streetlight so we mistake often the space of indicators by the space of problems and space of research and this happens in a variety of areas one of the areas which is very problematic is health there is a lot of research on therapeutic measures versus research on preventative medicine on understanding the environmental origins of lack of health poverty for example being one of the major reasons why people don't have good health but most of the research in medicine goes to therapies and we often don't know how we manage research we are not addressing problems outside of the light so the idea is that this focus in narrow indicators is contracting the space of research rather than this is happening in a moment in which there is a lot of rhetoric about the importance of broadening the impact of science in the world the issue about societal impact so if we have to expand the space of research the idea is that we will have to expand the area of indicators and in this sense I think the talk by Emilio showing that Google Scholar has this much coverage with a very clear illustration of how by looking at other sources of data you might expand the area covered by indicators and potentially come up with more nuanced and more beneficial types of evaluation so let me finish by the question I started with and I'm sorry being of Friday morning that is not a more cheerful question but the question is whether we are so interested in gaining our credentials for excellence that we forget what research what science is about thank you thank you very much Ismael I think that your presentation was thought-provoking quite interesting well we have only three minutes before the coffee break so I think that we are not on time to make now the question to ask questions to the speakers so please if you agree we are going to make now the coffee break and at the end of the round table you will have the opportunity to ask them your questions you will find some food for breakfast at the hallway of the auditorium please I must ask you to come back at 11 o'clock to restart the session thank you very much okay we are going to restart the session with the keynote speakers on my immediate left is Dr Karen Boyle and in my immediate right is Mr. Steve Smith Steve is our next speaker he is institutional sales manager in Kudos Steve has over 15 years of experience working in academic publishing and with online platforms and tools to support workflows of the researchers he has spent time at MNL Group Publishing ProQuest, OCLC and Tonson Reuters before joining Kudos in 2016 today he is going to speak about Kudos a platform for researchers, institutions and publishers to increase the outreach and impact of their research publication which provides unique mapping of specific promotional activities to increase in metrics his keynote is entitled maximizing reach and impact cases studies from Kudos yes so Steve whatever you want okay great can everybody hear me okay you know the first thing I'm really interested to know this is the first time I've come to Catalonia to talk about Kudos how many people have heard of Kudos could you just show your hands if you've heard of Kudos at least okay so it's not bad it's about a third of the room so it looks as though we've reached you already so hopefully by the end of this session at least 99% of you will have heard of Kudos anyway so yeah as Raphael introduced Kudos is all about boosting visibility and impact of your research publications so quality research needs to stand out in the crowd and get red and reach the people who are going to benefit from it so it's a little bit like this golden tulip in the field of lots of pink pale tulips we want your research to stand out and get noticed so hopefully it won't be too long before in the north of England where I'm from these little things to come above the surface of the soil still not there yet so what are we doing with Kudos it's a free platform for researchers to come in and explain in plain language what their research is about and why it's important and what we wanted to do with this platform so Kudos is a young company we've been going for about four years now our motivation was really to help universities research institutions and of course publishers to mobilize researchers actually to motivate researchers to do more outreach and promotional work about their research to increase their visibility and impact but what was our driver why did we think this was a good idea and why did we think that it would be important for researchers to start to do this and why really it's all about pressure pressure in a number of different areas so first of all there's pressure over research funding of course there are more and more researchers applying for funding for the same pots of money the money isn't particularly increasing and so to be able to differentiate yourself and to demonstrate to funders or potential funders that you've got good past performance of generating impact for your research that you're reaching the people who will really benefit from the research not just that you've got a lot of citations but that your research is actually making a difference to the world demonstrating past performance but also showing these potential funders on your application form that you've got sensible pathways to generating the impact have you got a plan, have you got a strategy a clear strategy for after you've spent their research money and you've created your publication will you be disseminating it to the right places and will you be following it up and doing some outreach activity so increasingly actually Kudos is being cited by researchers on funding applications as one of the pieces of evidence that's being used to say yes we do have a plan to disseminate this research we have a growing impact agenda of course just across the board so obviously Karen is going to talk about the REF that we have in the UK our research assessment exercise or research excellence framework and no doubt you have similar things locally but it's not going away these bodies are going to require evidence of impact so when it's coming to assessment time being able to demonstrate this is going to be important just the huge growth in outputs so simply demographics and the fact that more papers are being published now than ever before that's not really surprising but there's just simply a big huge wealth of content out there actually when you're publishing your latest piece of research you're competing with many many more people than you ever needed to before so again taking steps to making sure that your publication stands out is clearly going to be important so we're helping with this fight for visibility and usage and then we've got other issues which are perhaps more pertinent to publishers so how many publishers or editors have we got in the room quite a fair few so I mean to be able to tell me how important the issue of off-grid sharing is to you I'm talking about platforms such as ResearchGate or academia.edu and your authors uploading and sharing their full text which they may not have the copyright to do and we want to help to avoid this situation as well puts people into a grey area which we don't really want them to be in so Kudos provides links to the publishers website so you get more traffic going to the right places not just you know for potential revenue purposes for the publisher when people want to download your full text but equally to make sure that usage is properly monitored and measured and collected as well as providing a link to the publishers website how many librarians are there in the room okay also good fair few librarians some of whom I met last night which was very pleasurable thank you very much for that but librarians tend to be the custodians of the institutional repository and this is of course a great resource which has the potential to be used much more in the context of generating impact and communicating research it's not just a box where papers are kept and collected and you can access them there if you want there's potential for the institutional repository to be part of the whole communications piece and so Kudos also allows you to link to the full text which might be available on green open access on the institutional repository so what are we working to achieve we want more impact for research more recognition for researchers but we also need to provide much better evidence to convince researchers really and how many researchers are in the room sorry we haven't asked about researchers how many researchers most of you are researchers as researchers you're probably thinking well Kudos so it's a free platform okay researchgate academia.edu Mendeley these were also free platforms so you're giving me a fourth platform are you Steve for me to put a profile on and start sharing my work thank you very much Mr Smith no it's not that so what we want to do is really show you that if you've got five or ten minutes available to promote your work that it does actually make a difference and that your metrics are increasing so if you explain your paper using kudos on the Monday and then you send that out by Twitter it's now Friday lunchtime nearly and you're going to be curious as to that five or ten minutes that you spent on Monday morning how many more readers have I got now have I got more readers are people tweeting about me now as a result of my promotional activity I probably haven't got any more citations I might have to wait a little bit longer for that but you want to be convinced that it's really worth spending your time so part of the kudos platform is showing you a wide range of metrics and how those map up against the activities that you've done better collaboration as well we also want to help to foster better collaboration within a university or at a publishing house I would say that at a publishing house it's easier because you've got your authors you've got your editorial staff and you have your marketing team and so it's quite easy isn't it really in the context of a journal or a group of journals for the marketing team to be able to access the content which is being published by you as a publisher and then share that out via social media to really disseminate it it's quite a sort of I would say a flat structure within a publisher what's much more difficult to navigate however is the structure that you've got at a typical university so if you think about the different stakeholders who could who could have some influence and could lead to or contribute to the success of a combined communication and impact generation effort within a university who have you got you've got the researcher so the researcher is great at being able to you know explain in plain language what their research is about and why it was important they're the best person to do that and indeed in kudos that's what happens you claim your paper you explain it in plain language however what about social media networks and then sharing your explanation who's the best person to do that it's not always the researcher is it because you might be great at research but you just don't have time actually to go into Twitter and Facebook and LinkedIn and Instagram and all these other places constantly sharing things and making sure that you've got a growing network and that they're all engaged all of your followers and so on so you might not actually be the best person to share your own research departments of communication ah so you've got departments of communication at pretty much every university or marketing PR press whatever they happen to be called they could be useful couldn't they because they do have these big communication networks they do have these big social media networks with 20 or 30,000 people following the Twitter accounts so they could be quite useful at least controlling the top level of institution-wide social media accounts but they also might not be the best people because if you look at who those 20 or 30,000 followers are they might not really be that interested in your very niche area of research so we need to find the social media channels where there is a receptive audience the followers are interested in your research in your discipline and also that there's a lot of followers maybe more followers actually than you have personally for your own personal Twitter account so what we have in Kudos is this platform where universities can access the plain language explanations that you're writing and then appoint administrators across the university so they might be people for example who are running a departmental or subject based or a research group Twitter account they can see your explanation and then they can tweet that out to their much bigger and ready to consume audience bringing this all together is not easy but Kudos at least provides a platform to make that at least a possibility and then the final part the final stakeholders are perhaps the research office or the people who are looking at the metrics you know how many more citations are we getting or are we starting to look at altmetrics are we starting to consider how much attention our work is getting aside from citations because this could be for example a pre-indicator as to where we're starting to have some make some differences where we're starting to generate different types of impact before we have the evidence from things like Scopus or Web of Science or even Google Scholar and their citations and who are those people they might be in the library they might be in the research office it's handled by different people but also bringing them into the picture so that they can see that the communication efforts that these teams are making between them are actually leading to material gains in measurable metrics measurable metrics so how do we do it so I'm going to actually rather than talking about this slide I'm going to show you what this looks like a little bit later now it's all very well me saying we have this new platform and you use it to explain your papers and share it and you're going to get more impact and you're going to find many more readers and so on that's just my claim as a supplier really as Kudos but we have had this independently tested by a group of people in the altmetrics team at Nanyang Technological University in Singapore if you don't know that institution the altmetrics team there is very well respected it's quite a leader worldwide in the area of altmetrics and what they did is they took a sample size of 4,000 academic papers that had been explained and shared using the Kudos tools and then took a similar control group of 4,000 papers which hadn't and what they found, well there's no secret because it's there on the screen, what they found was that the papers that had been explained using the Kudos tools got 23% more full text downloads than those that didn't so that was the official study we do hear from some of our publisher partners for example Emerald who I used to work for who tell us that for certain journals they're seeing more like a five fold increase in full text downloads when Kudos is used to explain their publications so the evidence seems to point to the fact that Kudos does work and that was very good validation really for the around about three years of work that we did building the platform to that point because until that point we couldn't be completely sure how effective Kudos was so we're starting to see some evidence now so what's the workflow then so I'm going to show you what the workflow is for the researcher starting up with a new Kudos account so I don't know if you want to take any photos here or if you want to scribble this website down because I'm going to give you something for free right now your own Kudos account so you can do this you don't need a publisher or a university to back you can just do this yourself you go to growkudos.com can you see that down there bottom left growkudos.com and then you register yourself an account there it takes about 30 seconds to complete the registration form and then when you click on the confirmation link in your email then you're ready to start promoting your papers so number two is find a work so you then want to find the piece of research find the paper that you really want to promote right now and that's how it's different from things like research gates or academia.edu this is not an online profile site at all this is a tool kit to promote a specific piece of research so what you don't do with Kudos is register and then claim every paper that you ever written or co-authored you don't do that, please don't do that when you think about it I mean if you started explaining a publication which you had published six or seven years ago and it doesn't have any particular relevance in today's world actually your Twitter followers might be confused as to why you're suddenly starting to promote this old paper so start with the most obvious paper that you want to promote probably your most recent publication for most people so most people will start off just using Kudos once for one paper to promote that it may be the case that you had a paper published a couple of years ago and it just didn't have the kind of reaction from the community that you expected and you were disappointed and certainly your funder was disappointed right so you might want to go back find that paper and then give it the Kudos treatment as well but this is absolutely not an online profile site just to make that clear having said that we do have a partnership with Orkid that's that green symbol up there how many people have got on Orkid? okay a very well trained audience so if you've got on Orkid and you've already claimed all of your papers or some of your papers in your Orkid account then when you register with Kudos you simply add in your Orkid ID and then any papers that you've claimed as long as they have a cross ref DOI will be there available in your Kudos account ready for you to promote so when you've done that the next thing number three this is let's say this is the hard work this is the hard graft number three explain your research now most of the people in this room they'll be working in the area of communications and media studies right is that right? okay is there anyone working in any other disciplines here? actually yeah okay it's kind of cheating but I know what you mean is my but communications and media studies so for the most part for the most part if you think about the title and abstract that you're submitting for your paper it doesn't generally contain language so technical that's a non-expert could completely fail to understand the purpose of your research is that fair for me to say? yeah I mean there might be one or two occasions where you are using some technical terms which people from outside communication studies would fail to comprehend but anyway this is your chance regardless of which discipline you're in to put your research into plain language so that indeed a non-expert could understand so this would perhaps admittedly be more pertinent to let's say a life science or natural sciences audience or medical sciences audience putting it into plain language so that everybody can understand because the purpose here is not just to reach people in communication sciences but to a much wider audience to the public anyone who could benefit from your research once you've done that you might also want to add some other resources so if you've got video links or if you've got data sets that are relevant to your research or if you've got an infographic or other links then you can also link at the explanation page and sort of create a hub if you like an online hub of all the resources that exist about that research right then the next bit is the exciting bit really because this is when you get to share so number four is sharing so this is where you link up your Twitter, Facebook and LinkedIn accounts if you've got them and you share that explanation page so that's what people are going to read when they click through on the link so you want to make sure that you do a really good job of number three if you want to share on ResearchGate or academia.edu or Mendeley then that's absolutely possible through Kudos you simply generate a link and then copy the link and paste it to those places if you've already got a good network in these other sites you should also use it there's no reason why not to the interesting thing is that wherever you decide to share so Twitter, Facebook, LinkedIn or these other places ResearchGate and so on wherever you decide to share you're going to get feedback in the form of metrics as to which channel so which social media channel which of these activities led to the biggest increase in new readers or new people interacting with your research so that mapping that correlation between activities, promotional activities and increasing metrics that's what really makes Kudos unique okay let's move on so here's an example this is what a Kudos page looks like you've got the original title and a link to the article at the top you can see this is from a Royal Society Journal and underneath you've got the what it's about and why it's important text so this is what the researcher is writing sometimes people ask me about Kudos oh so have you invented some kind of special linguistic algorithm to create by machine a plain language explanation from my abstract and the answer is no we haven't and we're not going to try to do that either so the researcher is always going to have to do this bit I actually think it's quite good that robots can't completely replace humans in every function of our life don't you I certainly do you can also see that this person is on the right hand side added some resources so one of the resources you could add and I'm just going to embarrass somebody in the room at the moment where's Mireia where's Mireia so Mireia has used Kudos this is her real life example so hopefully you can see that so this is one of her first efforts at using Kudos so a nice plain language explanation and also a personal perspective but look what she's also done she's added on the right hand side here knowing that the people who click on this link from her Twitter feed might not have subscription access to the journal in question but it's held on the UPF institutional repository here so if I don't want to spend 30 or 40 euros reading the full text I can perhaps access it here from the UPF repository so that's quite a nice thing to be able to do there's the original article so she would then share it that's what the share screen looks like then she's got her metrics so she can keep an eye on these to see how these are progressing so what have you got there you've got the number of times that it's been shared the number of clicks on the shares the views on Kudos so this number is sometimes bigger than this number because the search engines are also crawling Kudos pages and when you think about it when you're writing your plain language explanation you're going to usually generate more language more let's say lay terms about your research than you did in your original abstract so it's possible that a user of Google could discover your work without using the kind of technical search terms in their Google search so it potentially makes it a bit more discoverable as well you've got the click through to read the publication you've got the altmetric score how many people are familiar with altmetric this multicolored donut not that many the thing is I don't have much time to talk about another company's product and it is another company's product but we have a partnership with altmetric what altmetric does is very briefly it aggregates a number of different indicators not only from social media but also have you been cited on a Wikipedia media page or has your work been cited in a government document policy are you mentioned in any newspaper websites or blogs on the web in all different languages so this altmetric company tracks all of these mentions about your publication across this really wide range of different media so it's really interesting actually to be able to see that in the context of in the context of your sharing how this number is increasing and then also to be able to see exactly who is tweeting about you so look at this already Mireya has had all of these different people tweeting about her work not just herself but you know all of these different people and actually if you're on this screen if you're on the share screen here and you've just done your plain language explanation and you don't really know where I should be sharing it then actually the altmetric screen could give you some inspiration about where you could share it or you know which which hashtags that you could use if another Twitter user is using particular hashtags to talk about your work maybe you could experiment and use those hashtags also so actually I'll just show you a little bit more how am I doing for time Raphael two minutes okay so back to the metric screen I've also got this detailed metrics which I just wanted to show you so this is where you've got that mapping that correlation between the activity that she's done to promote her work and the increasing metrics so what you can see along the bottom is the date and along the y-axis you've got the metrics if we've got it available we add the full text downloads from the publisher so the publisher sends us this information and we can add the full text downloads as well in this case what we've got is information about what Mireya has done so she's claimed the paper there and at this point here this is when she shared it via social media as well as adding this explanation text so you can see that at this point the metrics increase the views on kudos increase the click-throughs to the kudos page increase and then we add the full text downloads on top of that so when I'm talking about motivating the researchers to do something to increase their outreach this is what I'm talking about being able to show them that by putting a few minutes effort in it really does start to to pay off okay I did have some more material I'm running out of time though maybe I can give that to you over lunch or later this afternoon but in the meantime thank you very much okay so thank you Steve thank you very much for your presentation it was quite interesting invite to try kudos it was very nice okay so now well now last but not least we have Dr Karen Boyle she is professor of feminist media studies research director in communications media and culture at the University of Stirling in UK in the 2014 research excellent framework Karen was a joint output assessor for panel 35 that includes the areas of music, drama dance and performing arts in panel 36 including communications, cultural and media studies and library and information management she sits on the executive of MEXA the UK accepted association for media communication and cultural studies and she is on the editorial boards of feminist media studies and sexualization media and society she regularly undertakes peer review for a wide range of journals and publishers as well as for other academic institutions in relation to her research interest she has largely focused on gender, violence and presentation also she is author of media and violence gender in the debates editor of everyday pornography and she has published widely on all these things in academic journals and edited collections Dr Karen Boyle will present now the second keynote entitled publishing in for breath in the communication field in the UK Thank you, thanks for inviting me to be here today a few caveats before I start what I'm going to try and do today is talk through a little bit what breath was what it's likely to be as far as we know at this point which is somewhat up in the air but I'm going to try and focus on what it means for communications journals and those of us who publish in them but my big health warning on this is that I'm not here to cheerlead for the breath process and say that this is the way that everyone else should do it very far from it we were discussing over coffee that many of us who are involved in these processes find that we're in the position of trying to advocate the least worst solution and certainly that's a discussion we're going through in the UK very much at the moment largely hinging on how evaluation will take place in the next exercise the role of peer review and to an extent the role of metrics so I will talk about that only very briefly what I want to try and get to at the end of this talk is to gesture towards what we're trying to do at Sterling and in other organisations where I've been undertaking peer review as part of our breath preparation process which is to try to think about how we might use some of the quality judgments that drive breath in a way that's hopefully much more individually and personally constructive than the process often feels so just very broadly to start with then breath which other speakers have gestured towards already is a means of assessing the quality of research in UK universities it's been going for a long time or at least it feels like it's been going for a long time formerly as the research assessment exercise and it takes place every six to eight years the last exercise was in 2014 breath assesses quality but it does so for the purposes of determining the distribution of government funding so it's by no means a neutral process and it's one that universities invest a huge amount of money and anxiety in shall we say as you can see from the panel titles that are on my slide and in the introduction the breath assessment process is organized around very broad subject groupings and for fields like ours there are very multidisciplinary this can present a number of challenges which I'm happy to talk about in questions if that's helpful universities choose which panels they submit to at least they do currently and which individuals to include within the submission so it's a selective submission process or at least I think it was in 2014 my own role in the last exercise was primarily as an output assessor which meant that I was reading the work that my peers produce monographs, journal articles book chapters but also and although I won't talk about this much I'm happy to answer questions about it in the two fields music, drama dance and performing arts and communications media, cultural studies, library and information management there were a number of practice based submissions so people submitting practices research, films, databases photography other forms of production so that's maybe something we can think about in relation to the challenges that pose for things like some of the metrics models we've heard about today I assessed around 450 outputs in the last exercise and that was in a period of about 8 months so you can imagine that was quite time consuming and that's what I'm going to try and draw on today so I want to first of all just give you a very little bit of background on the last exercise if I can work out how to actually advance this is it just on the there we go okay so the last exercise then the definition of research that's used by the ref is up there on the screen and there's three elements that were evaluated in our process of evaluating the quality of research in UK institutions and they were outputs which I'm going to focus on today which made up 65% of the score for a unit of assessment submission impact which was new in 2014 and environment and the words in italics are the key words that were used to make quality judgments in each of these areas I would just like to pick up on something about impact because I think impact we use that term in a number of different ways in academia and impact in the way that Steve was talking about it in terms of dissemination how many people read your work is not what was measured in the ref exercise to demonstrate impact in the ref exercise you had to be able to demonstrate almost like a change in behaviour or a change in the way something was thought about so impact as a quality measure is not about the quantity of your dissemination and I think that's quite an important caveat here the other really important thing to stress about the ref process is that it was and the proposal for next time is still will be largely based on peer review so for outputs everything was read by two reviewers and we could bring in a third if there was disagreement and two really important points to note here for all of the panels that fell within humanities is that the type of output was not taken into account in any formal way so we weren't asked to look at which journal something appeared in at the impact rating of that journal or the number of citations a piece had we were looking literally at a piece of work itself so metrics was not a formal part of the process we had access to metrics data if we wanted it but our panels the panels I was involved in did not use metrics data okay so just very briefly what did we find well I suppose the good news the headline good news story is that we found that our field is a field in rude health and I'll talk through these categories of 4, 3, 2, 1 star and unclassified mean in a moment but 4 stars the highest quality rating and we found that 29% of all submissions or 23.4% of outputs were at that top level and when I show you the descriptors you'll see that that is really quite a high judgment to make there were 67 submissions to panel 36 which is the one that I imagine most people here would be interested in and you can see there the number of outputs we assessed I've put a quote there from the report that went back to the sector from panel 36 because I wanted to pull out two points about that two points from that quotation the first thing to say is that all kinds of output were found to exist in all of the quality levels so we assessed 4 star films unclassified books 4 star journals unclassified journal articles so everything fell at every quality level that said one thing that panel 36 in particular found and this is interesting in terms of thinking about where we might place our work the shorter pieces of research is that book chapters came off the worst by quite some way now of course there were book chapters that were 4 star but on the whole why I'd say book chapters came off the worst and I'd reflect on my own work in a very similar way is that often there's a less rigorous peer review process not always but often I think one of the key reasons that book chapters can fall foul of ref is that the book chapters are read on their own and sometimes they're less self-explanatory about their significance in the context of an edited collection and they might be within a standalone journal article because the introduction written by somebody else makes that case for the significance and originality of the work and so that's something that I'm kind of increasingly discussing with colleagues as they choose where to place their work the other challenge I think we found with book chapters is that people were often kind of trying to shoehorn their own research into somebody else's big idea and so that could be a less comfortable fit sometimes but also book chapters on the whole were shorter than journal articles and although length is not a prerequisite of rigor and there are certain sort of more scientific areas within communications where very short pieces could score very highly indeed as a general rule it's much more difficult to attain the levels of quality in short pieces and so I'll just take you through now what those quality levels are so these are the overarching descriptors that were published for what four star, three star two star, one star and unclassified work what that work was two things I want to say about this firstly that they are exacting criteria if we judge something as four star we are saying it is world leading in terms of originality significance and rigor the other point I want to make about this is that the funding really shapes the way these judgments are made because last time the cutoff for funding was at three star so only work deemed three star and above was deemed fundable and so although we are making quality judgments we are also making quality judgments incognizance of this broader funding context I want to take you through now the slightly more detailed quality descriptors because I think these are useful in trying to just give a sense of how we define quality and perhaps sometimes why that kind of quality can't be easily metricised this is the detailed descriptor of four star and I think you will see kind of immediately from reading this that not all research can or should be four star and indeed we all publish research for a variety of different audiences purposes and so on and we should still be able to do that there were certainly journal articles that were graded at four star but if you think about the description of research offering a major expansion of the range and depth of research and its application you can see that that kind of judgment can be quite difficult to achieve in shorter pieces of seven to ten thousand words but I also think it's really important to state because certainly in some areas in the UK you get this kind of oh you've got to publish a monograph it's got to be a monograph, you need a monograph that's going to be four star I read monographs that were judged at one and two stars absolutely no guarantee that writing a monograph is going to in itself ensure that you reach these quality descriptors because two and three were really where the key decisions happened because they were the places where the decision was about funding I thought it would be useful to really run through what we thought made a journal article for instance three star and not two star because this is what I think institutions are most exercised about now how do we ensure that we concentrate our work in three and four star and probably the first key thing to say there is to remember that in this process those of us who were reviewing work were assessing the output and nothing else so when it comes to criteria like determining whether it's a recognised or important point of reference if it's had influence what we're looking at here is whether it deserves to be those things not whether there are metrics that prove it has been these things and that's a really really important distinction and it's necessary because of the census period that the REF covers so in the last exercise we looked at work that was published between 2008 and the end of 2013 so material that I was reading may have literally only been published a month ago so the idea that we could use metrics as a way of kind of tracking the quality of that is obviously deeply troubling for me another key distinguisher around the two-three border line was whether work was thought to be a catalyst to new thinking or new approaches or seemed to be an incremental and cumulative advance with the incremental and cumulative advance being for me really the very definition of two-star work so two-star work is still quality that's recognised internationally it's high quality work but it tends to perhaps be a bit more focused, a bit more modest in its claims I just want to say something about recognised internationally and recognised nationally because I think it picks up on the point that Ismail made earlier about locally relevant research it's absolutely not the case that to achieve the highest quality level something had to be international in scope and comparative in scope so for instance you could have a piece of work on Welsh language television that's very locally specific that still achieved those highest standards because it was absolutely achieving all these criteria in the context of its own field and that might be quite a small field working that small field that achieved the highest standards of four-star would be more likely to gesture to broader debates so in the example of Welsh language television it might gesture to broader debates about media and minority languages or it might engage with broader debates about public service broadcasting for instance so it could still be making a contribution to debates internationally about public service broadcasting although its focus was really quite specific okay so that's where we were where we are is a matter of some speculation at the moment we're undergoing a review and there's a consultation published by Lord Stern which is out at the moment closes next month and the link's there if anyone wants to look at it so there's a number of proposals about the next exercise but for our discussion today I think the most important of these are the outputs are still going to be the single most significant factor so the search quality of institutions is still going to largely be determined by outputs there are other aspects there I can talk through if people are interested but two other things here is that the process is still going to be based on peer review largely based on the metric TAID which was mentioned earlier Stern is proposing only to use metrics in an advisory capacity if panels wish to do so so the likelihood is that for our subject areas metrics will not be used in a heavy handed way at all the other thing which has already come in is an open access requirement however and that relates partly to some of what Steve was talking about and I'll pick up on that so the idea of sharing and disseminating research being really key moving forward okay so just to wrap up I want to sort of run through what I learned from reading 450 outputs or watching some of them or interacting with some of them and what I'm trying to kind of use this to think about my own writing but also the work of my colleagues one thing it's just worth saying at the beginning is the editorial work in and of itself is very difficult to claim as research when I moved institutions when I went to Sterling one of my last publications was an edited volume and I had fully intended to submit the entire edited volume to REF and was strongly advised by one of my colleagues not to do so and to submit a journal article instead and considering the amount of time my edited book had taken versus the journal article I felt a bit sick but actually he was completely right because of course I was making a judgment about the value of the editorial collection that wasn't necessarily based on my work it was based on the quality of other people's research contributing to that so had it been judged for me the person wouldn't have read the whole book they'd have read my chapter in it and I did have journal articles that were stronger than that so editorial work is important that kind of works important for our research environment but it's it doesn't count very well in REF I'd say that at the moment where we're heading is the open access is more important than journal rankings so not all journals in our field do have impact factors and whilst there's going to talk about screen later screens one of those that doesn't a very very prestigious journal in film studies so I would say that open access and ease of access to material is going to be more important than the journal rankings but within that although where something matters where something's published doesn't officially matter the reviewers are your peers the reviewers are people in this room and so I suppose we do unconsciously or consciously bring to the table understandings of the reputation of particular journals of whether something does or doesn't fit within them and that might impact on the way we think about significance I'd also say that journals and this is one of the reasons again why book chapters didn't always do so well is that journals have been through a rigorous process of peer review and as you can imagine if I read 450 outputs I wasn't reading outputs that were all things in which I would classify myself as expert so knowing that something has already been peer reviewed by people who are expert in that field is some guarantor of quality even things like the quality of copy editing started to sneak in really badly edited books really got on my nerves but I think the other thing that I've learned about journals is that doing peer review really helps you understand peer review and so when and where you've got the opportunities to engage in peer review I know it's one of those academic citizenship jobs that we don't really get credit for and we do for free to keep journals going having said that I think it's a really good way to learn about how you write and how you present your own research but also to be aware of the different criteria and of value so the criteria and a journal are using won't be the same as REF and just to have those different criteria in mind and then just two kind of final points sorry I don't quite know how I did that I don't know how that came up but there we go what I've been discussing with college recently is what's the magic formula how do we write with REF in mind and I think the thing is that there is no magic formula variability is very key some journal articles won't ever be REF-able and I would be really really sad if universities moved to a context where they simply weren't encouraged and weren't supported sometimes we need to be able to write commentary and criticism pieces in a really quick way to respond to really important current issues the fact that they're not based on ten years of research evidently that all that isn't visible in the two, three thousand words shouldn't stop us engaging in that process I think one of the things I've found we've just gone through a mock-ref process at Sterling and when I've been giving colleagues feedback that they've had from external readers one of the things that they'd often say is a really good case study but it doesn't really gesture towards a broader debate and so we're seeing it as two star not three and the colleague would say to me yes but what it does is it advances the debate in this and this and this and I'd say okay does the article say that and they'd go oh well no okay so that's what the article needs to say so we actually need to think about the way we write our abstracts our introductions our conclusions so that they're actually clear not just to the six other people in the world who really know your area very very well but to the six thousand other people in the world who are interested in the broad parameters of what you're saying and that also I think means that you've got to think about quality rather than quantity of outputs from a specific research project one of the deputy principals in our university talks about salami slicing of outputs and being really against the salami slicing of a research project into lots of slices and I think I mean as a reader not as a reviewer but as a reader I find it massively frustrating when I read oh yes but if you want to know about this bit of the research read the other article and often that can really undermine the score of an individual piece for originality or significance because you know third of the article is saying exactly the same as the other article over there and so really thinking about making that one really valuable contribution rather than maybe three more middling ones although I do understand that that's often at attention with priorities around recruitment and promotion so it's not an easy fit and the other thing that I learned from talking to colleagues about the peer review that they received was the importance of benefiting from peer review but not allowing it to distort fundamentally what you want to say and I think that's really important because we've got to have integrity as scholars but also if peer reviews coming back saying well this and you know you get often anyone who submitted to a journal will know this you can get two reviews one of which says this is absolutely perfect please don't change a thing apart from the common page three and another review that comes back two or three pages saying well really I wanted you to write this article and you haven't written this article and it's just about making sure you find your own voice I think in terms of how I now advise colleagues to think about where they publish fit is key timeliness and I think this is where in certain subjects within our field we really need to think about balancing the rigor of a long peer review process with the timeliness of publication and I think that can be really key in areas like journalism studies I was having this conversation with a colleague recently who'd written a really important piece to do with journalism but had actually published it as journal as pieces of journalism sort of for a couple of years before he then consolidated it into the academic article which meant that the academic article was no longer original and didn't have the same kind of reach as it could have had otherwise so timeliness I think is really important open access requirements I've mentioned those they're not going away ensuring that we meet them I think that also means that the attitude towards online only journals is changing I think maybe in the not last but process before there was a kind of snootiness towards online journals that I think is going and I think it's going largely because much more established scholars are now publishing in those journals as well so if you're trying to decide about an online journal looking at who's published there already is no bad idea space how many words does this publication need if it really is a 10,000 word article don't write a monograph I think that's the thing I'd most like to transform in academic cultures is that we all write less more succinctly and more powerfully but that's based on the experience of reading 450 outputs of which about 80 were monographs and then also finally just in that special issues I think are incredibly valuable often we want to be in special issues because we want to be visible in our field we want to be shaping a specific debate but sometimes they can also mean a bit like the book chapter that we end up having to shape our research to someone else's agenda and so that's just something I think to be cautious of so that's what we've learned from the process I'm absolutely not cheerleading for it or saying this is the model that we should have but trying to find a way of working within the model that we do have so I'll be happy to take any questions