 Okay. Good. Good morning everyone. Could we all be quiet please? Good morning everyone. My name is Jens Aral Osam. I am a senior advisor at the License and Open Access team with what was previously known as Kristin, which has now become part of a new organization called CERES. In fact, it's so new that we still had to use the old Kristin logo. And I'm here to welcome you all to Oslo to the seventh in an ongoing series of open-air workshops tackling issues surrounding open access and open science. And the theme of today's workshop is impact and measurements of open access. And we have prepared two sessions for you today. The first session will start with measuring open access and the second session will move more towards impact of open access both on the research community and society in general. And we also hope to give you a glimpse of the issues surrounding measurements of the impact of open access as well. Now, first of all, I would just like to remind everyone present that this is being recorded and streamed live across the internet. So please do ask questions to our presenters, but wait until a microphone has reached you so that everyone can hear you and hear your question, not just those fortunate enough to be present here today. And the last thing is, before you ask a question, please state your name and institution. Thank you. Now, enough of me here to open today's workshop and give you a proper Norwegian welcome. He is the rector of the institution Generous Enough to host today's workshop, Ushla Nakashus University College of Applied Sciences. And he was the previous head of the board of Kristin, the last as well. The first and the last, the only. Kurt Reis. Thank you. Good morning, everybody, and welcome again to Ushla and to Ushla Nakashus University College of Applied Sciences. It's a privilege and a pleasure for us to host such a distinguished crowd here who are engaged in very important discussions about open access. It's often the case that guys like me, rectors of host institutions, come up here and clear their throats for a couple of minutes and then let you get down to serious business. But in fact, open access is a topic that I care deeply and passionately about. And so I asked the organizers if I could have a few minutes to share some of my reflections with you on the state of the issue. And I don't know very much about what you put into the concepts of impact and measurement, but I would like to say a little bit about what I do with a special focus on impact. Open access, as I see it, is a classic example of an opportunity for disrupting a field, for disrupting a set of behaviors, for disrupting a system. And for the most part, I would say that we have failed over the last 10 or 15 years to exploit that opportunity in a way that has meaningfully improved the activity of scientific publication. That is a tragedy in my opinion, but it's also academically interesting and it deserves our study and our deeper understanding. And we might ask ourselves, I'll say a little bit more about what kinds of disruptions I would have liked to have seen, but first we might ask ourselves a bit about why it is that an opportunity for profound disruption, which is so clearly needed, has failed. And there's enough blame to go around and I don't want to park it all in one place, but certainly a strong factor in our failure to convert the opportunities on the opportunities of open access is a lack of political leadership. In the case of Norway, we have a very, very simple way to change all scientific publication to open access tomorrow. And that simple way, you can all predict. It's because we have this system where financing of institutions is in part based on publication activity. We have this elaborate point system ensconced and managed by Kristin. And the government could easily say that we only will reward publication in open access journals. It would be super easy for them to say that. It would be super easy to implement. It would have a profound effect on the nature of publication. The failure of the government then to make use of that opportunity has to be, one can only speculate on that. Is it the case that the commitment to open access isn't as strong as they actually say? Is it nervousness about being a small country taking such a strong position? There's some indication that that is the case. The situation in Norway has improved some over the last couple of years and the political leadership themselves say that that's a result of a meeting that they had with the Dutch Secretary of State for Higher Education Research Sonderdeket. So we want to cheer the Dutch and if any of you here are from the Netherlands, please know that we support you in the efforts that you're making and not only that, we appreciate the influence that you give our politicians. So part of the failure to convert the opportunity of open access is due to a lack of political will and a lack of political leadership. Another part of it, of course, is that it's an incredible demonstration of the power of monopolies and publishers are in a monopoly position and they exploit that in just the way that every economist tells us that monopolists will exploit their position and that's a very, very strong position. They have used that strength in part to negatively impact the opportunity for genuine disruption of the publishing industry. And the third factor that I'd like to point out that I believe is a very important factor when trying to understand the failed disruptiveness of open access is the prestige that we in the professorate assign to certain for-profit monopolistic journals and that we have been unwilling to sacrifice or to reassign to open access journals and that gets in the way of making much progress. The disruptive opportunity of open access is, of course, related to the core intention of publication and there are only two of those and you all know what they are and the reason that scientists publish their work is one to get it distributed that is read by others to make an impact on the research that others are doing or on the application of research. So distribution is the one and the other one is as part of our quality control system and it's especially the second one where the connection between open access and quality control is not immediate or superficial but that's exactly the kind of thing where a genuine disruption of an industry creates the opportunity to do better at also the parts that aren't immediately tied to it. So we might think of open access as being first and foremost about changing the way that we do distribution and that would be a very sensible thing to think but the disruption that could be created, that could have been created by doing that in a different way would have opened the door to changing other aspects of the publishing enterprise for example the quality control aspect of it and we all know that that is in deep crisis. There's the crisis of reproducibility, there's the crisis of faked research there's the crisis of exploding retraction rates there's a genuine and deep and profound crisis of quality control in scientific publishing and that feeds wacky conclusions by politicians so there's a senior politician here in Norway who now has decided that all the research on climate change is something that he doesn't have to care about and there's this complete nut job in the White House who has as a matter of principle that he's not going to hold himself accountable to any kind of scientific research and that is in part a result of eroding public confidence in science which is in part rational because we have a crisis of quality control and that is something that is going to have terrible consequences for policy which in turn will have terrible consequences for society and for the standard of living if we don't do something about it and we had the chance to profoundly disrupt our scientific publishing industry and we didn't make it happen but maybe there's still hope I sometimes have that hope I see that finally there are some experiments with changing the quality control system with open reviews with more public discussion along the way at various versions of papers with the push for open data that will make it easier for other scientists to try to reproduce the work of their colleagues so in some ways maybe there is hope but we've squandered 15 years and that is in my opinion a tragedy part of the political context for this in Norway as I mentioned is this point system that we have and that I'm increasingly convinced has an unfortunate symbiotic relationship to monopolistic for-profit publishers like Elsevier who are in the business of trying to break the backs of publicly financed universities and I'd like to very briefly explain that we have this two level system where some journals are considered you get more points for publishing in some journals than in others and what I would like for somebody to do the research on is to find out if there are any publishers who are over-represented at level two compared to level one and if that is the case that has to constitute an act on the part of the Norwegian government that gives an unfair competitive advantage to those particular publishing houses and I'd like to just make a prediction that Elsevier is over-represented at level two compared to how it's represented at level one and if that is the case then as I said that's an unfair competitive advantage given to Elsevier which our government should definitely not be in the business of and then the skeptic might say but what if they actually have more good journals than other publishers do to which I would say we know that the concept of a good journal doesn't make any sense we know that there is no correlation between the quality of the work published in high visibility journals and low visibility journals in fact we know that there might be reasons to think that the work published in high visibility journals is of lower quality than the work published in low visibility journals for example based on the higher rate of attraction that we see in high visibility journals so the very concept, the concept of a good paper and a bad paper is clear the concept of a good journal and a bad journal is not at all clear yet in Norway we have a financing system that necessarily builds on the concept of good journals and bad journals and probably gives an unfair competitive advantage to certain large monopolists and that's all I have time to say so I want you to have a really good time here at your meeting and I wish you well and I hope that you come up with all sorts of new insights about ways to really exploit the opportunity and promise of open access thank you okay thank you Kurt for engaging and inspiring opening to this workshop our next speaker is Natalia Manoula from the University of Athens and Athena she is the project director of the open-air project and she will give us an introduction to the workshop impact and measurement of open access okay but hello before I start I just wanted to say to everyone that today is the 15th year anniversary of the Budabest Acclaration and as we heard before is that we all have to think and reflect on the disruptions we've made I think we all wish that we would have made a bigger disruption but we are getting there let's see how we do that so what we will present today what the workshop is today is one of the series of workshops from open-air I have to now advertise that our next workshop will be in RDA 4th of April about legal issues so it would be good to have you there and it's about impact and measurement of open access whatever that means it means different things to many people so the first is about monitoring open access and what I would like to see is in order to monitor to say a few things to set the discussion for today is in order to monitor open access first we have to measure open access now what we do in open-air right is we get repositories we get open access journals we get we see a label we sometimes see a license and we label it open access and you can click on probably a PDF viewer and you can see it is this what we want because what we believe in open access for publications and for data is that we should apply the fair principles so we should find them it's not only on the data because everyone is talking about fair principles on research data and they forget the publications we should apply the fair principles on publications also so it's not enough just to label something open access you have to label it nicely so you can reuse it you have to fill in all of the metadata so you can interoperate with it you need to make it sure that it's text and data minable and of course you need to find it so this is what we should do what we should think of when we say measure open access what is open access and we should think about it in the very short future so what we do in open-air to monitor open access for EDEC I will just get you some graphs and these are graphs so we're measuring the FP7 we're measuring golden green we're measuring publications in social repositories or not we're measuring per program per project and we do that for the other funders so what we want to do in open-air is to have an open access open science monitor across Europe if we're able to get other funders on board but we need to go beyond that because this is very simple we need to measure openness this comes from an OSI 2016 working group we had nailed down some facets of openness and this is where we should go to we should measure all of openness on all products research objects we should measure the proper licenses availability measures how available they are for what can we use them and of course performance and format measures so this is something that we should be working to for measuring open access and of course you can imagine something like the open science monitor similar to the open data government monitor which deals about open licenses if it's machine readable, availability how full of metadata it is broken down by different facets so this is a country and we can have a monitor around Europe and this is what we would like to do in open-air we're doing it just very in a primary way but this is the way that all member states and all research communities should think on part two, so this is the discussion for this morning part two, the impact of open access this is... I would like to put some questions on the table there is the commissioner Moedas open science, open innovation, open to the world we have indicators that it does actually have an impact but we haven't measured how and we need to measure for research we need to measure for innovation and we need to measure how it affects the sites in the public and then what is most interesting is how does it relate to the research assessment as the previous talk said about how we can enable open access and open innovation that we have to change the system because it starts from the research assessments but this is only... I think we make the mistake of primarily looking to researchers but we need also to look at the funders and the other policy makers in open-air what we do is we try... we have a pilot usage statistics so it's a Google-like analytics service for literature and data repositories and we make sure that we get all of the stats there and analyze them and return them back to whoever wants to use so when we are measuring open access and the impact all we can do as open-air is to get the metrics and then the people will see... will let the appropriate stakeholders to do the analysis so when we are measuring open access impact I think is that we have to realize that there are different and multiple metrics for different users we don't know yet how they go we see bibliometrics we see Scientometrics we see Webbometrics we see Auto-metrics how are they used what is their impact so what is our job is to facilitate the process and make them available but as a community what we should insist on is to have an open-metrics framework what we have in the past with Thomson Reuters with all of these metrics that are being calculated for the research assessment everything is closed what we should change is to go to the open-metrics framework and this is a start that we are doing in open-air but I think we need a lot of work in order to go there so this is framing the discussion and I hope that we will get a chance to discuss all of these things today okay okay, thank you Natalia for informative presentation it's time to start the first session of today on measuring open-access and we begin with a presentation on behalf of the Knowledge Exchange its presenter is Mikal Svensson he is open-access coordinator at the Danish Royal Library can everyone hear me or do I need a headset? I'm going to take off my glasses otherwise it's not going to work oh my ears are too big I'm a technical person you can see I think I managed now cool okay first of all thank you to open-air and also to Jens and Nina for inviting Knowledge Exchange in today and tell a bit about the work we've been performing the last two years and not so much measuring maybe because that's something that takes part nationally mostly but also to try and raise a bit above and look at how we actually monitor open-access amongst institutions and also countries and partner countries in Knowledge Exchange and I was really glad to see the program that one of the aim of today's workshop was to invite a wide variety of stakeholders when it comes to monitoring because I think that is really needed and that's also some of the recommendations that I'll show today that pops up from the workshops that there are many stakeholders in this business and we need to address all the problems and issues that there are when it comes to monitoring to those of you who are not familiar with Knowledge Exchange I'm just going to tell you a bit about what they do as an organization otherwise I'm going to more dig into the workshop series we held the first one in Newtrix in 2015 and last November in Copenhagen we had the second workshop and what we did was we tried to tell you a bit about how we tried to scope those two workshops look at the background, the method we applied and what the objectives were and then giving a status on some of the keynotes that were given by the community outreach we did and then also look specific into the country reporting on monitoring OA and then finally as the second part of the workshop we had breakout groups who discussed in-depth key selected topics and some questions and that turned out to be more than 50 recommendations given and I allow myself to cherry-pick a bit of those recommendations today but the report will hopefully come out Christian in two weeks or something like that so you can dive more detail into some of the recommendations and then finally give an outlook on what's needed beyond these workshops and also around the published report that is coming up so Knowledge Exchange consists basically of now six organizations with the last members CNRS from France coming in and these six organizations are working together in Europe to support the development of digital infrastructure to enable open scholarship and it's a network consisting of expertise within both open access research data and also open scholarship and it's a very fruitful network with a lot of interesting collaboration I think the background and method we applied was that in 2015 what we call in Knowledge Exchange a task and finish group was set up within the open access expert group to try and plan and scope a workshop series around international collaboration addressing especially the challenges but also try to come up with some of the recommendations in monitoring both open access publications and also the derived cost data that we see so there's a need for monitoring in terms of economic restraints and also in order to so institutions are more capable of allocating budgets when it comes to this and then what we applied was a kind of I call it a mixed method because it was both consisting of quantitative data from running projects and then also quantitative data that popped out in the breakout groups and inviting also some of the keynotes, most of the keynotes are also present here today you'll see that later that gave a kind of baseline for the breakout groups to discuss some of the topics more in depth and we asked them to come up with some really practice based recommendations on how to maybe solve these challenges and what are needed for the future the objectives were quite clear we had a goal of pushing for transparency in exchange of open access metadata and cost data that was the goal of the workshop and how, what's the aim of this goal it is to influence as Kurt said in the beginning evidence-based policymaking around data and then promote better outcomes in negotiations with publishers that was the clear aim or those clear aims for the workshop I also did a bit of meta monitoring on the workshop it was really an international analysis exchange, we had 11 countries participating we had two inter-organizational analysis exchange more than 41 organizations took part and we had both licensing and open access experts participating with more than 50 workshop attendees and if you want to check out the hashtag KEOA16 there are a lot of good tweets in there if you're active in that social media okay as I mentioned I think one part that was really crucial to analysis exchange was that we were working within analysis exchange we know each other as organizations and countries but what is actually happening outside analysis exchange also in open-air there was also open-air participation here and we said okay there are actually in open-knowledge foundation there's actually research taking place within open access and one of you probably knows Stuart Lawson from Bergbeck he's at Bergbeck now did a really interesting keynote on some of the research he's been doing and also some of the stuff he's been working on with JISC in order to collect data from some of the offsetting deals there and as you will hear today we also invited in Crossref Rachel is going to give a talk here later today about what work is Crossref doing in order to disseminate open access metadata from publishers at Crossref and that was really interesting and then to the projects on especially the Springer compact deal how do we collect cost data and information from offsetting contracts but trivial it's actually difficult stuff and we had Kai from Max Planck and Graham Stone from JISC and then we actually had six country presentations the first one from Dirk who's also here today I see on the INTAG project that is running now within the DFG we had Frank demoing who's also here today demoing UK and one to UK and local monitor and we had the Danish Open Access Indicator that Karen is going to talk about later so you have a really good opportunity to get something repeated here and then from the Netherlands we had Schuster who's also here I saw and Robert van de Voren from the VSNU about what they do at the negotiation deals with some of the big publishers and finally we had Yerke Ilver from the National Library of Finland around the work they do in Finland and from France Couperin was representing France here by by Saint-Lin-Mellatot and we had François Luce from University of Lorraine around what they do locally in France at a university around collecting APCs and data from Gold Open Access Publishing I think that gave a really solid baseline for actually splitting up the next day in workshops and in breakout groups and discussing things really in-depth and we set it up Null 6 Change did a lot of work to try and say okay why are we going to discuss it it's such a complex field it's so big and one of the things that were really really important was okay data collection from sources quality again as Kurt said in the beginning we need to address quality in the way we collect data from available sources and that breakout group was led by Maurits van der Graaf who's also doing some other work for Null 6 Change at the moment he's a consultant and the second breakout group was on workflows how do we efficiently monitor and what are the workflows are they set up efficiently in the way we do it especially Kai Kichun from Max Planck Library has done a lot of work in trying to figure out how we can obtain efficiency in workflows and of course there always needs to be a standard track in a workshop so it was led by Frank Minister and here we addressed how we aggregate open access publications and costs via standards is it done via standards are the standards sufficient what do we need a lot of questions popped out I won't go into detail with all the questions here but as you will see in some of the recommendations you can probably guess the questions and the last thing that is also really important was governance and policy making how do we build in trust in monitoring open access and can we align policies across nations and policy making as well as by Angela Holzer from DFG okay I am going to dig right into some of the recommendations and as I said in the beginning I have allowed myself to cherry pick a bit there are close to 50 recommendations and they are set up in the report so you can see it more detailed in two weeks time but when it comes to green open access some of the recommendations that were given in the breakout group on data collecting and sources we need to identify the corresponding author in a metadata scheme we need to identify the potential green open access it is a concept that you will see in the open access indicator as well in current talk via the Sierpa Romeo API you will actually see the publishers self-archiving rights and policies and we need to monitor open access after the end of the embargo actually do publishers fulfill what they promise when it comes to self-archiving rights when it comes to Gold OA well matching metadata from our Chris systems or repositories either with the DOAJ or as we have seen coming from Bielefeld the ISSN Gold Open Access List that has much more in it and in order to identify the pure Gold Open Access articles do that matching is really important and then both look at the APC expenditures and the administrative handling cost of handling APCs that are involved here and it is part of the cost structure it is done maybe by the author himself as libraries for example we are totally bypassed when a researcher takes out his mastercard and just pays for the APC we never have the chance to monitor that and what about the APC funds that are set up at universities what do they do we need to be able to track it hybrid that is the really difficult one it is the requirements we need from publishers because they contain the data and it should be agreed in the offsetting contracts we need both corresponding author we need license information we need the exact publishing date standardized data formats for author affiliations is also needed in order to know who have paid for this when it comes to open access publishing around the workflows there was also some recommendations here given and mentioned in offsetting contracts that could be in the terms and conditions publishers should include in CrossRef a license statement for each publication that would make the automated way of tracking this much more possible ORCID was mentioned as maybe a solution to workflow challenges let's say CrossRef did the auto-updating of ORCID profiles if they contain the right metadata ORCID could be the solution to workflows it's not at the moment but it's a recommendation so people could work on this if they want using CrossMarkets the standard that most publishers use could also be a container for open access metadata on both versions and costs so there are some available stuff out there capturing both cost and license rate metadata making it visible in the invoices as well I've seen a lot of invoices from researchers you will never have the chance to say what is this? sometimes not an article title it's really hard so DOI that's key it's key in invoices otherwise you will never be able from accounting systems to see which article is this and also the license info author names and affiliation funder identifiers and ORCIDs if they have one again it was also recommended to investigate the complexity of many to many payments and maybe to look at third party e-commerce solutions there is someone out there who is responsible for monitoring this there are maybe e-commerce third party vendors that could solve this as an issue especially among universities and publishers well there is no answers there are some recommendations here standards again use standards when depositing articles that is needed and it could be done by some kind of validation for example in danmark the library does the validation of the quality of the data in the chris systems and you should be really specific about the standards you need from publishers just cast here a best practice for publishers that they've made adding a new field for apc publication to the open archives protocol that is not done mixing and matching staff is also important here again stakeholders it's both accounting people that sometimes pay for invoices it's licensing people it's technical staff working with chris and repositories and it's metadata specialists they should work on the data and then ensure technical visibility for publishers to provide data if they want to provide data to the repositories it should be done by sort api or native api whatever the chris and repositories systems have set up looking at governance and policies it was recommended that measuring must be done more broadly not only on article levels at the moment there is snow there is a lack of any kind of measuring or monitoring of books anthologies, chapters there is a tendency to focus really much on the journal article funders and other stakeholders they should use clear open access definitions in the policies they should support filtering and they should help clarifying if the target goals are met in the policies and align the monitoring results monitoring should look beyond current systems as they change in order to so they become more flexible and adaptable over time collecting cost data in one system is recommended there should be set up in a standard way so the data can be collected and exchanged and the institutional processes for making it clear what are the concept total cost of publications what are they including both APC administrative costs infrastructural costs for running software and these other costs that have always been there like page and color charges they're not often not part of the APC at the moment but it could be additional thing coming up so this is a complex field what we then did with some of these nearly 50 recommendations was that we need to point out who are the key stakeholders for actually working on these areas in the future and what action points are needed and one maybe not surprisingly one of the key stakeholders and the key factors for successful managing access publications is the CRS system it is crucial both in the way that for CRS systems they should improve the integration between both CRS and institutional repositories and CRS should at least follow the open-air interoperability guidelines that was some of the recommendations and they should integrate different categories of open access in CRS for example in the CRS system we have in Denmark it's just open access but what kind of open access is it and they should collect and report data on these different kinds of open access green gold or if it's possible to validate hybrid open access and try also as a recommendation to create a DOI linkage between accounting systems and CRS systems that would make it much easier to identify the costs as well and who paid for it another key stakeholder are of course publishers they are essential actors in monitoring as they hold crucial information they're not just often not so willing to share it but okay a recommendation is for publishers to use standardized data formats for authentication license statements and indicating the status of the publication is it green is it gold or is it hybrid libraries and funders when they negotiate with publishers or set up policies they should be really specific about the metadata requirements and they should include these in both offsetting and licensing contracts publishers should also include all cost and license related metadata in their invoices as I mentioned earlier on another one the last one libraries they have a central role because they often have the know-how in both monitoring and measuring open access at least from also the licensing work and subscriptions work they've been doing for decades it's a new field but I think they're central they should collect as much data as possible and ensure the data to be opened via IPA and also as Natalie mentioned to be fair according to the fair data principles and they could be obvious let's say key stakeholders as working as centralized OA funding offices regarding APC administration in some universities most APC administration takes place at libraries but often it's also locally at institutes and faculties requirements of transparency to all costs related to everything that has to do with publishing and ensure that non-disclosure agreements are avoided also when it comes to if there are some kind of APC offsetting taking place locally make sure the data are open non-disclosure so and then set up try and set up processes that openly monitor and institutions total cost of publications these were the key stakeholders and some of the recommendations given but there was one maybe as a take home one of the most let's say a mission statement you could call it or a superior workshop statement that was given was that okay if the monitoring open access is not open it will probably fail so open open open all the way down and I would say this work I need to okay you need to acknowledge all those who participated there were really lively discussions and I'm not sure that this work has done but this is just started but here are some recommendations to be given and well as I mentioned the published report will be out in two weeks so I hope this was a bit of an advertiser for that one and I also want to say thanks to OpenAir for coming and I think we have do we have time for any questions otherwise there's a coffee break and I think I'm in the panel later at the end of the day so if you have any more questions welcome to contact me or prop my shoulder okay yeah yes any questions for Mika no yes good morning everybody I guess the library is taking very important role the monitoring of the APCs maybe we should think about how can we add some features to the library automation system to APCs like ordering the books something like that it should be very important what do you think about that in which system what part of the system do you think library automation systems yeah what are library automations I'm not a... I'm not a... I'm not a... I wouldn't be able to say if that's the solution I think one of the way that the recommendations that were seen here was that ORCID could be some kind of solution because it's applied both for publishers it's applied in the Chris systems often integrated into the Chris systems and it's connected to the researchers so that could be maybe some kind of container for if their profiles are auto-updated if their metadata are exchanged that was one maybe possible solution to automation but I'm not an automation system expert so I wouldn't... sorry I wouldn't be able to tell but maybe someone has a reply to that in the audience but I have a question we run an open-air pilot on APCs paying APCs for post-FP7 publications and what we saw is that the libraries they are doing the job but they're not very much organized because they are missing the IT systems many of the libraries used Excel sheets and I think you said about open protocols and what we did is we made a system which is automated and then we wanted to talk about exchange formats to exchange the data to a system with other stakeholders even though the publishers are not willing to do that but the other stakeholders like GSCO should do that what does it take to get a group through knowledge exchange perhaps an open-air to get these things in place because we're talking about open protocols and open monitoring but we're not doing much I would say that's a really good activity for KE to try and look more into but I would say for a local perspective for example we used in Denmark Navision as a really close system for accounting system from Microsoft we had no chance to try to communicate with that system at least it's not open but it could probably be developed but often I think at the moment there's no connection between Chris or repositories and accounting systems that's the huge challenge so we need that in order to have that exchange of data meta data yeah corners yeah yeah I would like to see the first open-licensed system yeah the open science monitor ok ok ok I think that's it again we will now have a break and we'll return in roughly 20 minutes 45 minutes past 10 thank you welcome back everyone now our next speaker is she is a senior research analyst at the Technical University of Denmark DTU next so while we're waiting for the slide to appear thank you for inviting me to this workshop to talk about the Danish Open Maxis Indicator my name is Karen I work at Technical University of Denmark we assist the Minister of Higher Education Science in operating analyzing and further develop the Danish Open Maxis Indicator this is a joint effort with the library at the Technical University of Denmark and I can't take credit of any of the things that has been done so far because I'm fairly new to the team for the next half hour I will give you a background of why we measure in Denmark what has been measured how it is measured the challenges connected to measuring Open Maxis suggestions to developments and improvements and how we can use the data from the indicator so in Denmark since 2014 we have a national strategy for Open Maxis which is green primarily green which has a definition of Open Maxis as scientific and peer-reviewed articles and conference interviews in journals and proceedings with ISSN this is the first challenge I think with measuring Open Maxis that this is actually a quite narrow definition and in other countries you might have a different definition of an Open Maxis publication when defining the strategy there was some talks about actually including book chapters to the definition of Open Maxis but due to input from the university that was actually taken out again and ended up with this definition because in regards to anthologies and monographs we're just really not very close to having Open Maxis so this is the definition we ended up with so the strategy has some very ambitious targets the idea the goal is in data from 2016 80% of those publications are supposed to be Open Maxis and by 2021 100% are supposed to be Open Maxis so that's quite ambitious and as you'll see later on in my presentation you can see the state of affairs as we are right now and there's quite a long way to go but it's always good with some good challenges so with these targets obviously we need something to monitor how well are we doing in Denmark on a national level and on a university level and hence the Open Maxis indicator was developed the indicator is actually a further development of the Open Maxis barometer which was developed as a part of a deaf-funded project which ran from 2012 to 2014 developed the foundation of what we have now in the Open Maxis indicator the indicator measures once a year in springtime last year we did a pilot a pilot harvest and this year we did a pilot harvest which will be launched in around April with the brand new numbers so the aim of the indicator is to monitor how well the Danish university fulfilled the target of the strategy it's an add-on to the Danish national research database that collects already all the research from the Danish universities so we have quite a good data base already it measures once a year as I just said now we come to another definition there's a lot of definitions here the way the Open Maxis indicator actually can measure Open Maxis there are two kinds of Open Maxis to be measured in the indicator the first one is a gold Open Maxis so even though we have a green strategy obviously there will be Gold Open Maxis publications as well being published in Denmark they have to be real gold, not hybrid and we use the directory of Open Maxis journals as a foundation as a base to validate the Gold Open Open Maxis journals also we measure the green Open Maxis we do that by measuring the publication downloaded or uploaded either in an institutional repository or in an external repository what we don't measure is hybrid Open Maxis only if it's uploaded into one of these repositories and I'll get back to that a bit later so the data sources we use for the measuring of the Open Maxis output is first of all the publication metadata from all eight universities all eight universities in Denmark are using the same Chris system, it's pure so we have like a unified data model and have very consistent data that we can use as a basis and in Denmark we also have the Bibliometric Research indicator that you have in Norway and which Kurt talked about this morning it's a funding funding allocation system and since we have that funding allocation system I'm not going into detail whether it's good or not but we have it and it actually means that the Danish universities they actually register most I would say next to 100% of the research I actually registered in the Chris systems which you don't get the money from the BFI system so that's also a data source we use Directive Open Maxis journals as a source we use Sherpa Romeo to validate whether or not there's Open Maxis potential and then a new addition to the data set from 2015 is an authority list of trusted external repositories this is this is collected or compiled annually with input from the Danish universities they can actually make suggestions to this list of external repositories and they're being decided by the National Steering Group committee of Open Maxis in Denmark furthermore we have an authority list of journals with extreme embargo periods since the strategy says that you're only allowed to have a maximum of 12 months embargo what we found was obviously a lot of the journals indexed in Sherpa Romeo said yeah, there's a green Open Maxis possibility but a lot of those journals actually have more than 12 months embargo periods all the way up to about 48 months which is not really Open Maxis potential in our view we actually collected lists of embargo periods from the publishers and the university assisted with those lists so that's compiled annually as well and works as a base also a data source so I'll walk you through the steps of the indicator measuring yeah, first of all we collect the metadata from the CRIS systems and that results in a subset with duplicates a number of the publication in Denmark are co-authored by two or more universities but all universities put an entry into their CRIS system so we have one data set, a subset which we use for the university calculation with duplicates and then we have, we make a deduplication with the data from the BFI and which results in a data set for the national calculation without duplicates with unique entries those data are then first of all being matched with the metadata to see if any of these journals are indexed in Dubai to put an extra quality filter on it it has to be also on this bibliometric research indicator authority list, there are two lists one for journals, series and proceedings and one for publishers so you have to also be indexed on the BFI list in order to be qualified as real gold open access so if this is a case if the publication is published in a journal in Dubai and BFI it's classified as realized open access if not we move on to check if there's a publication attached to the file or to the entry to see if it's downloadable either from the institutional repository or one of those recognized and compatible external repositories if this is a case they are being and if there's something to download it's considered also as realized open access if not we consult the Sherpa Romeo to see if it would have been possible to have an open access green open access to see if the journal is indexed in Sherpa Romeo in the green category and also in order to get into account the journals with long embargo periods we match that with that list as well to make sure that the journals with long embargo periods are not being classified as unused potential so we'll end up with in this process two data sets one at university level and one on a national level the results are based via the Danish research database and via spreadsheets that are downloadable so it's 100% transparent and you can use the data I'll get back to that later as well so the three categories that we end up with are the realized open access potential that are referring to publication that are freely accessible online it's unused open access potential that are not freely accessible online but which have been published in journals which allowed green open access with an embargo period up to one year and then we had the last category the the rest is the unclear open access potential referring to the publication that are not freely accessible and which have been published in journals with an undetermined open access policy and that also includes printed journals in Denmark we still have printed journals that are not digital yet so that's sort of the rest of the publications so when measuring open access in this way we're presented with some challenges first of all comparability and definitions the open access indicator measure what we have defined in Denmark to be open access publications definition of open access and it can only measure the data that is available from the Chris systems and as Miguel just said there's a lot of data that we don't get from the Chris systems we don't get information of whether it's hybrid articles or whether it's corresponding author from a Danish university stuff like that we don't know we can obviously only measure the data that we have so when comparing with other countries in Europe or worldwide you have to keep in mind that you're not necessarily comparing the same numbers so you have to be aware of that also we don't measure hybrid publications with the data we have we can't measure that and it's really a shame because we don't have no idea how many hybrid publications are actually being paid for in Denmark they are taking into account if they're uploaded to the Chris systems, to the repositories but then they are being calculated or measured as screen open access which is really sort of not completely true we have as you see in the process a lot of different data sources a lot of different data has to be matched and compiled together and obviously that it would be easier if we just had to push one button and we would get all the data in one go also there's some manual work in the process so that could be a challenge as well and then we had the challenge with the long embargo periods that we have tried now to take into account with making this authority list with the journals with the long embargo periods but that's also a challenge because we have to collect data from either the publishers or the universities which is also a bit it's could be a bit tiresome for the university to have to go through all the publications so that's also I think it's a challenge as well so I'll just show you what it looks like this is the front page of the Danish national research database you can see here we have all in the left-hand side all the publications from the Danish universities and other research institutions and then we have the OMEXS results only from universities it's only universities that are covered by the national strategy of OMEXS so far so when you go into the OMEXS you get information on different levels first of all you get on a national level we can see that the last the last run we did for 2014 data actually only resulted in 80% OMEXS so it's a bit far to the 80% so it gives a nice overview of how much do we have and how much is unused and how much is unclear then we also divided into main research areas and I don't know if you can see it but it's actually quite interesting to see that humanities are actually very close to science technology that we actually have a lot of quite a big share of the realized OMEXS potential is in the humanities then we had at university level and which is I think what is being looked at the most because I think the university find is really interesting and they obviously compare themselves to each other so we have it in order Technical University of Denmark are doing the best and so forth what I just show you was with the free categories we also have a view where we only present the two categories the the realized OMEXS potential and unused potential because it's a bit difficult to actually reach 100% OMEXS if 30% is unclear if there's no possibility of publishing OMEXS if it's printed journal or just journalists say no we don't want to there's no possibility so you could say that these kind of numbers are more accurate and they definitely look better for national on a national level and on the university level we also have a view where we see the development of the indicator and you can see here we in the 13 we had 70% 14 18 but that was before there was really OMEXS that we have now I think most of the universities have a strategy now and workflows have been established how to actually accomplish getting more OMEXS publications so you can always see the whole time where are we according to the to the target and yeah it's going to be really interesting in the beginning of April the new data are ready we hope and it's going to be quite interesting to see how how much the university have actually managed to to get OMEXS so in the indicator we give you also documentation and data set there's a documentation of the technical aspects and the overview of how we measure and you can get the summary data and the publication data and records data so everything is transparent and each university has access to their own data but also the data of the other universities as well so obviously we have also some things we would like to do in the future step by step first of all we would like very much to be able to distinguish whether or not OMEXS journals within and without APC charges because in Denmark we actually have quite a lot of journals gold OMEXS journals that are for free that do not charge an APC so it would be quite nice to see actually how does it distribute between them we would still need the information of corresponding author it wouldn't give us the data to be able to say we have approximately paid this and this for OMEXS gold OMEXS journals but it would give us a starting point and then to be able to distinguish between that and also distinguish between OMEXS green in the local repository and the external repository that's hopefully something we'll be able to do soon the last two bullets then we would like to measure the hybrid articles and also in the future maybe draw on global bibliographic data for instance from Crossref so actually to simplify the processing pipeline not to have so many data sets and data sources to match but actually to have data from a more consistent data provider so the data these are just examples what you can do with the data the university can monitor how far away they are from the targets and which journals that do not allow OMEXS publishing they can use that information on a more strategic level on a national level it would be quite interesting to monitor the development in embargo periods and data that DEF who will negotiate the license agreements could be handy for them to have to see what does the publisher make it more and more difficult for us to publish green OMEXS and then whatever you want to do with the data it's only available and it's free for everyone to use so if you have any questions thank you Karin we have time for a few questions and again could you just a reminder please wait for the microphone so that everyone can hear you I have more related question what makes a repository a trusted one in your country and what is the relation between the Chris and the repositories in Denmark yeah the trusted repositories as I said all the university can suggest different repositories and we have a national steering committee OMEXS and they actually decide which of the repositories are going to be marked as a trusted repository and it's we have archive we have cenotow as one and we have actually the same journal systems in Denmark are being marked but I can with the new indicator run there will be also a data set with the repositories that are marked as a trusted one so that also will be only available and your second question yeah the relation between the Chris's and the repositories in Denmark most Danish university use the Chris as a repository they use I don't know how many but I think most of the university use pure as a repository and I know it's not a repository as such but that's how it's being used in Denmark it would be nice to have a national repository independent of publishers but they're yet okay one last question yes you said that you did duplicate the publications but how can you give the numbers of each universities because how do you know the publication from which universities they are but we have two data sets one where we did duplicate them where it's on a national level and then one one data set for each university so that's before the duplicate okay thank you okay again thank you Karn for your presentation it's now time for the last speaker of this session Rachel Lamy she is outreach manager at Crossref okay you can hear me even better right good so I'm going to my name's Rachel Lamy and I work in the member and community outreach team at Crossref if you're not familiar with Crossref what we are and what we do we are a not-for-profit membership organization for scholarly publishing and we collect metadata from publishers and then make that available so that it can be used to locate articles to cite them to track them and we're one of the registration agencies that assigns the DOI or digital object identifiers to lots of different things predominantly journal articles but more so now things like books, data sets, pre-prints dissertations in numbers we work with over 5,000 publisher members some big publishers but more and more so a long tail of small open access publishers based all around the world we've got metadata related to over 85 million content items and that includes pre-prints so in about November last year we set up a specific a specific part of our metadata schema to accept and assign DOIs to pre-prints and I think we already have around 9,000 pre-print DOIs registered from people like MDPI and BioArchive we collect funder identifiers Miguel talked about cross mark for displaying corrections and retractions a tool to check for potential plagiarism in articles and we make the information available via an open REST API and search interfaces again that's a quick overview but if I had to sum it up we're probably about infrastructure the stuff that sits behind the scenes that when it works you don't notice it when it breaks you notice it pretty quickly and our specific flavour of infrastructure is that metadata that we collect from publishers people predominantly sort of know cross ref because publishers assign DOIs so that their content can be indexed and discovered more readily whenever papers move from one platform to the other the information can be updated so that the DOI provides a persistent link that doesn't break in the same way as a normal URL would and they also give us things like full text links to the content which can be used for text mining and other indexing services bibliographic metadata is great it's a really good way to pair it with DOIs and to provide a way for someone to be able to find and persistently link to the definitive version of an article on say a publisher site but from what we're talking about today there's actually a lot more information related to articles that is going to be useful to pad out or to add to that sort of standard bibliographic metadata Karen was already talking about the fact that institutions want to be able to track publications from researchers show how much content is being published under open access licenses there's a real need from funders to be able to identify the outputs from publishers publishers need to do this themselves whenever I worked at a publisher and this is a while ago but we had an Excel spreadsheet to track the publications that came from specific funders that doesn't work and one of the things as well is that this information needs to span disciplines institutions, geographies, jurisdictions so it needs to be collected in a standard format no matter who's providing it a lot of this is done through our API which there's no point us collecting all of this information from publishers we don't make it readily available and in a way that means that systems can talk to each other and developers can easily integrate that information into different systems so we make the information openly available anyone can come along, query it our API is open source you can go in, have a look and you don't need to sign up in order to use it we're keen for people to go experiment and see what they can find so how would I use that to be able to look and track information or track research that's subject to funder mandates and ensure that it's being disseminated under the correct license so that it complies with that so we've talked about the fact that it's not just bibliographic metadata we can collect information on publications that ties them to a specific funder we're a big fan of work at IDEs so again if publishers provide those we'll collect them and we'll also use them to push the publication record into the author's orchid profile if they allow us to do so so it just means more complete orchid records to be able to track the outputs from a particular researcher I'll go into more detail on the license information that we collect from publishers and we also collect as I mentioned the location of the full text of the paper normally as you know if you click on a DOI it will take you to the landing page of an article so you can see the abstract authors, that kind of information if I want to try to retrieve the full text in order to mine it I don't want to have to go to the landing page for an article and search around for it if I can find and link directly to the full text then that's much easier for me long term preservation so archiving information we can collect that as well and also information on clinical trials so that people can see information on papers from a range of publishers that are linked around a specific clinical trial and clinical trial ID I want to go into a little bit of detail on the funding and license information that we collect our metadata formats they've been pretty flexible in terms of letting publishers provide additional information the only thing is as you know if people are given free text fields to provide information it doesn't work very well people can type in they'll make mistakes they'll spell things incorrectly so we needed a way to collect information in a standard way from a standard list so we've created a funder registry which as of last week is a list of over 14,000 international funders and we actively curate that list so that it can grow I think when we launched it it had about 4,000 funders listed which again as you know isn't even a drop in the ocean there are so this is growing continually publishers so publishers whenever they are collecting author information upon submission can collect the funding information and match it to one of the funders in that registry and each of those funders has a unique ID as well so their own DOI so that means if the funder changes name merges for example then you're still able to track the funding that came from that institution or that organisation and the main thing that this allows is large scale analysis and reporting to funders so to go into our api a bit what I can do and what I did this morning I did the thing where you know you shouldn't edit your slides in the morning that you're going to present but I wanted to have a look as to how much funding information we actually have so at the moment we've got over 1.7 million publications linked to specific funders in our database and if I go into a little bit more detail so looking at a specific springer article I can see that it was funded by Arthritis Research UK and that's the specific identifier that relates to that funder we use our api a lot but obviously there are different what you can do with the api is that you can build stuff on top of it so we built a really simple search interface so that you can go in put in a funder name and be able to find all of the publications that we can see at Crossref that have that funder associated with them so if I did a search for Medical Research Council it's going to bring up all of the articles that we can see in this case about 6000 results that were funded by that by that organisation I can see the grant numbers if those have been provided and that's across different publishers not just by publisher if I want to I can group that by publisher or publication as well so it's an easy way to see and manipulate that information if you're interested in it we also collect license information we collect that as a URL from the publisher they provide that information to us again the information collecting it as a URL means that it's machine readable as opposed to collecting license text looking this morning so in this example with the api I've asked it to tell me to let me know how many articles that Crossref can see have been published using Creative Commons CC by 3.0 and whenever I ran that query there were about just over 200,000 so I can see that information as an overview but I can also see it on the level of an individual article because there's as we were saying there's no point tracking this information on a journal level because you've got hybrid journals you need to know the information on an article by article basis for those of you who really like metadata I'm a fan this is how it actually looks in the deposits that we get from publishers so we've got an access indicators schema so it's telling me okay this article has been published under Creative Commons license it's a Hindawi article so and I can also see as a little plus they also provide me the full text link for text mining as well they've got as a big up for Hindawi they do provide really good thorough information in their metadata obviously I would say the majority of the license information we can see at the moment is related to open access licenses because it's simplest for open access publishers to say okay everything that we have is published under this license and they can upload that information to us pretty quickly but it's not as simple as that for hybrid journals embargoes etc so the way that we collect the information can be more nuanced than that so in this instance I can see that the publisher has said okay this specific license applies to the VOR or version of record and it starts on this date this license applies to the accepted manuscript so you can by saying which date specific licenses start on you can convey if the paper is being published under say a proprietary license for the first year 15 months and then when it moves over to being published under an open access license publishers can also give us if they have a specific license related to text mining they can provide that there and then you can see they can also provide archiving information as well as I mentioned being able to tie funding information with the article but being able to tie that with the article plus with the author as well by other Orchid ID gives a really useful set of information as of this morning we had we could see about 0.6 million Orchid IDs that have been deposited with us and tied to articles from nearly 850 850 publishers so again you'll have seen that some publishers have started to mandate Orchids, some funders as well and we're definitely seeing that reflected in the in the amount of Orchid IDs that are coming in to us in the metadata associated with with articles, books pre-prints etc and just a quick example the majority of our metadata is in English we've got some in different languages so you can see that we can collect the information in lots of different in different languages and you can also tie it to an Orchid ID which makes it especially useful I know I'm here to talk about the the measuring part just to talk a little bit about what we're doing around being able to track to look at impact so we do collect we do collect reference information from publishers as well and publishers can choose to make that available via Crossref publishers can look at the information to be able to see how many times people are citing their publications again it's interesting because it's looking at a different subset of information from say Google or PubMed so it's something else to add into the mix we're also working on a service and you can link to it here called EventData it's been really interesting to us over the past couple of years because we measure who's resolving DOIs and where those resolutions are coming from you won't be surprised to know that a lot of the resolutions come from publisher platforms but more and more we're seeing DOIs being resolved via via Wikipedia social sharing platforms obviously things like Twitter, Facebook and we're setting up what we're calling Crossref EventData so we're aggregating information on where events so anything related to that happens related to a DOI happens on one of those platforms we're making it openly available via an API and we're not this isn't us providing say getting into alternative metrics it's a way just to provide the raw data so that it's open it's auditable it's transferable as well so that if a journal moves between publishers that information isn't lost you can still measure how many times an article in that journal has had these events associated with it this was really sort of based out of an initiative that came out of the plosalt metrics group and then via Facebook so we've sort of picked it up because it was one of those things that it didn't make sense for each publisher to try to do individually we're hoping to launch it in I think May, June so if you're going to measure the events or things that happen around an article it's really important that you can dig down into the information underlying that see what the source is and where it's coming from so that you can you can infer your own judgments on that information to go into everything that happens is an event so there's been a bit of talk about being able to look at and measure things like data citations so we're working with data site on this so being able to look and see if a crossref DOI is being referenced by a data set or vice versa so that you can start to make those links between articles and data or continue to do so being able to see if an article was referenced in Wikipedia using its DOI on Twitter, like on Facebook we're going to feed those in to event data so that people can go and access and use that information other things that I wanted to flag up there is some work going on so I said about the fact that it's really useful to be able to link orcids and funders and article information I think the real the kind of the other piece of the puzzle in that is being able to being able to collect organizational or institutional identifiers in a standard way I know some people here are involved or following the organizational ID project which has crossref representation but also from people from different organizations data sites off the top of my head I think there are about 17 organizations directly represented on the kind of working group for this and the information that they work on is going to be made openly available for comment but in the same way that we've got the funder registry having a list of institutions with IDs that can then be collected with article information is just going to make it much simpler to be able to track the outputs of institutions and specific researchers another quick piece is that with certain open access mandates it's not good enough for articles just to receive DOIs at the point of publication so we're working on a way for publishers to be able to deposit DOIs and metadata for articles at the point of acceptance so that institutions, funders can start to track them earlier on in the process again I've linked to our blog about that and what we're planning to do there's always a caveat isn't there we need good metadata for publishers to be able to do this and part of what we can do is Crossref is we can provide better tools for publishers to be able to deposit this information with us as simply as possible and we are doing so via integrations with things like open journal systems which Karen mentioned earlier we also we need to do a bit of rallying for publishers to make sure that they that we show them as simply as possible this is what you're providing Crossref this is what else you could be doing and just having conversations with them to show how useful this information is and it saves them having to provide infinite numbers of spreadsheets to be able to to be able to provide the basic information that people need to be able to track research so this is the big one that we're working on as well but we're definitely seeing progress and I think my time is about up so I'll stop there I apologize lots of information all at once but do come and ask questions thank you thank you Rachel we have time for a few questions yep hi thank you very much that was really informative I have loads of questions but I'm just going to stick to one I think you've answered the one about the linking and the DOIs so right at the moment I'm involved in doing a research evaluation of arts and humanities an institute I'm trying to benchmark it against other arts and humanities institutes worldwide this is really difficult our journal articles we can track those but what about tracking the other research outputs so here's an area that we became aware of from working with Autmetric.com everything that had a DOI could be tracked but so much of our material it didn't have a DOI and a lot of it is, well there's books there's book chapters, there are reports we discovered in Ireland that government reports don't have any kind of identifiers of any sort so it's almost impossible to track policy impact and there's a whole lot of other areas as you mentioned to do with data sets as well so the DOI and linking the DOI to handles and repositories seems to be a really obvious thing to do give us some information about what you're doing in this area and what's the way forward sure so I can talk about so books is one of our fastest growing areas in terms of people coming to us and registering DOIs for books, book chapters etc and then with things like preprints as said we just sort of extended our schema to be able to make sure that we can that there's a really clean and clear place for people who want to register DOIs for preprints we're also working to, because we don't want people to sort of cram things into under designations or kind of top level things that don't make sense but again we are working to okay let's have something specific for say reports or if there are things that are more related to data and data sets we might pass them over to data site as well but we are working to make it clearer for organizations that you can deposit DOIs for lots of different things like peer reviews etc so we need to get better at talking about that I think as well my default is that I talk about publishers but actually a lot of our members wouldn't necessarily class themselves as a publisher so it might be a funder or a government agency or an NGO who are coming to us and saying there's stuff that we need to allocate identifiers to so that they can do that via us so we're getting there and we are conscious of that because there's so much more that needs to be tracked related to articles than just the articles themselves okay one last question thank you for a lot of interesting information we're really following this I was just asking you did say that 850 publishers are depositing or kids out of the total amount of what you said you have no that was fun to so how are things getting along is it growing is it growing do you believe it's growing quickly enough can we believe that this can be used as the main sort of ideas in the fairly near future how do you see it growing we're fairly I would say to declare my own our own conflict of interest ORCID kind of grew out of a sort of a side project of I think we used to we talked about author DOIs at one point and ORCID grew out of Crossref and then because of the way that ORCID needed to be governed for example it didn't make sense for it to sit under a membership organization for publishers so but we work closely with ORCIDs and we're pretty we think they're a really good we think they're a really good thing I know that they've just passed 3 million ORCID IDs and the real pushes for people to integrate that into their systems what I can say in terms of numbers at Crossref whenever we whenever we launched the sort of auto update for authors I think we had maybe just under I think about 280 publishers depositing ORCID IDs with us that was in that was at the end of 2015 now at the start of 2016 we've got nearly 900 publishers depositing ORCID IDs and the number of that has really grown so I think the key to ORCID now is people are getting the IDs and that's great but the real kicker is that people need to use them and we are seeing that happening so the signs are good on our side thank you Okay, since I'm a nice guy one last question then Just a question for small publishers usually open access so there are a lot of new metadata collected such as licensing find drafts and that kind of things so what about all the records already deposited does it mean that all this metadata of all records must be really positive the whole content or it could be just expanded with this new data yeah so if if you're well if you're any that was one of the things that we needed we realized needed to be easier to do so if a publisher wanted to come to us and say okay I want to give you license information for all of our publications we don't want them to have to re-deposit the full bibliographic metadata for every article because it's a big job for them and it slows up our system so there is the option for to basically to give us a CSV file with the DOI plus additional metadata and we patch that information into existing DOIs if somebody does want to go back and provide that information talking about small publishers as well is that obviously a lot of them use open journal systems so we're working with PKP to develop and enhance the existing plugin so that you can go in and you can do that via OJS and again you don't have to add that information by re-depositing all of the metadata for each article so we're conscious that that's something that we need to make as easy as possible okay thank you again cool thank you thank you it's now time for our second session of the day and attention more towards the impact of open access and our first speaker of this session is Antonio Perianis from the European Research Council where he is a policy analyst and let's see if we can find his presentation good morning thank you very much I would like to thank the organizers for the invitation and the opportunity to present this communication entitled open access measurement at ERC it's a preliminary analysis conducted during the last two weeks and we are focused on figures on productivity, visibility, influence and excellence as introduction there is an extensive literature on the citation advantage of open access papers in short, the main idea is that when papers are openly available they are more cited than papers published in pay for access sources on the other hand skeptics argue that the advantage of open access is partially due to citations having a chance to arrive sooner another artifact described is the selection bias according to which authors pick their best papers to make open access in any case some of the limitations of the existing literature on the subjects whether supporting or refuting this open access citation advantage are the small number of articles analyzed, the limited size and diversity of citing sources and the short citation window considered so the main objective of this exercise is to quantify the current situation concerning open access in macro, meso and micro levels analyzing publications by fields, regions and countries, journals, institutions and of course all publications we also try to contribute to answer is the hypothesis of the citation advantage of open access publications is correct our dataset includes all publications available in Scopus between 2005 and 2014 the list of indicators has size dependent and size independent variants the size dependent are total number of publications total number of citations total number of highly cited publications included in the top 1% of the highly cited in Scopus the size independent that allow us to make comparisons between different domains proportion of publications percentage of citations normalized citations proportion of top 1% publications and the relative ratio of excellent publications in the top 1% for top 1% and normalized citations we consider citations distributions for publications of the same year in the same field and in the same document type it is important to clarify that we consider in this exercise open access those publications of the journals listed as open sources by Scopus in those journals open access is provided by the publishers so it's a synonym of gold access other the category other refers to the rest of journals where open access is not provided by the publishers where open access is not provided at all reported publications ERC publications are collected based on the list of published results in the reports of ERC grantees the consideration again where there are publications is open access is exclusively based on the list of sources provided by Scopus in summary the person exercise analyzed 18 million publications 28,000 journals of which more than 3,000 are open access sources and more than 50,000 ERC publications our first analysis is focused on the excellence and visibility of the Scopus and the ERC publications on the top figures concerning highly cited publications provide three main findings first the proportion of excellent publications of ERC grantees on the right is outstanding secondly the performance of open access publications the blue lines is lower in both distributions firstly open access publications are below the expected rates is 1% expected rates in Scopus are present at decreasing trend in ERC publications on the bottom trends in normalized citation are quite similar to the previous figures again the normalized citation of ERC grantees on the right is notable and the performance of open access publications the blue lines is lower in both distributions for all publications in Scopus and for ERC publications the comparison of percentage of cumulates publications and cumulates citations shows interesting results usually citation distributions the red lines are above the publication distributions as in the figures on the bottom however citation distributions are below the blue lines in open access publications this behavior is a typical and surprising because one of the theoretical advantages of open access is that citations arrive sooner due to the easy and early access to the publications by the researchers the effect is stronger in all publications during the last four years of the period in the case of the ERC publications that's in 2014 in the last year also interesting is that the proportion of open access publications is similar in both distributions is a little bit higher for ERC publications probably because of the open access policy in our agency and other incentives for our guarantees and in both cases is around 15% in the last year of the period the increasing trend in productivity is similar as well in both Scopus or publications and ERC publications table one present the total number of publications and the total number of top one publications highly cited publications and the percentage of both publications and excellent publications and the relative ratio of top one percent basically open access publications represent 10% in Scopus and 15% in ERC publications on the other hand ERC publications present a remarkable incredible 8.7% in top one percent publications the relative top one percent basically we split this number into open access and other sources and the open access relative ratio is significantly lower in both distributions but note that for ERC is slightly above the expected rate is 1% the expected rate in this case is 1.1 and 0.1 for Scopus publications turning to the fields table two presents is quite simple table the proportion of publications and citations by fields more relevant finding is that open access publications attract less proportion of citations for example in the case of multidisciplinary 11% of publications attract only 2% of the citations similarly table three compares the excellence rate of all ERC all ERC publications first for all publications agriculture biology, environmental science, mathematics and health present higher percentage of top one percent publications in open access sources five out of 27 fields in ERC just four and only mathematics coincides in both distributions in mathematics open access is above the other sources notes that the proportion of ERC publications is always above the for example in mathematics is 9% versus 2% for all publications in Scopus moving on to regions the main findings in the comparison Europe versus United States is that Europe presents higher proportion of open access publications is the blue solid line 14% in 2014 versus 11% however the rate of highly cited publications in open access the dotted lines is quite similar in both regions about 9% in the last year of the period analysis by contrast the proportion of top one percent publications over the total publications is higher in the United States 2.7% versus 1.9 in Europe after the disaggregation figure on the rights open access publications dotted lines present a very low proportion of excellence below 0.3 this is the scale for open access in top one percent with respect to countries and similarly to the results of the regions and fields open access publications of excellence red bars are lower or present a lower proportion than that of all publications the figure presents Europe EU 28 countries and just only 6 of the countries present a higher proportion of open access top one percent publications in open access higher than in for all the six countries are EU 13 countries and the reason maybe is because of the national policies in open access the analysis of journals provides the following findings there are three journals in the top 20 of the more prolific in fact the first journal according to the total number of publications is an open access journal however there is only one open access publication with the higher rate when we order the list by the proportion of top one percent publications in the position 16 finally the 19 institutions with higher number of year see grantees are analyzed the institutions are commission the energy atomic, CNRS, the SIG Ecole Polytechnique in Lausanne and Zurich Imperial College in CERBs Universities of DELF Coven, Tel Aviv College London, Ebreu Amsterdam, Cambridge, Copenhagen Edinburgh, Munich Oxford and the Baseman Institutes the first figure on the left compares the proportion of open access publications in Scopus and the ERC we can see that the proportion of open access in Scopus is around 10 percent in the majority of the institutions the blue triangles the proportion is significantly higher for ERC publications near or above 20 percent in the figure on the right focusing on top one percent publications, excellent publications ERC also presents a higher proportion of excellent publications around 10 percent or higher than that in Scopus well in this case it's 5 percent more or less the proportion of highly cited publications for both ERC and Scopus is notable in all the institutions is above the expected 1 percent the decomposition of the previous blue and red circles into diamonds and squares blue and red of course allow us to analyze the importance of open access in this case in institutions the main findings are that open access publications diamonds present a lower rate of excellence moreover all open access publications in blue are below the expected rate in this case below 1 percent on the contrary the red ones are typically above the expected of 1 percent other publications squares so are not worthy rate of excellence especially remarkable in ERC distributions the red squares before summarizing and listing the main findings it's important to remind the double analysis conducted in our exercise using two types of sources open access and other on one hand and two distributions of publications ERC publications and all the publications available in SCOPUS on the other hand now it's time to recap some of the main findings of this preliminary exercise and well four key messages the first one is the proportion of open access publications fluctuates between 10 percent in SCOPUS is the minimum and 10 percent in institutions with ERC publications in this interval 15 percent in ERC 14 percent in Europe 12 percent in the United States second open access does not provide significant advantages invisibility citations influence normalized citations or excellence top one percent publications despite this the steady growth in productivity there are some exceptions in five fields six countries some journals advantages invisibility influence or excellence were found four ERC publications provide an incredible excellence rates including those published in open access publications moving on to the discussion the evidence from this preliminary exercise confirms the findings of recent studies refuting the idea of the open access citation advantage in our case is limited to gold sources the current open access paradox is that an increasing number of publications fail to attract the interest of researchers especially in the last years even though the theoretical advantage of the sooner arrival of citations some questions rise by these results what is the role to be played by factors such as cost of publishing or peer review to reverse this perception should quality control policies be revised in open sources should publications policies in traditional journals also be revised as demanded the Gates Foundation a couple of weeks ago as you can see more questions than answers but I think that some of them probably will be addressed during the panel discussion this afternoon and the last slide and the last question is for us and the question is how to deal with the shortcomings of this exercise in the future because the classification use provides a good indication of the current influence of open access publications synonym of gold access journal or sources open access provided by the publishers but we would like to split other the category other into other types of sources open access sources and known open access sources but in this case is true that determining whether a paper is available open access is not easy some research groups are working on algorithms to code hybrid papers as you know in this case hybrid papers are publications originally published in green journals and also available for free in open archives in between our current situation and this algorithm solution and I think that is an easier proxy for this purpose is to use CERPA Romeo classification of journals of sources in order to refine and extend our current results and well that's all, thank you very much for your attention and your question Thank you Antonio for the thought-provoking presentation we have time for questions I want to say some qualification, Bulgaria was very high and I can explain why we have a mathematical journal library which is part of European mathematical journal library and this repository people are all the work coming more than between 100 and 200 types every single day to read it's open access and from there maybe Bulgaria is so high we are very small content maybe Bulgaria and the mathematics also yeah thank you very interesting thank you for the presentation I have a question on the data when you say ERC open access where do you get this data from because the data we have from open air we find that 70% of ERC is open access yes so do you mean open access gold or do you mean see but the numbers should be different if you had open access in general because many of the ERC ERC publications are in green yes so would that would you know if you did the exercise again if it helped you do the exercise again probably the numbers because some of the as I explained we try to split other into green and non-open access sources our preliminary exercise is based just on gold sources according to the list of scope positions this is one of well I think that the position of gold journals are going to remain as now the question is for example of non-open access journals or the green journals so the blue line must be split into a green line and another I don't know green sorry gray or another color the problem is that I think that the results are completely valid for gold open access sources the question is the position of non-open access for example in previous exercises made for example in Canada and the position of the green access so other types of open access the question is that gold open access sources performs regularly that's the point and it's one of our limitations okay we have a question from the front here then one back there hi I'm somewhat critical about the choice of using Scopus data for this exercise well we have to note that Scopus is a product of Elsevier who is very heavily involved in publishing and I would this is my doubts are based on this fact and also on some particular data which I checked in Scopus on small independent open access journals covered by Scopus so I think it would be really very much interesting trying to repeat this exercise and trying to use other data for instance data from Web of Science or from other sources and well this is one of the topics of today's workshop whether the open access community is up to providing statistics and numbers and we are clearly not yet but maybe we should well we can discuss about the database or the indicators is true but at the end we have the citations made by other scientists to these publications in the case of for example using Web of Science probably the results I think that would be even worse because there are less sources so less citations in play sorry maybe maybe it's true I agree we have both databases but for this exercise we decided to use Scopus but we can use Web of Science also in fact the list of journals can be used in other databases so the list is the same and I think it's interesting but always in this kind of studies sources and indicators provide limitations of course but I think that it's interesting to make some reflections about the numbers in order to be self critics with this kind of sources because there are also inconveniences in some open access or access journals for example the price for the authors etc okay we have time for two more questions my question has been partially addressed sorry my name is Craig I'm from Norwegian Business School and it does seem to me that going to a database that's owned by a traditional academic publisher that seems to be a clear conflict of interest in terms of providing citation data and I would like to see an independent replication of this with a different database I don't know what the alternatives are but at the moment where in Norway we're looking at potentially using scopers to provide a citation indicator to base our financing system on as well and that seems to prevent a conflict of interest in my opinion yes so I'll leave that there but are there plans to replicate this with another database with more time okay two questions in the front another question concerning the data and I didn't quite get whether you said something about it but in Norway with this two level system it's a continuing problem that open gold open access journals are less visible so there are fewer open access journals among the very visible journals the very highly high impact factor journals I would expect that ERC grant publications would appear in high rated journals so did you have any sort of look at the one of the reasons behind a lower citation rate could be that these journals not because they are open access open access journals are often not so highly ranked at the moment that that could be an explanation a natural explanation for having less citations but in this case the analysis is based on the number of citations so the number of selections made by the scientists to those publications so it's not a problem of the visibility of the journal if the someone has mentioned before that if the paper is good it doesn't matter where it's published it's gold, green or non open access in this case the problem with the ranking of the journals is so I think because we just use number of citations so it's not a problem of the journal it's a problem of the individual publications included in those sources so one of the explanations could be that it's that is still not quite true it isn't true that you can publish anywhere but apparently if the source is available in Scopus all the publications published in that source are available in Scopus so but you didn't correct for that in any way so that is one of the issues in the data maybe, yes, thank you for the suggestion okay one last short question I think the source is possible but I'm really concerned about this because I'm working with Scopus and Seinfeld right at the moment I'm focused around everything that I do with caveats about the data and the information and one of the biggest ones I'd say not even I'm not going to talk about Scopus but is about the use of gold the publishing there are entire countries including my own country Ireland which focuses completely on green open access we know from our activities in the peer project that all of our ERC publications I should say beforehand are deposited in our repositories we know from the peer project that our green open access publications drive traffic towards the actual published versions of the journals so what we're looking at is a completely distorted view until we have an analysis of the green articles which we sent to the ERC we report in our reports so the ERC has a complete list of the green open access articles in our repositories until we see this analysis that actually tracks the impact of those journals linked to the actual published version of the journals then we will have no idea of what the real impact of open access is thank you again thank you we will now break for lunch and return here at 15 minutes past one thank you would you have one of those pointers to add the balance to the screen here you get it five and two I'll see if you can keep your microphone on until the questions are done so they can hear your answers yes thank you okay welcome back everyone our next speaker is Salvatore Melle he is a head of open access at CERN and he's also co-architect of the scope 3 initiative well thank you welcome back after lunch I have half an hour and 97 slides I'm going to over fast so I would like to discuss we had the presentation before about ERC a very specific institution and where the open access was making a difference in citations now what I'm going to do is to look at a very specific discipline high energy physics and where the open access is making a difference in something else which is downloads so we're taking a very orthogonal view to that and I'm going to talk about green, about hybrid and about gold high energy physics, you all know about this we have these very big machines which smash things in these very big detectors where there is a lot of people behind the detector and these lot of people then find these very beautiful things and we look at the paper we look like every other paper with the exception that is published by El Sevir but the copyright is CERN and this open access this was back in 2012 and this is what the papers look like every other paper roughly and one point I want to make now at the start of the talk is that I'm going to talk a lot about the publication in high energy physics but this kind of publications are very very very few 90% of the publications are written by one, two, three, four people so it's like everything else what is not exactly like everything else is that papers in this field have a double life, they have a secret life they start their life like vet on archive and then they become published later on and 97% of the yearly output in the journals is in archive there is much more stuff in archive than never gets published at the notes, conferences and things like that so before we discuss about the impact of open access in this field I would like to understand why, I would like to help you see why there is such a pervasive green open access because this then helps understanding how people use this information and where are the downloads and the access advantages again a very fast story so the story starts quite some time ago the story starts when there was no internet, if you wanted to send information to somebody you had to use airmail when people were using airmail they were writing papers this was in the 40s, the 50s and the 60s and the 70s at the same time somebody was writing a paper in the sickle style who remembers what was a sickle style come on come on everybody with white hair should remember that so don't be shy don't think I should say I don't remember sickle style that makes you young so you would pass them through the sickle style and the original copy would go to the publisher to get then send the airmail to a referee which would send it back with the comments and then send it again and then print it put the papers printed so they could go in a lorry and go in a library to be indexed and then put in the shelves three years later but all the copies which were produced in the sickle style would then go via paper mail immediately to libraries that's the same library in the photo of 1963 these are all the preprints received this week this gentleman actually still works at CERN, he's 93 now I'm actually serious but air styles have changed at the library desk and we moved the library desk on the other side of the room that's and I think that's the only change even the chairs are still the same so what happens is that in the years it takes for this piece of information to be processed quality assured produced and shipped the information has already been accessed in paper form there is something interesting when a library gets the preprints and the library acquires information in this way is that what the library does is actually to catalog it we still have the library catalog in the office of the library in CERN but you can pull it out and there are really the preprint numbers and they're all indexed by title by institute or even by person this is the open access repository in which we use it to put the preprints in the sense that everybody could go off office hours search by preprint number and get the preprint out and was on good faith that they would put it back in again and another library at Stanford they were actually having a list of preprints and they were typing the list of preprints in a computer in the 70s then something big but interesting happened every time somebody from CERN gives the talk we have to show that and with the arrival of the web something happened is that the list of preprints which was a computer in California became the first website in the US which was this one so the first website in the US was a list of preprints in high energy physics so that you could search and figure out which preprints were there and where they were published so you would go there and you would search for a preprint you would see if it was published you would get the there was no DOI, you would get the reference you would actually walk in the library you would hope to be in a rich library which would have received the cyclo-celled version by the scientists because of course you had only so much money to send out cyclo-celled versions at CERN in 1995 we were still spending 1 million a year in stamps to send out our own 300 papers a year to 3,000 institutions every single paper if you do the math even with discounted post-prices comes to 1 million a year in disseminating information then archive went on the web and what happened when archive went on the web this is as a function of time 92 to 2016 as a function of disciplines this is astronomy, this is high energy physics this is mathematics how much stuff was on archive the beginning archive was only high energy physics now it's only like 10% but if you see how many papers were on archive at the beginning 10,000 a year in high energy physics there's still 10,000 a year so every single paper which had to be put on archive in our field was put there in 1992 and that amount has barely moved meanwhile the question then is what happens as stuff gets published so 8 years ago we did a study, we just redid it it didn't change this is how much of the content of a journal you take every DOI which is in a journal of the 5 largest journals in this field you see was this paper on archive yes or not was this paper on archive before publishing yes or not you do this and 97% of what is published in a journal was in archive before since 1991 now if you do these for 25 years and you look now if I take everything which has ever been published since 1950 in the 5 largest journals in this field 60% of the entire knowledge is green as already there more on this later now the question is why was it so successful because there were mandates or there were incentives it's a matter of speed so what I said is that as of 1997 about 90% of the content of the journals was online on archive in 1997 journals were not online if you wanted something to be read on the internet in 1997 there was no other way this is a jewel this is a press release from 4th of March 1997 which tells us the company called Riegel Sevier will start a product called Science Direct which will put science journals on the web this year so you understand that for the time it took before journals went online there was no other way so one thing is speed the other is recognition this also rector Rice said at the beginning if you go back to the list of a list of preprints which was the first website in the US now it looks and this was what it looked like in 1993 now we redid it so it looks like the internet in 1997 so we tried to have this retro style because otherwise users would be a bit stressed we are going to do it and we are rewriting it now and now it's looking modern and the third iteration but the interesting thing in this this is something we run a certain together with colleagues in America and China in Germany, in Japan and in France is we have 1.2 million records 1.5 million open access papers inside 20 million citation and disambiguated authors which are basically half of the entire community it's searched like twice per second people are going in and tapping a search to look for a paper once you find a paper you have a tab which says citations once you go into the sorry you have a tab which says references who has who this paper is citing and then citation counts who has cited this paper now what we do some papers are very old here so you have physical review letters physical review letters whatever some papers which are relatively new you have the paper it blew because it resolves to another record in the system but the reference is an archive reference then once the paper gets published we add the publisher reference what happens is that we count citations between pre-prints and pre-prints between pre-prints and papers between papers and pre-prints and between paper and papers when something gets published we sum together the references of the pre-print version and the cited version which means that if in our discipline as an individual you go on google scholar wherever you go to count your own indicators you have more because we are counting the citations from the pre-prints and the pre-prints don't exist they are the dark economy of the system and then you get something like this where a retired physicist can enjoy reading how big is the H index and how many papers they have written and these numbers always come bigger and these are the published ones and these are the ones which are citable so there are things which have never been published and the numbers are relatively bigger so my H index is three units bigger for things that I only put on archive and I never bothered publishing so you see that there is an advantage in going there so which is the overall impact of green open access is recognition this is a study again that we are redoing now we did this seven years ago this is how many citations an average paper get if it was just published how many citations it gets if it is on archive how much bigger this is so at the time we saw the papers which were on archive and publisher were getting five times more citations the papers which were only published more interesting is the speed if you look at this plot again you look that papers are getting citations one year before they get published because they go on archive people read them from archive they write another paper they send it to archive so all this transaction does not exist in the normal ways we look at things because citations usually get counted when something is published and this process not only is enormously higher but it happens one year earlier by the time something is published in this field the amount of citations they get is already to big down to go down is already old so technically when you look at speed what the fact that you have ubiquitous open access means is that you have one year saved out of the time it takes to read and access and understand something so why was open access successful which was the impact of open access recognition more citations access because just stuff wasn't online in 1937 and speed and this is the same for astronomy and astrophysics because the ADS has exactly the mirror system as the one we run in SPI it was the same in economics with RPEC it was the same in the social sciences with SSRN so those disciplines just went and did the repositories because in 1995 if you want to put something online there was no other way now we have done a study of citations 7 years ago now we are trying to look into downloads nobody has seen this yet that's very preliminary we have worked with the team we shared with us anonymized blogs so we got two packs of logs from archive what people were doing in the second half of 2013 and the first half of 2016 we have taken only the papers that were published in high-energy physics journals and we just have looked at the average download per paper we cannot go any more specific than this for privacy rules which archive has very strict about and they are very right about that so you see there are very preliminary plots actually I have the real plots I just passed over them on the iPad so that they look more preliminary the error bars I didn't manage to do so this is the average number of downloads as a function of time so everybody who has to care about something on archive is the first two weeks then there is an endless long tail and this tail goes on for like 15 years and it's flat it's more interesting to look at this not as a function of the dating which something is put on archive but the dating in which something is published since we know the correspondence from every archive preprint to every DOI we can remap this and this is what it looks like so this is in weeks and there is a bump there is a bump around publication and then it goes down again in weeks up to, I don't know, two years so this was very strange I'm a physicist I say wow I discovered something here so what have we discovered here this is fascinating what we have discovered is that that's an average paper on archive I just put the first one I found in my email this was sent on the 20th of December and then was revised on the 10th of February so those two guys on the 10th of February they went on archive they submitted a second version where they add this comment EPJC which is a journal and they added DOI so what happened if you click on the DOI and you follow it is that they got published on the 9th of February so what they did the first thing they did was to go back on archive and they posted a new version of their paper since archive sends you an email every time something changes what happens is that you see this pic here of people who go and put a new version what we did was to check how often people resubmit to archive this is the frequency of resubmission before the DOI date the DOI date and later so within a month of the DOI what happens within a month of the DOI 70% of paper gets resubmitted to archive this is only papers in the five largest journals in high energy physics this number was 50% 10 years ago and now it's 70% so yeah there are the journals there but that's exactly what is the new social contract you go there and put stuff there so we decided to do gold open access now many people say why do you do it because you have everything clean and that's exactly what I want to discuss now because we've been trying to understand what changes the gold open access model has done so the idea of gold open access in high energy physics started in 2005 where this was the smartest phone you could buy but you could already see cats on the internet because it had just started so we started at the same time on YouTube so the way scop3 works for those who have not been exposed to this is that the researcher sends papers to publishers and doesn't care he doesn't pay APC he doesn't have to understand the copyright he doesn't have to put them in a report he has to do nothing he just has to do research and they keep the copyright a contract from CERN for the benefit of scop3 pays publishers to make everything open access the money to pay publishers to make everything open access is set around some notional APCs in reality we have a price that covers an entire journal but these journals were closed access before and we have flipped them to open so publishers are very founded to library the money which was used to buy journals before this money goes in the consortium and is the largest source of money to then free the journal later and some founding agencies support in addition in countries where there were not so many subscriptions before so what we have done is to reuse 90% of the money which was in the system to make journals open now this creates a very interesting study on what happens to these journals do people use them in a different way and that's exactly the kind of impact so we started long ago we have been in sustainable operations from 2014 to 2016 we run for three years now we are running for the second three years from 17 to 19 we are already planning to eventually become an invisible and sustainable infrastructure that you can tell is from Jeff we want to become invisible so we start to close the books financially and at the level of impact which has been the impact of doing this field where everything is green for three years funds have been allocated by 3,000 libraries and 8 founding agencies in 44 countries so this is the scope 3 partnership today we have published 13,500 articles these articles have been published by 20,000 authors in 100 countries the fact that you don't pay an APC you cannot create a barrier to publish in any of those journals and there is not even a negotiation which has to be done by the author whether you can publish or not you can see that the light yellow countries do not participate in scope 3 you can say that they are free writing or they are being supported by the system and publish the purple ones are the ones which are financially contributing to the system the articles that we have published are 13,500 with Elsevier in Dawey, IOP Jagiellonian University Oxford and New Japan physical society now the interesting thing is that apart from those three journals all those others are learned society journals which Springer publishes in this case on behalf of their society only 5 minutes left so I want to discuss about the impact what we have done is to ask Elsevier and Springer about the difference by making open gold things which was available green for free the answer was that in the first two years the downloads in total for these journals have doubled from the publisher websites now what has doubled is the access to all of the journals some of those journals have 80,000 articles inside 30,000 articles inside but what we have made open so we try to go a bit deeper and ask the publisher can you look a bit deeper what's exactly happening in this field the Oxford University Press and the Japanese Physical Society did a study this is incredible so this is a progress in theoretical physics this journal is very old so here is what happened to the journal when the journal was closed this was the average number of downloads per paper about 8 in 2008 so what they did is that every time they were getting a Nobel prize in physics in Japan and was published in this journal they were opening this article just these three or four articles is the blue stuff here then the journal went to open access before scope 3 and this is what happened there was a factor 5 increased downloads from the journal the moment they went to open access then the journal joined as scope 3 so many downloads the new articles only are getting since they joined as scope 3 the downloads have gone up by factor 18 is almost 20 and now here we see which are the new articles and which are the old ones so the last thing we have done is to discuss with a publisher which I cannot yet name to say can we get article by article downloads from you and we compare them with the archive article by article downloads article green and gold what are people using this is really preliminary so the unnamed publisher to whom I am grateful gave us their log files we kept only the archive preprints which are published in the Indo journals by this publisher so there are exactly the same things and then we sort them by age and we see what happens now in 2013 this is what happened this is 0 is the date of a DOI now remember that the archive is up there somewhere 3 floors above here because in the first week when something is there is downloaded so much more then from the last floor of the building you come slowly down here then something gets published so what happened in 2013 is that for article which where this is in month one year old the publisher was almost getting the same amount of downloads and archive this is closed closed access 2013 there was no open access and then slowly you get this effect for articles which are 5 years 10 years old the same article is downloaded 1.5 to 2 times more on the green open access then on the publisher side because everything is there anyhow in 2016 we have log files from the last 2 years where there are 2 big journals there are 6000 articles here so implicitly I'm telling which the publisher is what has happened once the publisher flipped the journals to open access nobody has ever seen this before now you do you go gold and archive has the same shape and this jumps up so what happens is that for the first 2-3 months before publication people still go and read the green version and then there is a larger number of access and the plot series not even 2 years and a half because we did the analysis last year and we only had the scope tree running for 2 years before that in the same time though this is the comparison for the publisher so you can see what happens going gold so we know the when we go gold it doubles the amount of our website is used in reality you have factors 3 to 4 for very fresh papers which are like 6 month old and you have factors 3 for papers which are about 1 year old and then you start to have factor 1.5 I've just 2 minutes in the same time what has happened is not that gold is hitting into green absolutely not this is all of archive log files so if I only take archive since 2013 to 2016 has gone up of almost a factor 2 in usage this is what has happened in high energy physics so if I only take the high energy physics papers on archive which are published on a publisher website archive still has gone up by 60% so what has happened is that we went gold archive went up by 60% for fresh paper and 200% on less fresh papers the most bizarre thing that nobody can understand is this these are articles which are older than 3 years this is closed articles there is no open access nothing matters these are the downloads in 2013 these are closed access articles there were some interesting articles which were about 3 years old and then it goes down it goes above you remember in 2013 archive is above the publisher in 2016 what happens is that the closed access long tail tall access stuff in the publisher website goes up over factor 2 and these are the same papers these are papers which were about 3 years old and then 3 years later these are papers which are about 4 and a half 5 years old so what is happening is that eyeballs and users who have access are going back to publisher website because everything is used more and the key point I want to say because there was a sometimes there is a bit of acrimonicity a bit of competition is green is gold is me we have big the bigger pie so we should be very surprised and we are very very surprised by the pie in the world flipping of journals to gold open access has massively increased downloads for everybody the downloads of the open access content goes up the downloads to the closed content in the same journals goes up the archive downloads increase as well and what's next is we are going to do 3 more years of scope 3 and I don't need to give you the details it's not that our average gold investment per article in the second year is going to be below 1,000 euros per paper thank you thank you Salotore for a very interesting presentation now we have time for a couple of questions yes my name is Yadran Kostronovsky and I'm coming from Croatia so I really appreciate what are you doing with scope 3 project but that was always wondering did you ever consider to organize a kind of open peer review around archive and just invest that money to some other level of quality the mission of scope 3 which I did not put on the screen is to flip to gold open access existing journals at zero burden for researchers so the idea was not to change the model that the researcher want to use but to make that model open so this idea has been considered but was not something that was felt by the community to be needed at this moment in time if somebody will start an open peer review or any kind of overlay journal which will be felt as a journal and become something that the community wants and wants to use which after all one of our journals was that in 1996 1997 then became an IOP journal then you see now that is in scope 3 so the idea is that there is no change here and all the change happens with invisible infrastructure to get the public value without a change here that was what we tried to do at this moment in time okay any one more question I was expecting more no it's very interesting what you're doing here what I don't understand is because you haven't said it now clarify is how is this business model sustainable with respect to what you have in mind and if you reach up a point where there's no need for other libraries to jump in because you already have enough money for the journal to survive and this becomes there for saving money for other libraries do you expect the libraries that are today paying to keep on paying for that how do you select the payers basically knowing that there are other libraries that are not contributing so three points on this how does it work as you can see at the moment not all libraries in the world are participating the way the system is built is that the very little bit of money which is missing which is very little we are going to publish our accounts in three months the very little bit of money which is missing for the time being is paid by my organization is paid by CERN if you just compare to the amount of money we are paying to put stamps on the preprints I mean this costs less if you think how much it costs to pay people to run institutional repositories this costs less so this is not a large amount of money especially for an organization which is worth 1.2 billion the key point to retain here is that we have already done this cycle now for several years after the first year you would imagine people would say I go away on the first year we had 1,500 libraries now we have 3,000 if any we see that the community starts to see that compared to paying APCA whichever price having a social pact where we all do it together actually works well the agencies start to come in because with what we are doing agencies are putting in like 10% of the value of a public APC now everybody is such that everybody is much better off if this exists that it doesn't exist it's the point of invisible infrastructure that you tend to support the common good so we feel like we are actually becoming sustainable because the community cares that's the only answer I can give you Paolo thank you we have to cut short now thank you again Salter thanks to you guys thank you very much okay today's program is a lightning talk session now due to the cold climate here in Norway our lightning talks are a bit slower so we have 10 minutes and we will have updates from 5 different countries starting with Portugal and Iloy Rodriguez he is the director of the University of Amino Documentation Services and his talk is monitoring a national funder open access mandate the Portuguese case challenges and working solutions thank you now it's switched on thank you very much I'll try to present in 10 minutes briefly what we are doing in Portugal regarding monitoring so it's a challenge to talk about the public funder the only similarity is that I have too many slides for the 10 minutes but as Salvatore I will manage to comply with my time so I'm talking about our experience of monitoring our national funder in Portugal there is really just one big and important public funder there are some other small public private funders who have been working on the FCT foundation for science and technology since 2014 FCT has defined an open access mandate it's a green open access mandate it's based on institutional repositories it allows for embargo periods from 6 to 12 months there is a requirement of associating Creative Commons license and as you can see by this it is very well aligned with the requirements for Horizon 2020 and this mandate and this policy was built on top of already existing infrastructure that is the CAP is the national open science repository that was built in Portugal since 2008 and is aggregating overall the Portuguese repositories so for this policy and for the monitoring of this policy we rely on the repository infrastructure so we have the repositories where all the publications must be deposited we have the CAP, the national portal that aggregates all those publications and identify the founding information in the middle of that we have a validator a tool that validates the information that we aggregate from the repositories and then we have the reporting and monitoring that is made for the founder if we look a little bit closer to each of these components so for institutional repositories the main question of course is the identification of the founding and of the projects so not surprisingly we use the open-air API to do this so every Portuguese repository apart from Horizon 2020 and the FP7 projects has also a list of authoritative list of all the FCT projects and because again this question as it was mentioned in the morning we clearly identify the founding is a problem and so this authoritative list is very useful and then all this information is aggregated by the portal and with that information and with the metadata that we aggregate from the repositories with the information about the founding of the projects we have like also in open-air knows how open-air works this is very similar to what our open-air works so we have a project page for each project where we have some basic information and then the list of publications and some tools that the project can use and also from this information the cap portal produces the official publications report for the founder so it is a static so it is an automatic generated report that can be generated at any moment but when it is generated it is a static report it generates a URL that can then be used for reporting purposes and also on the cap portal we have a monitoring dashboard that allows us to collect and to present and to export information about the founding I will just give you this number this is not very important let me just remind you that the mandate was from 2014 the first grant agreements that have the mandate incorporated from 2015 and the number of projects is still very low because of the crisis that we had in Portugal the founding has reduced we are now starting to recover and the number of projects is still very very small but I want just to you because I will be back on the last slide to this number of 5800 founded publications that we have identified on the cap portal because I want you to compare that at the end of my presentation and finally the report and monitoring so it is important that in Portugal contrary to what happened currently the only official way to report publications is through the cap portal so every publication to be reported on the project has to be on the portal because otherwise it will not be reported to the founder currently we are still doing it semi automatically so they just need to copy paste that link that I just saw into the reporting system but it has been worked out to be completely automated so all we are working on the monitoring so in a way we do the monitoring in two ways so for one day of doing the reporting if we just consider the publications that are repositories then you can say you get a reference because all the other publications are not counted for reporting purposes but that's not completely true because we know and we have also some evidence of that that some publications that are referenced the funding from projects are not still currently on the repository so we want to have completely accurate numbers and know percentage of publications that are really compliant on with the policy so we do that by using of course the project identification using the information that we have on the search portal and on external sources we are still working and modeling different possibilities to do that Authenticus is a national system that uses cross-ref web of science and other sources to grab metadata so we are still deciding this and then of course the monitoring is done, I'm just finishing is done by the statistical department of the government and by the FCT so the final remarks so the first thing is really to and this is a big problem is to identify all funded publications because first because there are problems on the project identification because especially in countries like Portugal and especially in social science and humanities there are many publications that are not RZI and Scopus so we need other information sources to really have a complete view of the publications and even for the sources that we have I just want to show you for instance if we go to crossref and then compare with the 5800 if we go to crossref and see what is there on FCT it's I think 6700 publications if we go to open air we have already 11000 publications so again there is a problem on identifying what is the total that we want to compare with for the monitoring and of course the other challenge that we are addressing is how we will relate our effort our national effort because we don't want to duplicate with the efforts that we need to do at a larger scale at the European scale with open air and also at the institutional level because again also the institutions want to know how they perform on their compliance level thank you are there questions? Keep it on Thank you Eloy We have time for a couple of questions if there are any Again thank you Eloy Our second slow lightning of talk of today is Hungary and András Hol Deputy Director General of the library of the Hungarian Academy of Sciences In this talk is open access mandate of the Hungarian Academy of Sciences how effective is it? Hello the Hungarian Academy of Sciences is a rather complex organization I won't speak about the Hungarian open access landscape in general but you will see some references in the slides but I have to speak a couple of words about the Hungarian Academy of Sciences and its mandate the Hungarian Academy of Sciences MTA is a learned society and it's also an owner of a chain of research institutes and also acts as a funder so it's a very complex organization and it has a mandate which is color neutral so it allows gold, green and even hybrid it covers books as well and the way to monitor such a complex mandate is only is provided by the Hungarian National Scientific Bibliography which is called MTMT I want to speak about the impact of the mandate those numbers are the number of articles in the repository of the Academy and I have a pointer here the mandate became effective in 2013 you see there is a big jump in the number of deposited papers we have just finished revising the mandate in last year and learning about the experience of the first three years we amended the closes of the mandate first of all we allowed somewhat longer embargoes for humanities and social sciences the argument is that the papers there are mainly published in Hungarian in the national language and the market is very small Hungarian journal publishers are not having such a huge profit margins like the multinationals I can say that some of them doesn't have a profit at all so to keep them in business we need a little bit longer embargoes so now we allow two years embargoes for humanities compared to one year embargo of the science technology medicine field we became a little bit more specific about exception handling directors of the research institutes can provide waivers also in the first version of the mandate we didn't specify any license no we specify that we require or we recommend CCBI and CND this is because there was a big worry about mainly researchers in the humanities field that if there is no license at all or is it in public domain then people might think that they can just grab the publications and republish them somewhere else so by their request no we recommend this CC license which doesn't means that a more liberal version namely CCBI couldn't be employed and also we also specified the venues of open access more clearly so open access can be in an open access journal which is in DOAJ in Hungarian repositories and we have a national certification system for those and the database has a list of trusted international subject repositories like Archive and the others about monitoring so or national publication database MTMTE can monitor open access all the open access journals in DOI G are recognized but if a certain paper gets deposited in a repository this is something which should be entered into this database by the author actually they can use a SWORD protocol to upload their publication from this MTMTE database to a selected repository and if this tool is used there is no need of manual tagging of green open access and this applies to hybrid as well if an article is made open access in a hybrid journal it should be entered into this database and let's have a look at the results for the Hungarian Academy of Sciences the numbers are close to 60% for 2014 and 15 means that very close 59. something percent of the publications from those two years are either open access already or under embargo and will be open access next year or so which is I think is a pretty good number and we can compare the statistics on major universities in Hungary those numbers are for the five major universities and their output access ranging from 13 to 30% what is interesting there is no clear correlation between a mandate between the existence of a mandate some only two universities have mandates in Hungary one of them are close to 30% the other one is somewhere in between the reason could be that those mandates are fairly new and the researchers might not be aware that they actually need to tag their open access articles in this empty empty space so thank you thank you Andrew are there any questions nope thank you again okay our third lightning talk today is Croatia and Jadranka Stojnovski I did it she is an assistant professor at Ruder Boskoic Institute University of Sada and her talk is impact of open access on research in Croatia small incentives big achievements so for this talk I decided to take very good according to my opinion starting point of classification of impact of open access and this classification is focused on three groups actually readers, authors and institutions and in my short presentation I will add also two groups which are editors and I forgot the fifth one so Irina told me before that I am too pessimistic sometimes so I decided to start with the worst part on the beginning and that's really strong impact of the politics in Croatia which is not positive actually usual so we have many governmental changes and they have quite different priorities from open access so all projects we had and which ministry is supposed to finance in the form of year stopped and very actual in Croatia some ethical issues some members of the government are in some plagiarism stories involved and so on so there is a lack of appropriate recognition of open access and this is the context of the second part of my presentation and incentives are really small we don't have any national strategies reports, missions, visions, mandates and what is worrying that it's not easy to see government goal and what is our future what is this point where Croatian government wants to bring the country so these small incentives results in that we are just looking for some sentences in present strategies or acts or laws and when we find something public or free or then we use it for our initiatives so for example these two sentences from the scientific activity and higher education act we use for creation actually of national infrastructure for institutional repositories which was launched in 2015 and at the moment we have more than 100 institutional repositories it's just a year and a half so and it's really touching to see 10 working groups with a lot of members more than 100 people working voluntarily on institutional repositories which are not fully open access at the moment you see that almost 30,000 deposited works and only 12,000 is in open access so there is quite of conflict between active scientific activity and mandate to put PhD Master of Science and Bachelor of Science thesis in open access with creation copyright law and some issues should be solved and we will be grateful for any help in that area so speaking about impact on institution I will just take as an example Roger Boschkeich Institute institutional repository full year not so much the number of papers is not big but 90% is in open access and it's really nice to listen how researchers at Roger Boschkeich Institute are speaking more and more on open access even they are ready to put the content in open access which are actually not allowed to put so if you compare the downloads in 2013 and downloads in 2016 you can see that numbers are quite bigger from 1,000 to 4,000 so four times more so and related to authors actually they are familiar kind with open access from 1996 when the creation scientific bibliography was launched and this bibliography from the very beginning serve also is some kind of orphan repository so if you don't have repository at your institution you can provide access to paper through creation scientific bibliography the number is not impressive but they are still what is nice to see that this number is getting higher and higher and then we connect creation scientific bibliography with DABA repositories that will be really good so open access has maybe the most impact in creation on editors and publishers so speaking about creation journals maybe the majority of you heard about repository of open access journal so at the moment we have 431 open access journal with 160,000 full text articles and it's really huge impact on authors, editors and publishers that when we started in 2006 it was not open access it was not mandatory but maybe sometimes when you are into the desert and you have a beautiful flower coming up that's the similar situation so creation journals were lacking during whole history some copyright statements or they didn't have agreements with authors it was sort of chaos but from this chaos this huge adoption of open access 10 years ago was really fantastic so recently we find out that creation open access journals are not real open access journals because most of them still don't have statements on copyright and licensing so and during the publication to director of open access journal last year really the number of creation open access journals dropped down almost in half in this directories so we are now working with editors and publishers to solve this and finally to declare this open access statement so and I will just keep to this but it's important that since really advanced one were very related to the grant we got from open air on improvement of herchic repository and ministry of science and education it's actually subsidizing from 400 journals 160 are subsidized by the ministry according to set of criteria and what is good that is really advanced that if you want to get money from the ministry you should be on herchic and your content should be in open access and why creation journals important it's also related to automatic data for example you can see this paper from the field of social sciences in it's on employment of women related to mobility in Croatia and you can see more than 1500 downloads and more than 3000 visits but only one citation is there so it's really important to look also and herchic has a huge usage so what we want in the future and I will finish with that to really to establish some teams bodies, agencies on dealing with open access and open science and especially with research assessment because small countries and small scientific communities like Croatia are relying too much in research assessment to databases like Web of Science and to only to prestigious journal with high impact factor and so on so we are preparing to build creation system and what we would like in the future to motivate researchers to adopt open science not to mandate and and what will really make us happy is that our future strategies will go beyond PDF and this paper-centric way of publishing and article processing charges business model thank you thank you any questions no thank you again now our fourth talk is Norway and Lars Eglom the director of learning center and library here at our host institution today Oslan Akershus university college of applied sciences his talk is strategy and implementation of an open access of open access at HUA I'm honored to have you here at Oslan Akershus university I would say because we sent our application to the department to the ministry some few days ago to be a university it's a strange thing this last period that we are called a university college in Norwegian but the English translation is university because we have we have the resources to be a university in the European definition of a university I'm not going to talk about open access in Norway but tell you a little bit about how we work with open access in this institution I think that isn't very different from many other institutions in Norway but I think we have reached a little further perhaps than other institutions this is just to show you that as early as in 2011 the board of the institution decided on a policy for open access you don't have to read the text I guess open access policies look almost the same in many countries but we started in 2011 and we made a publication from and the point of that is that it covers open access journals, books and open data repositories we established our institutional repository ODA open data archive also named as I'll show you after a Norwegian person today we have 2800 scientific documents and 800,000 downloads a year and that means that we have far less documents than the big university in Oslo Oslo University or the technical or the university in Trondheim the use is on the same level as the repositories in Oslo University and in Trondheim ODA is ODA Krog the picture she was not so famous as an artist more famous as a woman or a femme fatale but it was a good name to use for our our repository peer reviewed journal articles approved master thesis approved PHD thesis and the learning center and library is responsible for running the archive and also responsible for copyright management and we heard about high downloads the first 2-3 months and then a long tail we have some documents that have a rather thick tail so this document has been downloaded 6000 times since 2010 that is 2.5 times a day and 2000 documents 2000 downloads the last year it's about library then when we started with our repository the scientific staff wanted to have one repository for only scientific documents and then we started with I have translated as the academic archive only contains 700 documents but that's going rather fast and that contains it's for professional or academic publication not scientific not peer reviewed and also students assignments with grade A or B the student as a researcher in education and profession 10,000 downloads so it's been used and we also publish open access journals we have 12 journals 11 on level 1 with support from the Norwegian research council the library offers technical infrastructure to establish the journals by using open journal system but the institutes or the faculties they have editing boards at the institutes we had 100,000 downloads in 2015 and that is for our university which or used to be a college then with a lot of professional educations these journals has been important to establish some professions as an academic field so what have we achieved we have 13.5% of the publication from our researchers are published in OAA open access channels gold channels 90% of the peer reviewed articles are deposited in Christian of these 50-60% of the last 5 years are open available in OAA and that means that if we look at what have been published by our researchers for the last 10 years approximately between 50 and 60% is available open access either in gold publications or in our archive and the average for Norway is 13% I think but there is huge differences between the institutions and of course you heard Kurt Reiss so you know that we have strong support for the open access work we do but it is open access because we have a policy for openness and sharing so open access concerns the scientific publications but we are also very willing to share our educational resources so we have our publication series published in paper and in open access we have here a film archive with 650 educational films open to everyone made by those people who are filming here today and we also produce MOOC and learning resources available for everyone on the edX platform that was a cooperation between the library and the academic staff especially on the health field and we started last fall with 1000 students following MOOC anatomy and they had their exam and the people that didn't make their exam was reduced from 36% to 6% so we really see that that works yeah, thank you and what I have been telling you about is that we have worked on by Trude come to raise up Trude Eikebrock and her team in the library so if there is some difficult questions thank you Lars any questions nope, okay we'll be gone to our fifth and final slow lightning talk and it's the UK and Katie Shemesh scholarly communications and lists at disk and her talk is article processing charges and subscriptions monitoring open access costs in the UK hi, so I'm going to be talking to you about article processing charges and how we monitor these in the UK and what what we found, what some of the results we found from these are so just to give you a bit of background about this project in 2013 when it started we knew that institutions were paying APCs we didn't know how many or how much they were spending they weren't being recorded in any kind of standardized way so my predecessor at disk developed a spreadsheet that institutions could fill out and we asked them to fill it out on a voluntary basis and what helped it to really get off the ground was that it was adopted by funders for their own reporting there was a mandate to report to them how much had been spent and on what articles and they could use the same data format to report to them when it started in 2013 we had 17 institutions filling out the spreadsheet now last year in 2016 we had 40 institutions and the same this year we just did around the data collection and we have data on over 30,000 APCs and it's all openly available CC0 license on Figshare and the principles behind this initiative were that this data should be standardized and comparable it maps to metadata standards whenever it can and that it should be open and available to everyone to use and just kind of a sub point that it should be as easy for institutions to give us this information as possible and the reason that it was important for it to be open is because for us at disk we do negotiations on behalf of the UK University we want to be accountable to them to show them that we're working in their best interest to have the data available to negotiate with publishers to make deals that really work and it really does help our negotiations with publishers and we get publishers as well coming back to us using that same data that we have put out online and that's great because then we can verify it and we know where it comes from it also helps institutions to see that they're paying a fair price compared to others and it helps funders to know how their money is being spent and to set policies that benefit open access and the sector as a whole and really what's important is that this is creating a transparent market for APCs so a big difference from the market in subscriptions now people know what they're paying, how much everyone else is paying and it's obvious what this market looks like so here are some of the things we found with the data that we collected the first thing we found APCs are growing and they're growing really fast so between 2013 and 2014 we saw more than double increase and this is because we had some funding that came into the UK so all of a sudden there was a lot more money available to spend on APCs but then between 2014 and 2015 that funding didn't change much but we still saw quite large growth 9% growth in number of APCs and 20% growth in expenditure and the reason that the expenditure is growing so much faster is because the average APC is rising so it rose about 100 pounds in the past two years so we saw a bit of a dip in 2014 when we introduced some offsetting deals but by 2015 the average APC had gone up quite a lot and this is really important because APC expenditure is becoming a big part of library budgets we found for 2014 that we had subscription data available that it was about 12% and these are just the APCs we know about that are being paid by the library central fund and subscription expenditure was the other 88% but in 2015 we imagined it would be more than that and this means that subscriptions are really competing sorry, APCs are really competing with subscriptions for a share of library budgets and we have data that shows that library budgets are being squeezed because they're not rising to keep up with inflation and that's why it's so vital to consider subscriptions and APCs as part of the total cost of research and it's even more vital because the market for APCs looks a lot like that of subscriptions we're spending on the same publishers with the exception of some fully open access publishers like Public Library of Science and we're spending on the same journals 80% of what we spend is going towards hybrid and that's partly because more articles are being published in hybrid journals than in fully open access journals but it's also because the hybrid average APC is significantly higher than the full open access APCs about 500 pounds higher on average but worryingly we're also finding that the full open access average APCs are rising a lot faster than that of hybrid so the gap is starting to close between the two so just to sum up we're spending more on APCs partly because more articles are being published Gold Open Access and partly because the average APC is rising and the APC market is looking a lot like the subscription market money is going to the same big publishers it's going to the same hybrid journals and the cost of full open access journals seems to be rising to meet that of hybrid but if it's well sounding a bit depressing and a bit doom and gloom I want to reassure you that it's really not that bad just the fact that we have this data available means that we're in a much better position than we were a few years ago when we did it and we've used this data successfully to negotiate offset deals with publishers where APCs and subscriptions are considered as part of the same deal and these are just some of the offsetting agreements that we have negotiated and just to show you what this looks like in practice in 2014 we negotiated a deal with Taylor and Francis you can see that's the purple line and the subscriptions dropped down we negotiated a deal with Wiley that's the brown line at the top and you can see that sorry the adventure dropped down and the one publisher that Orange Line that we didn't have an offsetting deal in place you can just see what the effect was what the expenditure just went up so high so what's next for us is to further standardize how APCs are reported especially we have a problem with how offset APCs are reported and how do you determine what those cost and how should we record them we want to ease the burden of reporting for institutions especially as institutions are paying more and more APCs the time that it takes to record these and track these becomes quite a bit more than it was before and so we've got two new products available within GISC Monitor Local and Monitor UK Monitor Local helps to record APCs in a really user friendly way it automates a lot of the process and Monitor UK takes all that data and gives the national picture it generates reports of what's going on in open access in the UK we've got 13 institutions signed up so far and I think another one on the way 14 institutions and so if you'd like some more information about this I wrote a report that goes into quite a bit more detail and you can find it here and so that's it for me okay thank you Katie any questions yes one question here just wanted to ask is a very very nice informative presentation just wanted to ask if you've totaled the numbers so are you doing comparisons years by years of how much you were spending and how much you're spending now in total subscriptions APCs and some subscriptions APCs now yes so I think you can find that on the report it's a bit difficult because the subscription data comes from a different source so we only have subscription data openly available for 10 publishers but yeah I think I have done that and also you can find it there one more question at the front row here you mentioned those deals you are reaching with the publishers is data about those deals public I mean it would really influence other countries as well to see how the UK or the Netherlands or those countries manage to deal with the publishers yeah you can find information about our deals on the discussions let's say okay one more question Eric Loris from the Federal Science Policy Office in Belgium I was wondering you note that the APC, overall APC cost is rising because we pay for more APC and because the single APC level is rising have you had any chance to look into the reasons why the level is rising yeah so I mentioned before that there's the hybrid APCs are rising quite quickly but that the full open access ones are rising a bit more quickly partly there's a shift towards more expensive, more prestigious fully open access journals and another part is just an increase in APCs from certain popular journals that people are publishing in already quite a lot okay I think that was it thank you again Casey well now it's now time for a 32 minute break see you back at 15 minutes past 3 I think I am, I think this is on okay welcome back it's time for our last speaker before the panel discussion which will round off today's program and he is Carlos Galandias and he is a research impact officer at the University of Glasgow thank you good afternoon everyone I was deeply concerned about the timing of my slot especially after the coffee break but then after such a day of amazing speakers I am deeply concerned now nevertheless just to recap one of the opening questions that Natalia asked this morning she said is the impact of open access measurable and if so how for research for innovation, for society how does it relate to research assessment for researchers and for funders I don't think I will be able to answer all of those questions but hopefully I will be giving you some indication of what we are doing at the University of Glasgow if I had those answers you would be paying me to solve your problems so I am one of you I am trying to get to the bottom of these things and just a warning this presentation may induce sickness and hopefully it is just the transitions and not me so if you feel that it moves too fast every time I click so just to recap is this not working there we go ok close your eyes I don't know why it went to the end there we go so we have a very quick overview because I don't really need to tell you what the open access is and if I did perhaps we just leave the room right now but just the overall overarching aspirations of open access which is access dissemination promptness of the communication of research data and I am using the word data to cover results as well and communications but also it has some social justice aims which is the within and between community equality just to get a bit of perspective on the challenges that we are faced and I think this is pretty important as of last Friday when I was just updating my slides Rohrmap listed 769 open access policies distributed in the following way which I think is a point of reflection for us because for something that is supposed to be open access and interoperable we are not doing very well of course this is down to institutions it is down to governments it is down to intergovernmental working but we are still I think quite a long way of streamlining never mind how to access documents but how to agree on what is important so I think for those of you who are macro thinkers there is a lot of work to be done with that regard and again another note to consider depending on which sources you visit today we have around 6 to 9 million active researchers across the world these are full time equivalents that is a hell of a lot of researchers in 2009 it was estimated that we have gone beyond the 50 million article mark peer reviewed articles that are in repositories and this doesn't include thesis or the repositories for pre prints that we heard from Salvatore and I suppose there is probably another million or two in the last three days the annual articles output as measured in 2013 was an estimated 2.4 million items and the interesting thing that unites us is that depending on who you site there is only about 25% of all these corpus available on the web which is where most people get their information or use that to then move on to a library or to a particular repository to get that so it is important it matters irrespective of where we get scopus or the other database matters so what does it actually do which is the question that I'm trying to answer and as I said it's a partial answer so I went to our comrades in Gisk and stole their diagram sorry it's usable and reusable and this is a beautiful illustration of all the things that open access is supposed to do sorry I'm jumping ahead of myself so basically it allows pretty much everyone in this planet to go and look at the research and in order to understand the magnitude of these the potential to recognize readers as of 2015 numbers for example our friends Elsevier reckon they have 10 to 15 million unique users or identifiers academia.edu has 16 million registered users a lot of these will be shared between them it's impossible to know and well Google Scholar who indexed a lot of articles claim that they have a substantial share of the market but when you drill down into their actual data it only happens to be the last 5 years of articles which is still alone but it's not as impressive as they normally claim it to be so let's move on to the meat of the talk so what are the academic impacts I think that without being too pompous about these I think there are clear benefits of course there are points of contention and how you measure something at what are your benchmarks for this but it is clear that open access help science so it's a help in itself into the same endeavor and if we look at some of the quotations out there according to the National Science Foundation the average citation for all countries has risen from about 1.7 in 92 to 2.5 in 2012 so that in itself is the inclusion of the World Wide Web which is quite phenomenal and depending on who you look at there is a very good paper I've got a list of references for those who wanted who have done a meta study of 27 studies about open access and there's evidence that varies between 36% increase to 600% increase of citations due to publishing in open access journals so I think we can confidently say that irrespective of methodology open access increases visibility and in some cases leads to an increase in citations for example if we move on to some other here on the metrics I was telling you a minute ago the societal sphere which is what these numbers are appealing to there is a lot of people out there accessing the data which may not convert and this is a question for you who are skillfully in databases and macro data analysis what's happening there because with these millions if you only have an increase of 1 or 1.5 in citation data what's happening with those articles what's happening with that information it is clear that downloads have increased due to open access and again 2015 data shows that there's 2.9 billion downloads per year and if we drill down to the UK which is where my institution is the high education sector reported 102 million full text articles downloaded in 2006 and 2007 so the question that I asked myself because at the end of the day I'm a psychologist is what are people doing with these downloads do people actually read them why don't we see this in the conversion of citations and I think this speaks about human psychology and it connects to Salvatore's point about this is a human problem this is about how people behave it's not necessarily about the infrastructure but I'll touch on that in a minute of course there are economic benefits and I think that with most things in life this economics matter a lot and we do know that there are significant advantages and disadvantages for open access so publishers of course some of them are rejoicing in the fact that there's open access just like Katie told us about they're making a fortune out of this and to give you an example of how badly we're doing I've been open about this somebody mentioned Stier, Lawson and Ben in the United Kingdom their study from last year which was a representative study about institutional subscriptions and just to give you an example of the disparities for the data they obtain for the year of 2014 the discrepancies with regards to Cambridge University Press they charge University London £60,000 for their subscription and Staffordshire University in England paid £164,000 and Elsevier another example from 2014 Imperial College London paid £1.5 million for their subscription whereas University of Bradford paid £21,000 so what is happening there is a big business economics matter so this then transpires into the rest of the University environment so we have researchers who can do more with less there's institutions who cannot do more because they don't have the resources we also have the develop and the developing countries issue which may not have access to these resources and therefore open access is important and then we have the aspiration to include the public sector and the private sector and the third sector but in my daily work I'm yet to find a policy maker a government worker or a person in the third sector asking me for an article because that is not what they normally read so that again is something to do with us on how we translate our research papers into something that is easily accessible for these people so I think this is a point of question on how these economic benefits actually transform the way that we do our research sorry I do need to look at my notes as well as the presentation so do we really have evidence for the societal benefit claims and this is an open question because I don't think we do I think we have indications of how these things work but I'm going to give you an example that I actually received an hour ago from Impact Story you know Impact Story all of you they sent me this amazing email that just made me feel really well that says dear Carlos here are your achievements so I have an 88% compliance with open access sorry 87% of my research is free to read online this level of availability puts you in the top 10% of researchers I'm not a researcher by contract I do research so I've got a top 10% in that then I have a green circle called greatest hit which is the top 50% that says your top publication has been saved and shared 12 times only 36% of researchers get this much attention on a publication and then there's a blue circle that says global south top 50% of people who save and share your research 21% are in the global south that's a high proportion only 33% of researchers publish work that inspires this level of engagement from the developing world so the question is so what what does that actually mean and I'm going to get this towards the end of my talk because I think we have to define as a community what impact is because impact has many facets the academic ones I think that we have to be bold and move on and accept it as the factor it is good for research it increases visibility it increases usage we should accept that and move on but what about this and I'll explain how we do some of that at Glasgow and some other universities so I think that with regards to that in that respect I would say that open access is unnecessary but not sufficient condition for impact outside our usual metrics we need to close the loop and what does this mean I have to ask this question all the time because that's my job I go around my college I've got about 800 academics and I help them realize the potential of their research and the scariest question that somebody can ask an academic is so what you publish 100 articles so what who cares because as we know you've been cited 1.5 times so that doesn't really make a difference if we do a very clever economic analysis of all of these we could arrive at some kind of monetary value but it still doesn't answer the question of what the impact of that is just tells us that somebody cited your work so Kurt Rice this morning said how are we going to disrupt the system and I think this is one of the ways that we can disrupt the system and I believe that we have to reconceptualize the understanding of knowledge in a digital economy because we live in a digital economy all these things that we're talking about to be simplistic whereby our users whoever they are and at large understand the attribution challenge and join the game of acknowledging because I believe this is the only way of closing the loop if nobody acknowledges the importance of this work either for your own work or for another objective will still be in the dark on what happened to that work so this is a quasi-immoral proposition because it is appealing to how people behave and it involves raising awareness and re-educating the user base which is no mean fit so in order to frame this problem I think one way that this can be done is framing this challenge in terms of three interwoven contexts these are the individual the social and the material so the individual context includes all the factors held by the individual that affect the choices and the behaviors he or she undertakes this includes an individual's values, attitudes and skills as well as the calculations he or she makes before acting including personal evaluations of costs and benefits so these are the people who make the research and those who read the research and I know that it's difficult to think in terms of the whole system but our non-academic users, whoever they may be do they understand this the claims of open access are clear within universities and research but are these people outside actually aware of this how many businesses have turned themselves into multi-million dollar companies without ever citing an article some of them may have read some articles some of them may have come to a talk, got an idea and off they went and I think Salvatore illustrated that very eloquently in his talk because he said this is a social contract so with regards to that I think we have to rethink this as a social contract where everybody understands that we all benefit now high energy physics seems to be highly evolved in the understanding that some of the system is better than individual contributions but I don't think that in the social sciences for example where I come from in this game we very seldomly put out an idea that is half constructed for fear of losing that idea or losing the acknowledgement of that idea and then if you take that out of the university circles you're in bigger problems now the social context which I've just lumped into that because I'm a psychologist so the two go hand in hand which includes all the factors that exist beyond the individual in the social realm yet shape his or her behaviors these influences include understandings that are shared amongst groups such as social norms and the meanings attached to particular activities as well as people's networks and relationships and the institutions that influence how group of individuals behave and I think this is extremely important and finally you have the material context which is the factors that are out there in the environment and the wider world and I think is where you come in with the policies and the ways of doing things and I think the question I would like to ask is can we not use our own systems to actually start tracking how this particular changes outside in the world can be measured and perhaps it shows my age but you know a risk tag would be fantastic because if people took the care if we could re-educate people to actually input that metadata we could then start seeing how a particular publication whatever that may be has changed something now we have the techniques available is it a big task you tell me can we retrospectively map all the IDs unique identifiers to this risk tag that will be about impact for example all the acknowledgements in papers can be data mined and attached to a particular author so you can actually see who's contributing to a field but it's not properly acknowledged as a co-author here's a question that sorry Rachel only proved me wrong already as of yesterday evening there were 2.3 million orchid users but now it's 3 million I don't know how it jumped they probably haven't updated their website but are they citing their work that has non-academic impact or are we still focusing and forcing these people to just focus on academia per se this is incredibly important for us in the United Kingdom but also for Europe we all have a commitment to doing something outside academia with our research Europe is still not imposing these our motto is academic excellence with impact in mind so whatever we do we have to demonstrate what has happened as a result of that research and finally technologically speaking can we mine IP addresses to the point that is sensitive that is not in breach of any confidentiality so we can actually map where our research is being read and people can actually use that as a proxy to approach those organizations that may be wanting to use that research so I'll show you 2 examples to conclude because I'm getting close now of how we do this this is a little diagram called the personal impact plan and it has an amalgamation of definitions but when I talk about impact I'm talking about the demonstrable effect change or benefit beyond academia it includes academic benefit but it could be absolutely anything and I think this is what terrifies people because if there are no guidelines that fulfill them and I cannot tell the community to just look everything but this is part of my project to raise awareness and change how people behave so they can actually start understanding that they can no longer just sit in the room and produce their articles and waiting for the citations to happen some people may be able to do that some people are very good at writing, we should support those people who are very good at that those who are not very good at writing or are not very good at getting citations need to start working out different ways of working so they can get to the next stage in this impact agenda so this has very simple questions I have copies here if anyone want to take one at the end and I already have to update this in some way or form because it changes all the time but I must stress I do not worry about open access in my institution because my library does a great job of that we have a system that works everyone has to be open access compliant because of the higher education sector in the UK our repository is both open to individuals but also you can delegate someone to fill in your information for you we don't have a problem with regards to that and we have the mechanisms to pay for it irrespective of the cost and the transactions that Katie mentioned earlier on the example the final example that I would like to talk about is how do you close the loop well we have this repository at Glasgow it's not unique it's enlightened everyone can have it but we decided to redevelop the repository to actually start gathering impact and knowledge exchange evidence because this is the only way that we can close that loop and go back to the funders and tell them what has happened as a result of your academic activities so if I was to do this myself you can deposit an item which can be an activity or a piece of evidence because remember it has to be demonstrable so I cannot go back to my office on Thursday and say oh I had a great time everybody loved it I have to find something that tells me that you did so it could be my 12 tweets that have reached the global south or it could be an email that says that this was very interesting or for example the reason why I am here I was at the conference last year I met Nina we had a chat something ran through in her head and she invited me to come so if I report back on this that is a measure of impact because thanks to me giving a presentation something else happened and that is the only way that I can reconstruct that famous impact outside academia so this repository is working it's got many many fields it's interoperable with all of our systems it has research funders data it has HR data it has contractual data and anything you put in gets curated by the library and as a research impact officer which is my ridiculous title I can sit down and mind that data and create connections between researchers I can look at who are who we are working with I can look at the opportunities that we have in the pipeline it connects with all the publications so I can actually map an entire college by publications I can look at those publications with regards to organizations that we work for I can find gaps I can find strengths and this goes hand in hand with my personal impact plan and I was speaking to somebody earlier on at lunchtime because we have to generate trust and report with our researchers we cannot just impose a vision on them and wait for them to change we have to do the network and show them how it can be used so as a result of this we've already had some successes which again goes back to economics thanks to some of this information we have put researchers together and won very lucrative research funding opportunities so we're at the point where people see the value and they can sit down filling the repository this fits into their annual reviews for career progression and everybody wins and if this was to fall down tomorrow they can download all their data and take it with them and put it in their CV so we're making their life easy because whatever they do is portable thank you very much thank you Carlos for a very interesting presentation we have time for questions we have one question up there I have a question on how do you practically do that with that database enlightened do the researchers just fill that out do they want to do that not all of them but they do because so this is back to the material and the social the way that UK higher education is going to impact you could be the best publisher in the world but for promotion and for acquiring certain funding you must do impact activities which tend to be knowledge exchange activities that may or may not lead to impact so the fact that your promotion will not be granted because you haven't done knowledge exchange activities is quite important it's measured and also an aspect of this is that because of this thing called the REF that most of you know which I've avoided like the plague in this presentation you have the opportunity to be picked up for an impact case study the more you use the repository because I couldn't possibly go around 800 people and know what they're doing so if they submit their data I get a better chance of what they're doing so it's in their best interest so you have to set the infrastructure where you're going to reward the positive behavior and the negative behavior you're not going to sack them because you cannot do that but you have to like Salvatore said show them the way and most people know that they're missing out if they don't engage with the tool we're about to relaunch the tool we did some technical user interface on it and we have a series of workshops prepared for the coming months it's still low usage for those who don't have the impact case studies in REF terms but we're seeing in the last year I've unearthed 25 different case studies just by looking at that because they were not confident enough to raise their hand in a meeting like this and say it's something great to say but they were confident to be in their own office and input their data and because they're giving me verifiable information I can double check if that is the case or not and they just bring them on board and before they know it they're part of a bigger discussion one final question could you please wait for the microphone thank you you mentioned that there is a gap to the social and the researchers do not care much about the social but how about in recent in the last in the recent years with the societal challenges that we have climate change, cancer diseases hasn't that changed? I don't think that it has sufficiently changed I mean if we were to quote Daniel Kahnman the behavioral economist that talks about framing the way that the world is going I mean it really is surprising that we're not clubbing together every single day to solve the world's challenges so on a global perspective it hasn't really changed we have individuals who care alone and we have people who are quite happy to keep the status quo my job in this respect is to help them see the importance of this you will have pockets of activity but I don't think it's a priority in everybody's minds because we all have a job we all have to eat, we have families it's not in your so to speak in your work in memory to go and solve climate change because we're a pleasure maximizers so we have to maximize the pleasure of this in whichever way we can so it's important for them they told you come and have a beer afterwards or go and fill in your repository nevertheless you fill in this repository I'll give you a beer because that will get us a funding it may work just an idea so could the research infrastructures help facilitate this process immensely this is what we're trying to do it's not easy and I think it does I don't have time to talk about what I do but I have a tier system of engagement I was mentioning somebody we have the systems which is the material and I have met I think to date about 270 of my academics one to one for an hour to talk about the research to map out their progress and I also sit at the college and I talk to the heads of schools and I talk to the university systems and they change the culture so we're talking years of work which brings us back to one idea that was a volunteer earlier what about the mandate and a vision a vision is important and you have to get people on board and you don't win that with repositories you need a repository for when you convince them to use it so I think it's different levels of activity I'm happy to talk about it later okay again thank you Carlos now it's time for the final part of today's program panel discussion if I could please ask the remaining invited speakers to come up we'll have some rigging to do the moderator of today's panel discussion is Patrine Vajsten Bjerde director of research services at Ceres we have one microphone so I would please ask the panelists to wait until they receive that microphone before they speak thank you so I can speak anytime I want and you have to wait because this is supposed to be turned on it's working so welcome dear panelists I think the most brilliant thing Jens did with the program here was to put Salvatore after the lunch I think that had a great impact on our ability to stay awake unfortunately he's not here now but we will sum up some of what happened today and my first question to all of you I thought we would just start by a comment from all of the participants here please just prepare questions from the audience whenever you want we'll take this round so speak a few minutes each of you and my first question to all of you is what do you remember best from the day what did you find most interesting or most important I think I found the last presentation the most interesting one because the rest of us are building infrastructures we work with the researchers we think we know how they behave but we actually don't and this is something that it would be interesting especially from the open-air perspective to integrate this the social aspect of how they see the impact and engagement from early on so it's a social thing organically grown to the process we go afterwards but not before and that I think for me that was the most interesting point yeah I guess maybe I can choose a couple of things but as I said I thought Carlos about some of the maybe slightly depressed but some of the the difficulties in measuring the impact one of the things that got picked up I was looking on Twitter as well was Katie's point from the research that she'd done about the fact that APCs are there is more spent APCs are increasing that there's more and more resources being money going into those and the fact that they're increasing whereas I know that some of the argument in the UK was that market forces would push APCs down and it doesn't seem to be happening and then the fact that if I'm ever presenting preliminary data I'm going to do on scribbled presentations I think that worked very well but as an organisation that we started to do more work with preprints that was really interesting to see how that whole cycle has sort of come almost past full circle you could really contribute to that so what I found interesting and important today I think was that obviously it's important to measure OMAXIS the amount of OMAXIS how many publications actually OMAXIS but I think it's equally as important to measure the things around the publication like we just talked about the APC costs how do the APC price, the list price develop over time how can we document that and how the actual APC prices how do they develop over time the embargo periods how do they develop over time and also get more numbers on subscription costs I think it's very important to have those kind of data from all countries because OMAXIS is not just making publications openly available there's a big business surrounding OMAXIS and I think those kind of data would really work as good ammunition when engaging with the publishers in the aim of getting more OMAXIS so that's what I think from this point I think I will add to most of you have said that I think what was shown today both in Salvatore and also Antonio's presentation and what you finally showed here is that there are huge differences in the evidence of OMAXIS impact and also the way we measure different kinds of OMAXIS different national governments have different definitions and we need to be really open about what kind of definitions we use what kind of impact measurements we use for what purposes the data that are in scope the target objects and also the bibliometric indicators actually available otherwise it won't make sense to benchmark numbers and it's 40% better than 30% in some countries I mean what are the targets in scope that's really important I think much today it's very difficult to pick just the one aspect I think if I was to pick one is how some people have well some countries have gone all the way down to the mandate and some people to the gentle you may want to do this and I think that's one of the solutions to our problem if we actually move towards having a bare minimum that we can mandate as well as measure because we have the tools so I think I picked up a lot of information that I can start having crazy ideas about how to measure these things I would a lot of you have talked about I mean it seems it seems evident that we need to be able to measure open access in order to be able to measure the impact it has and we are here we are talking together we are trying to compare across different areas across different countries it seems to me that there we talking about gold or green or talking about the scope that's a limited area there's medicine which is a huge area is it actually possible to find one way of doing this or do we need to I have a feeling that we need to identify the specifics of why has scope succeeded because APC's within scope have not risen APC's within scope have gone down whereas APC's elsewhere have gone up is it because of the characteristics of the different areas is it because of whatever what I would like to say about scope is that scope has an international organization behind it CERN that no other community does so this is a full force by itself and if you look at Salvatore's diagram the nice diagram is that we are this is what I also wanted to say before is that we the market now we're building a market around APC's and what we were telling in the break is that in the market you have all the intermediaries but the researcher who is the market force behind it is missing we're hiding everything from from that and this is this is I think we're building a reality that we believe should be so libraries, funders they're making all these initiatives they're making all these efforts and the researchers just do not know the value of what they're paying and this is something that I'm not sure if it's going to be successful at the end or not because we're trying to do the market without all the forces and also your question on scope is CERN behind it anyone else want to comment on the different areas the different Mikael I would try to highlight some of the different publishing models or business models at the open access market because we have seen a really sustainable one in scope and some that are not sustainable when we look at some of the first calculation from the offsetting deals have been done we have seen in Germany that they have a price cap on the APC but it's not even close to the goal APC to the price cap then we have in the humanities some interesting stuff around library partnerships subsidies and we have an APC that's below three dollars so it depends on what the community wants and for example the open library of humanities I think it's a fantastic business model that the community should embrace and try and solve these things instead of looking at some of the commercial systems and we have seen a fluctuation of some high esteemed journals into that publication platform because it's attractive to publishers so initiatives like this we should definitely support throughout the whole community I think also libraries and funders I would say something related to that because I'm not a librarian it's been a passion of mine in the past but never properly like yourselves I think there is a relationship between having impact through academia and having impact through other means and if we were to recognize that there are different types of ways that a researcher can make a contribution then we can actually get them on board so perhaps you're in humanities and your publication horizon will be probably 10 books if you're lucky maybe 60 if you're phenomenal maybe a few articles if you're absolutely out of this world but that is what your field looks like so how are you going to contribute can you include exhibitions can you include working with the community can you include making presentations to policy makers we do but who's going to be interested in picking up some of our data for that end I doubt many people will and there are some confidentiality concerns as well because that's our unique selling point so we'll be able to talk about the high level stuff of who we're working with but we couldn't reveal specifics but I think we have to widen our definitions I know that Julia Lang of Julie Lane she's been working a lot on the impact of science and one of her early findings was that publishing people reading has a lot less impact than people talking together and working together they were tracking medical improvements across the US and seeing how that spread across the country with people moving and working together with people moving and working together yeah okay opening up for the public now questions for any of the panelists I want to throw in another criteria when we talk about impact just recently I encountered a study by E cancer saying that researchers from English speaking countries have triple impact because they have much more the ability to get selected for publication in these journals that's again another bias that we have here in journals and then I see all these exploding APCs in the way we are handling and I see yet again reasons why we have to invest more in an infrastructure supporting all researchers from English speaking countries to get to excellence and to be visible do you think that we need we are talking all the time about new measurements but which kind of measurements can we take to help these researchers from non-English speaking countries next to our repository approach and infrastructure approach I'd like to comment on that because I think what has really been lacking in the Chris and repository community for the last 10 years is actually the kind of development supporting file sharing and maybe some social media platforms where researchers easily from peer to peer can share content and you have seen that the commercial sector has exploited this this gap with the research gate academia other platforms popping up now so I think there's a definite lack in the way we build the repositories that we don't facilitate that kind of social engagement to call in the social factor that's one huge chance I think we should look more into in the future again it's not no the social engagement is is the primary thing is that we can do a language translation text and data mining a layer on top that connects actually people so they know what each one is doing so that could be built on top of the infrastructure of repositories that we have probably not on top of the publishers because the publishers do not have all this legal and technological barriers but repositories could certainly fit in this role there is a repository called Scotland's environment portal that I worked in through another project a couple of years ago that tried to open up the academic community to policymakers and basically I think we were using pure and enlightened and it was an API with a plugin to actually create the option it was called Ask academia so you have all the documents that the policymakers used that we knew they were using because we had them on board and their portal of government documents were built into Aberdeen universities enlightened and pure tools to widen the access it never worked because it got to the point where we didn't have the infrastructure to actually have a person who would be able to moderate that engagement and our knowledge of the authors is always out of date so if publisher A published in 2004 under this university who is going to publish who is going to answer that question and who is liable for that content so I think it's possible the tools are there which is need to find a very clever way of moderating that engagement but I think the will is there I can share all the code but if you want to see GitHub it works but there was no capacity you touched upon it in your speech that if you invited those people for a beer as a reward for actually answering those questions I mean all the time if you keep asking researchers to do more with no reward of course their days are full so how do we change the reward system to make sure that they want to make an impact I still haven't found the answer to that question properly I think I can share with you an interview I did for the German Psychological Association I think very broadly I class academics in two typologies we have all fashion academics and new wave academics the all fashion academics thing is publish or perish I think that outdated they're going to just die happy the new wave academics understand that they don't need to publish as much it's about how they publish something and this is what I say in the personal impact plan your publication is the beginning of your research it's not the end if we can get them to think that way and we get the systems to acknowledge that it's quality not quantity I think we could move not the world I mean I'm an idealist a friend says that I'm only four letters away from being an idiot which is very true but we can get somewhere and I think it's highly needed it's the last thing we need it's more papers that nobody reads or that they download to do I don't know what so yeah I think I think you need to keep it quite close to your mouth so I think to your point that people have no time I think the more that can be done to automate things aspects of that for researchers the better you know in the same way that you know if you're if you publish with an organization you get a DOI there's a landing page your content gets maintained there but having ways then for the for and there are different initiatives there are things like Kudos and other initiatives that mean that things like this are happening sort of more automatically for researchers and for that to kind of be picked up I know that I used to work with a social sciences journal and they said well there was a bit of pushback against it by researchers because they're researchers because it felt like they would be boasting or you know or sort of you know blowing their own trumpet for lack of a better word but I think as you were saying it's the start of the process and not the end anymore and I think that is still something that that people need help with because it's all about getting your work published and not the steps after that I think the other thing as well is I guess sort of smarter search tools like looking at the kind of things that the kind of initiatives that are popping up now it is often sort of overlays on top of things like archive or overlays you know search services that sit on top of the scholarly literature to say hey if you're you know if you're a medic if you're a GP here's the stuff that we think is going to be most relevant to you so trying to put that layer on top of the masses of publications that are out there so that people can go to and find the ones that are most relevant to them and you know you can see that with like Chan Zuckerberg that with that kind of investments going which is interesting absolutely it seems that you're all talking about that we do need the infrastructure then we need the culture change to actually use it for communicating the results that lie in the infrastructure so you've touched on some issues that we what are the additional if we have the perfect infrastructure what are the additional ideas the additional tools or changes that we need to make in order to actually have that content make an impact and please join in from the audience Andras I've just noticed that actually we might have a way to broaden or assessment of impact of scientific articles reading different news pieces from BBC news or whatever platform I find more and more that news pieces about science and technology and whatever we really do refer to the research paper while archaeologists just find new finds on Neanderthals in there and there and you can read more about it in this journal and there is a link it would be really good to tell journalists that they should actually use the lies and try to mine back that information what are the research articles which are really making it to the to the general news So that would be a way that you said actually to be accountable and to cite what you've used Please use the microphone. How does that change the system if it's not taking into research assessment because I can have a very high altmetrics but it would be nice if any of you tells me that there is a committee a tenure committee around the world that is using altmetrics as an indicator So it's an indicator that something that we think is nice but we don't use it So unless we change that Again, it might not necessarily get into the research assessment of individuals but it certainly should get into the assessment of institutions, universities whatever So if a certain university gets referred in the press more often than the others it should certainly matter I think because I've always wondered why we are libraries talking about altmetrics and not communication departments at universities For example, where I work they still do handheld lists of where institutes are cited in newspapers and in international media and they never kind of discovered what altmetrics is but the problem is also with altmetrics you can track any DOI link but if you don't have open DOI's at media institutes so newspapers you will never be able to track the attention of DOI So we have some issues around the openness of data and systems as well so that's one huge challenge I think and it doesn't seem that we have kind of really gotten into business with media yet It's an interesting view Sorry, just very quickly Natalia, twice in the last year we've used for two years Just ask you to keep the microphone up to your mouth so that others can hear as well Thanks, just twice, you asked for examples twice in the last year we've used altmetrics to balance out very strongly scientific or health sciences research proposals both of which where the altmetrics information was really welcomed by these huge scientific projects that are going for major funding because they're able to show the bibliometrics and all of the rest of the information that they and all the scholarly information they needed to balance that with the kind of things that Carlos was talking about there which was a societal impact and engagement and the idea of engaged research has become really big at the moment so I can give you examples of what those look like in confidence later on both of those projects were really successful they got funded Yeah, I would also lost this one question up there in the meantime and even if altmetrics and actually counting something based on altmetrics is difficult it could maybe help the researchers in picking those stories or help the institutions in picking the stories they want to highlight I was just wondering before we talk about altmetrics should we not talk about maybe patents as impact on society and also maybe a little more abstract but also look at collaborations with industry academic we have 39 institute or government agencies here in Norway which has that as a purpose and a little final note that I used to be a marketing director for five years online business if you give me a hundred dollars guess what I can do for you with your altmetrics I think you're hitting this on the head and I think it's for the benefit of this session we don't have time at least in the UK what do you need, you need a bold action from the top this is how this academic impact in mind happened I'm not suggesting everyone does that it's creeped up slowly in the European commission you have your academic and your impact side of things but these are included in our assessments we talk about collaborations with industry that's why I talked about activities and outputs I then want to talk about books or journals or whatever so I think if we could together resist the temptation to be too specific and have a wide enough framework to capture most facets of academic experience that at a later time can be mined for patents it would be better and it doesn't take much for that environments like these have the good ones for that and perhaps press associations have standards I know somebody who does that in California the stock exchange have one of the best interoperable databases so it wouldn't be difficult to dictate that they must comply with some of these or that we include theirs so perhaps for the next workshop they're representative that's a bad idea also including industry anyone else commenting on this Rachel did you want to to your comment about patents as well that's something that we've been we've been working with an organization called Cambia in Australia for a couple of years and about assigning DOIs to patents we've introduced assigning them to things like standards and that's so that we can look at which papers are citing specific patents which is again it's not maybe inferring value or impact but it's a thing that then we can be looked at and measured and assessed in their own rights but I agree that it's a really important thing to look at in the same way that we've got publications grouped around say specific clinical trials so yeah to that point I think there are other methods like that I guess the other thing is I'm quite a sort of visual I like examples so a while ago there's the publishers for development conference in Asbron and they were talking about so impact for some of the some of the researchers and librarians who spoke at there were in cases and I I'm sure people like Eiffel can talk about this much more authoritatively than I can but talking about impact in say in places in Africa where the results of research have actually gone into you know into how areas are farmed irrigation things like that so you know again I think they're sort of highlighting and speaking to examples like that is really important to you I know that Cameron Neil at some point was talking about these different examples of picking up stories from altmetrics in saying that yes this this woman in this kindergarten read this article and that changed the way they were treating the children in that so it's a small example of impact but it's still impact you can't measure so that combination of quantitatively measuring and quantitatively assessing it's I mean like the UK ref it's a combination now isn't it of counting the number of articles but you need to add a piece of text describing the impact that it has done some sort of combination moving to stakeholders we were touching on high level bold politicians we've touched on institutional leaders we have the researchers we have the librarians we have industry who needs to move in order to make something happen I would love to see iColk much more active within this field I know there's a Berlin 13 conference coming up next month and I would love to see the some of the big library consortium moving in this area and yeah have a stronger mandate and focus on some of the needs of the community and I don't know how that should be solved and negotiated I'm not a vice negotiator but I think they are crucial to to disrupting this this field there are some of them in the room so any comments are the stakeholders that should could make a difference money talks what could they do Natalia the microphone the programs evaluation so because they are for example the commission or other funders now they are thinking of FP9 the next project how will they where will the money go it has to be a joint decision not decision but a general evidence from the research from the society from the economy so how can they how can we help them to close the loop around it and so to make the proper decisions and then on the project level you could have something like what Nif said is that all metrics in which means it doesn't show impact it shows engagement so engagement really helps so how could you do that so these are the small things that we can move forward but again we have to be really careful in the world because the old metrics doesn't show positive or negative impact it shows penetration engagement so it's a I just think that's a really important comment that Natalia has made the framework nine we've already seen what that looks like and it looks linked in very strongly with the UN sustainable development goals and there's a conference going on at the moment you can see it's trending on Twitter now on the SDGs we've heard the SDGs are going to change the way that research is done it's going to be about interdisciplinarity this is a massive opportunity for open access where we break down all of those barriers but I think that we need to find those kind of tools and we need to engage with policy makers and people who are involved with the SDGs whether they are the international development researchers are there policy makers and governments to recognize that impact doesn't happen at the end of a project it happens at the very beginning at the proposal stage when that starts off and the old metrics as you say this idea of engaged research what I'm looking for right now is simply to be able to use the publications in my open access repository and to be able to very quickly and easily pull out whatever information is there for a group of papers that shows the numbers of universities that are using this and from what areas the numbers of governmental bodies using the Gov.ie local domains limited as it is I just want to see that I want to see the schools in my country who are using this information and that is something that I think is available to us here and now but the tool simply isn't at my fingertips to be able to use it right now I have to add to this that we are proud that in Open Air Connect which is a project that just started in January we have a local SDG community we are helping them build the national SDG open data with open research data with open publications moving to Mediterranean if possible so this is the way that infrastructures can help because we have the knowledge they have they know about the data they just don't know how to connect everything together I just wanted to add something very quickly to that and that is that University College Dublin welcomed and Jeffrey Sachs who wrote the Sustainable Development Goals and is working with the United Nations on this and at the end of the speech welcoming him and honouring him with the James Joyce Ulysses award for services to humanity he a professor Paul Walsh professor of economics and international development said that open access was key to the attainment of the Sustainable Development Goals and I think he said that publicly it's on record with Jeffrey Sachs so that's something that we could really develop on more examples on places, institutions stakeholders that have made a difference I think we're something that's starting to move where I definitely get a feeling from this day that we are moving away from just saying let's count publications and let's count downloads to saying we actually need some sort of a culture change and we need to work on assessment and we need to keep working together yeah, absolutely I don't know whether we need to continue until this is at the end I don't think people would mind going home a bit early Jens, you want some final words? Thank you for your contributions That was difficult Thank you for a very interesting discussion It's been a long day and about to end almost but first we have some information for the next open-air workshop and it's organizer Pedro Pinsipa So we will have our next open-air workshop in a different city maybe warmer in Barcelona so and it will be in spring so in April in the 4th of April so it's a co-associated RDA Plenary event so we will do it associated to the RDA Plenary as a pre-event it's a co-associated because we will do it not in the same venue of the RDA Plenary but in the different venue we will do it in the alma magna of the University of Barcelona we have our colleagues of the University of Barcelona Austin this workshop from Spain is there is also supporting to welcome you in Barcelona to this open-air the 8th open-air workshop not the last one of open-air 2020 we will have another one and we will have also a fair conference in September so join this workshop and all the content from all the previous open-air workshop not available in this page ok this one all the content are available via our portal also recordings and slides and things like that as soon as possible recordings and slides will be available as soon as possible I would like to thank our speakers and panelists once again and I think that's it for today just so congratulations to the University of Barcelona University of Barcelona if you want to be