 Hello, my so my name is Pierre Mounier. I'm associate director of open edition and I have the honor to Chair and moderate this panel of honor open peer review In fact, it's the first time I'm playing this game So I'm a little bit stressed and I will ask you to to be nice with me Forgive my my mistakes. He saw if I make some mistakes So as Anthony said four speakers were planned, but unfortunately Unfortunately, Andrew Preston couldn't make it due to transportation problems But our three other speakers are there Ready to discuss the hot topic, which is that is open peer review so I suggest that we start and we are going to start with Short presentations from each of the speaker about what they do and what in their opinion is the main challenges That we are facing with open peer review and then I'll tell you after the presentations How we are going to organize the discussion. So we are going to start with Stephanie and during Your walk to the computer. I will present you Stephanie. Okay. It's a show. It's like a Ted conference with a headset. So But we are seated. That's a difference. So Stephanie is the medical editor at Biomed Central after graduating she worked in hospital clinic practice before joining Biomed Central in 2010 and Stephanie is also and I think it's the most important thing here Co-editor in chief of research integrity and peer review at Biomed Central. So Stephanie the floor is yours. Thank you So I'm going to be talking about and what I see as the challenges of open peer review just to give you a Little bit of background first as to what I do and how I'm involved in open peer review So I work in a small group at Biomed Central called the research integrity group And I'm the medical editor which means that I work across our medical journals that we publish And so we define our editorial policies and make sure that our editorial policies are current and up-to-date and And we also work with our editors to help them apply those policies We look at our peer review policies and we help them and with any difficulties They may have with a particular manuscript relating to publication ethics or maybe any peer review Issues that may arise and we also provide training for our editors and so I work and Across a lot of journals that have different that publish on different models of peer review And many of our journals operate on a single-blind model But we have a fairly large amount of journals that operate on an open peer review model and particularly our medical journals And so one journal that I'm particularly involved in is a very new journal Which we launched last month called research integrity and peer review And I'm one of the editors in chief of this journal and we aim to publish and research Into research integrity that the publication process and to peer into peer review to try and build up the evidence base And that this is around here and obviously we operate an open peer review model on this journal And I've also been involved in conducting some research Into the peer review process and looking at the quality of reports on open peer review and closed peer review And so I thought as we've been using open peer review to cover so many different Meanings and topics here I thought I would just explain that when I'm talking about open peer review in terms of the open peer review that I'm Experienced in working in I'm talking about pre-review open peer review that involves two levels of openness So the authors know who the reviewers are who is reviewing their papers And then if that article is accepted for publication the reviewers signed reports along with the authors Responses to those reports will be published online and available to readers So I've just picked an example of an article here that's been published with open peer review And you can see that there's a link to the open peer review reports Which you can click on and read the reports from that so The jet we have a series of journals called the BMC BMC series which spans medicine and biology it's approximately 60 journals in it and Approximately 40 of those journals are medical titles and all of the medical titles have operated on an open peer review model Following this system that I've just described and since they were launched So some of them have actually been operating on this model 15 years so we feel it's quite an established model because we've been able to To run this open system of those so when I was asked about the challenges of open peer review I thought oh open peer review works it works on the journals I work on and So what are the challenges? But then I thought that actually in this room We're all very in agreement that open peer review is a good thing for its benefits on Transparency reducing bias accountability, but actually when you go out into the broader community and speak to researchers A lot of them are very anti open peer review They like to hide behind the anonymity and they think that no one will agree to review a paper if If it's if the reports are being signed that the reviewers won't be able to be honest They'll never be able to provide a negative report. You won't be able to reject Manuscripts and actually when large-scale surveys are done of researchers The model that comes out as their preferred model is actually double blind peer review Which is obviously quite different to the open peer review model So I'm hoping that we can kind of discuss ways to change To change attitudes and convince people of the benefits of open peer review And then I think underpinning some of these problems is the other challenge is the lack of evidence There's a lack of evidence into open peer review But I think actually it's fair to say there's a lack of evidence into peer review itself as to whether it works What models work best and whether as I think is more likely to be the case different models are needed for different Specialties different communities and what really works best. So thank you and I will pass over to the next panelist So now I give the floor to Alexander Grossman Alexander is the founder and president of science open Until 2001 he was researcher and associate professor and physics at the University of Tobingen before he decided to join publishing industry at Wiley and Before he got an appointment as professor of publishing management in 2013 Alexander spent five years at Springer and the greater as managing director and vice president publishing Respectively, is it correct? Okay, so Alexander. Thank you. Yeah. Thank you the amazing Thing is that when I was at Springer that was 2008 Just the year when by much central it was acquired by Springer and that was a great time because it was the first time that Major publisher was so open. Yeah to cope with open access and also Yeah, the CEO of Springer sat at that moment at the management meeting in Amsterdam Well, that's a great opportunity for all of us because he was asked at that time. Oh, why? Why shall we go to open access? Why should we open us to new kinds of publishing? and yeah, so it's a yeah was a great experience for myself and That was also one of the reasons why I decided then to think about alternatives for both for open access publishing but also for peer review and Just half an hour ago. So Josh explained very accurately how it could work post publication peer review More transparency in any kind of scholarly communication and this was or is part of science open Which I launched with a partner of Boston in Boston three years ago and yeah And we are we are ready to find out more about the opportunities Not only for publishing and scholarly communication, but also for more transparency and the quality assessment of Scholarly communication and this is the topic of our panel discussion today That's why I want to ask two questions. Basically The first question is how to establish more openness and transparency and peer review in general Without and that's very important and we were carefully listening to a rich pressure this morning without affecting the quality of that process of scholarly communication and We we all know that there are a lot of differences between the communities between humanity social sciences and Life sciences and medicine for example in terms of peer reviewing Yeah, so there are different ways to assess the quality of manuscripts And there are different ways how to publish manuscripts or monographs in different areas and in the future I think the borders Yeah, will be not as as clear as in the past. There's more into disciplinary Research and as we learned this morning, there are more ways to communicate not only blocks or other ways to post opinions or yeah, the latest research results and there are some Communities where this has been done quite frequently frequently others Yeah, which want to keep or want to stay at the past or at the the present system of peer review the second question is Do we need or do we still need an editorial Moderated assignment of reviews for open peer review Because we want to maintain a certain level of quality for the peer review and I know there's Great debate or big debate about this this issue and there are also some service and statistics about opinions of Scholars about that point and hopefully we will discuss that issue later on them Thank you. No, we are going to listen to Pondellis Perakakis Pondellis received a PhD degree in clinical psychophysiology in 2009 and now he works as a postdoctoral researcher at the University of Granada in Spain In 2012 together with Michael Taylor and Varvara Trikana he co-founded OpenSchooler Which he currently serves as director Is it correct? Okay, Pondellis. So yes, it's true I'm a postdoctoral researcher at the University of Granada in the area of sports physiology and cognitive neuroscience and apart from that I'm also co-founder of this organization this international organization of volunteer researchers that was born when group of friends of colleagues that got together driven by the same frustration that Ulrich mentioned earlier and tried to discuss about ways to improve scholarly communication model that we would we perceiving that was confusing Quality with prestige and it was give offering the wrong incentives to researchers to pursue To compete against the prestigious publications in journals instead of collaborating to improve the quality of Science and of their work So that's how we started we realized that The main the main driver behind this this model is peer review So in some way or another all our initiatives and oral efforts were dedicated to providing innovation in peer review and mainly to develop infrastructure that will Put forward that will support a radically new a radically different model of peer review We started by trying to build our own platform that we called Libre We had many problems and while we were trying to overcome these problems We realized that somebody else that did something exactly the same as we were trying to do with Libre only better because he didn't He didn't have the same problems that we were facing So it was exactly the same in terms of both infrastructure and also principles so we decided instead of duplicating efforts that we get together and Enjoying forces and promote this this initiative together. So this is the search journal of science and Which is actually a platform. It's a non-commercial repository that Provides the space where researchers can meet and make peer review a community process without that. It's not mediated by any Any third parties they don't need any editors or committees of experts They just get together and debate about the validity of their own work and by doing that at the same time They they improve the quality of the work and they receive proper proper credit for it that they cannot to their academic record The same idea we tried to to propose also to institutional repositories We received the funding funding from open air last year and we created a module for this space Repositories for this space infrastructure so that the repositories themselves become this meeting space meeting place of researchers in order to Validate and assay assess each other's work and at the same time We participate in a recently formed working group by the Confederation of open access repositories where we We propose recommendations for the next generation of repositories in order also to become every repository can become this this this platform and evaluation platform and I can I can assure Josh that the sex appeal is one of our one of our points in the agenda of what needs to be done so The challenges the way I see see them for the future First and foremost to serve quality. There is a there is a stakeholder that's always missing from these groups Which is science itself? We always discuss about what's better for researchers What's better for editors for for reviewers for publishers? But we tend to forget what's best for science and sometimes there is a conflict between what's best? What's what is a best scientific methodology when it's best for knowledge and science and and what is best for individual groups? so we have to keep this always in mind that the final that the final aim is to improve the scientific quality and Also about the open ends hope openness how to define openness and we we we tend to promote openness at all levels that Tony described this moment this morning. So it's that should be Public meaning that open access reviews that the text should be public transparent always signed by everybody everybody knows who's every every comment and every action on This on this space is always signed and it's community It's not it's not a selective process anyone can contribute anyone from their search community can contribute to the review process And also dynamic something that was not mentioned according to the principle of rational of critical rationalism science can never be Scientific proposed statement can never be finally Accepted it's always under under debate and this is this is what the nature of peer review should always should also be like and Finally the infrastructure in this infrastructure, whatever Whatever is the infrastructure for these review services? It should be public. It should be federated it should be it should be working with open source software and We should make sure that there is interoperability so that we don't duplicate the other today's situation where what matters most is where you publish but It should be relevant which portal you you are using to to review the work and Also very important the governance issue, which is we should find ways So that nobody controls this new infrastructure and the best way not to control it is so that is that everybody controls it So we have to find ways that this new infrastructure is community governed. It's Decisions are made horizontally. There's no Oligarchy, there's no there are no decisions that are made by select groups and That it should be a non-profit So these are the challenges the way I see them and hopefully we'll discuss about them in the debate okay now so The four of us and with Tony and Arvid who is just sitting here We prepared a little bit the discussion by email and we identified six topics Through which we are going to go for the discussion the first one is obviously very important is the question of definition and delimitation of the open peer review topic I Don't want us to take too much time on that because it can be infinite So we will be short on that but it's necessary The second one will be the second topic will be about the relationship with research integrity and the question of quality Of research the third one will be the relationships between open peer review and the rest of the scholarly communication system and particularly particularly the publishers And after that we're going to talk about the challenge of the uptake inside the different scientific committees communities We will have a short part of the discussion about The recognition of open peer review inside the research evaluation process How open peer review practices are recognized or not recognized for the careers of the researchers and maybe the instant institutions as well and then Irina, I don't know where you are but be confident We are going to talk about the place of open peer review inside the open science Framework because I think also that's important So no, let's get back to the question of delimitation and definition. So my first question to the panelists would be How do you how would you define open peer review but more precisely? I think that there is a difference between commentary and Open commentary and open peer review. So what are the relationships between the two of them? So for you under what conditions in your opinion an open commenting framework framework can be considered as open peer review And on the other side a peer review process What are the conditions for a peer reviewing process? To be considered as truly open. So my question to the three of you who wants to start Pandelis? You have the mic Second one The question is The difference between review and commentary Okay, so starting by that from the definitions we I already mentioned that for us openness is at all levels for review there's no there's no restrictions in the in time in the group of The selection of reviewers or anybody or the community can participate in this process What what what is the incentive for the reviewer is that his contribution? Becomes integrated to the article so whoever reads the article at the same time He's able to access all the process that has all the process that has contributed to the final version of the article So this is the he links actually he links the name with the quality of the article He becomes a part of the article. So this is what differentiates a review process where the reviewer becomes a member of the of the of the team that has produced the article and commentary which It could be something more light a lighter version where the comment is not necessarily a Significant contribution that requires some revisions from the authors or requires it could be something more minor Let's say and it could be done without the intention to become part of the of the life of the article This is what we perceive as review versus commentary So for you, it's based basically on the level of engagement in fact of the part about Alexander do you agree maybe just a quick remark when I recall my time at the publishing house as well We had so many journals also Premier League or champion League journals Some of the reviews we received Were comments Condom teres following your definition, which is absolutely accurate just to recall that also the closed Classical peer reviewing is not always that what we assume What it is? Yes, sir I think one of the the key things that I will take home from today is quite how many different things we talk about when we're Talking about open peer review I would say to me. I I see open peer review as reviewer reports done with names attached to them so people know who those reports are and and that those reviewers are suitably Qualified to between them to assess the all aspects of the manuscript so that a decision Can be made and that article is is deemed sound whether that's done in a pre-publication model or a post-publication Model, but I think maybe we need to be clearer when we talk about open peer review what we're actually Meaning by it because I can see that and that the commentary after it Which which is very important and continues to evaluate the article is also open peer review. So Maybe Ulrich could you Answer also the question because I was really interested by your multi-stage open peer review process And maybe you can give your insight about how you see the relationships between the two of them Well, I found that very I never thought about this specific question. I have to admit But now I found it interesting also really I think this discrimination between peer review Contributing to the article which I stressed was important for us that you contribute to the final product directly Might be a good criterion again. There are many forms of review So I think it's still review the commenting afterwards even the citation is a kind of evaluation right to go up But I think if you want to discriminate between peer review and and commentary I would would agree to the proposal of Contributing to the final product or so and that might also really help to to attract people Whether it's are not so it's sort of opposite whether it's anonymous or not Right, I would say if you contribute to the final paper That's maybe a good way of talking about peer review, but again, this is just loud thinking Maybe just a final statement on openness. We just described Open peer review based on the review the physical review Which means disclosure of both identity of the reviewer and also the report But there's a second dimension. This is my opinion or not only my opinion Also other authors has postulated this as a Kriegeskater and others a few years ago Which say okay the second dimension of open peer review means openness in terms of who Maybe the right person or who fears that he or she is the right person To submit that review. So that means openness Second dimension everybody every expert to be very precise is invited to contribute Which is a complete contradiction to the classical peer review where peer review peer reviewers are Assigned by editors or in-house editors It's only reflecting what the first dimension which we were discussing and I think some authors Not the majority, but some authors also say there are two dimensions Which means really openness then so no bias because everybody who feels as an expert in a certain area Yeah may contribute and not the editor or the editors are deciding who will be or who will become the editor Absolutely, I think that's a key difference between a comment and a former review. You're absolutely right We see the review has to end up with a recommendation It always a reviewer has to end up with a recommendation of whether that he accepts all the statements in this In this piece of work or he believes that there should be some revisions major or minor in order to be accepted by him and If he decides that there should be some revisions then he should Clearly debate why he believes that this is pertinent to the article So this is another differentiation probably between reviews and comments. You can make a comment without any Recommendation Level of engagement and the presence of a recommendation or not the two criteria So let's stop there because We don't want to go too fast for Let's stop there and maybe switch to the second thematic So the second topic is about research integrity and quality. So we have The idea is that the evaluation of research publication is based the classical evaluation of research publication is based on the assumption that Anonymity during the evolution process Guarantees the evaluator's neutrality and the independence of judgments So my question is how exactly is open peer review challenged by that assumption or how open peer review challenge this assumption, maybe and I Alexander I take back your own question, which is How to establish more openness and transparency in peer reviewing without affecting the quality of scholar Communication and what are the means of course needed to achieve it? Do we need an editorial Moderated assignments of reviewers. So Alexander First and then That was my question. Yes, but I have no answer. I'll try to give an answer And as I said, there are good reasons to have an anonymous peer review a formal peer review and Maybe it's interesting to stop the discussion with a survey which was done by the STM scientific technical medical organization a couple of years ago. They were asking thousands of researchers in different discipline in that area in these areas and Yeah, they were asking the question Would you Do any reviewing for any journal if you know in advance that your report and your identity will be disclosed afterwards and 50% or 51% said no way then I won't go for any review at that journal anymore however, I think 48% said yes They would go for a review in a journal where the identity of the reviewer and also the report will be disclosed Afterwards and then it was interesting then the survey asked those who said yes the second question. Why? And that was very interesting. That was an open question so not to tick something and Yeah, the answer was very clear the majority of those who said yes, we would go for an open peer review They said that it that they expect at least more quality more quality of the reviews itself and Yeah, that was interesting a lot of more reasons also Credits for the review and so on but the quality issue. That's the issue of my question also the quality issues I think the most important issue for for quality assessment and scholarly communication and If you look deeper into that issue, you see that Of course, if you are a reviewer You will take more care very probably if you know that your name will be posted or published alongside with the final article That's quite obvious With the exception of those who do an excellent job anyway, but as I said I made the experience for so many journals in so many years That's the minority of reviews which came back to us to the editor offices Which were brilliant very often it was a situation and close peer review process that you got a report back which said rejection No way the the next report said oh only minor correction needed Accepted with minor corrections What are you doing? And then you had a report like this formally Everything was treated but in that way so half half a page and Then we asked the third reviewer which was in between so I Think that's that's a problem and that's why I believe that open peer review Open peer review in the first dimension which we were discussing in the beginning Will improve the system will improve the quality. I just wanted to add to that point say I I agree that Openness is definitely associated with with better quality people taking more care over their reports We've actually got some evidence that shows that the study and that Myself and my colleagues and did last year where we compared and the quality of reviewer reports and on Journal that published and on an open peer review model at BMC infectious diseases And then BMC microbiology, which is a very similar journal in the same series similar threshold But has closed peer review We found that the article that the reviewer reports when we rated them using a tool called the reviewer quality Index, which is a validated tool for assessing reviewer reports We found that they were slightly better quality on open peer review And as you would expect when you looked at the domains on which questions they were scoring better on it was being more polite substantiating what they were saying so I think this nicely backs up that That though even if they're so even if they're making the same bottom line recommendation that there's more there to really show that That's a true assessment of that paper and and so the other other thing I just wanted to mention relating to and that there's a Quality of peer review and sort of the integrity of peer review is that I think it's really interesting that when really large-scale Surveys are done of researchers and asking them which models of peer review They like best that double blind always comes out as the the favored Model there was a survey done. I think in 2009 by sense about science where that they double blind was was favored and much more Recently there's a Taylor and Francis white paper Which again came out as double blind being the much favored one And I think that's because on the surface if you don't kind of look into the process deeply Double blind appears to assess a lot of the problems that we're trying to address with open peer review So you remove the you remove the bias and you make it a more objective review But I think unfortunately the problem is It's not possible to fully blind They can always be be guessed who it is. So it turns it back to the single blind model. So I think that's why actually Open peer review is that the answer to addressing those integrity issues. I would say that I think it's important to understand why? the blind the anonymous review process has become dominant in the in the field and The main reason I would say for that is that it makes the review process a question of favors of doing a favor to the editor and This this system of favors of this peer review actually today is exactly what allows and also enforces the journal monopoly We supposedly major is is doing the best peer review that exists But if it was if it was if it was signed we would be able to question that and this is something very important that Cannot be jeopardized. So we cannot we could not have Signed reviews, but then because then we would be able to compare the reviews of different journals and question The fact that they are performing the best review and that's why they deserve the prestige and the impact factors So this is a very Significant reason why we have this model and if we don't want that if we want to to change to a different system Then we need this is another reason why we need to make reviews open Just if we looked for instance to the social science and humanities, I think that's a completely different situation I think it's not possible to make a double blind peer review because a researcher usually on Yeah before publishing a monograph or so after one or two years. So everybody knows In which field or on which which topic he's working I think that's a complete variance of the life sciences and medical sciences where you could do successfully on a double blind peer review And also blind peer review is difficult because the expert who is then yeah Diving into that monograph or a manuscript is also I think could be easily identified from From from the author. So that happens by the way also in the natural sciences Sometimes so it's not really blind peer review anyway I think that the blind peer review enables is that the closed academic circles can decide what is accepted and what is rejected without being Again, this is not open to questioning from the rest of the community and criticism, which is another positive for For blind peer review. Okay, so we have a consensus here Around the relationship between transparency and the quality of reviewing. Do we have it in in the room? Does it any everyone is okay? I wonder why everyone thought Ten years ago, maybe or still again that anonymity was related to to quality. So it's like a magic Yes, I would argue that if you want to scale it up one should really Consider allowing anonymity. So again, that's our experience. We allow it, right? And indeed it doesn't go down That the proportion of people who choose anonymity and I think there are many good reasons So I think and in an ideal world both double blind peer review and fully open peer review should work Beautifully right in an ideal world everything would work beautifully But but in a real world, I think one should find the least common denominator Which you did right and I because I think everybody talking about open peer review essentially Now goes for making the review public together with the accepted article more or less There are also other cases, right? but maybe one should focus on jointly implementing this least common denominator and Then try to go for making everybody also signing the comments rather than enforcing both at once, okay So yeah, Jean-Claude and it will be the last remark on this subject Yeah, I think it goes back to a remark I made earlier and I think that intersects the distinction between commentary and review And we show at the stage when you want to know whether a submitted paper or a piece of work Should enter the territory of science It may be useful there for people to say yes or no and at the same time perhaps not sign directly What they what they do although it would be I think very good to have the list of all those that were involved at that stage If you have the level of the contribution or the commentary it seems to me that it falls under real Common sense that it should be signed. It's a contribution. So it should be part of the The work of the ground conversation as I like to call it of an article starting to enter the fray of What the scientific and scholarly debate is all about and in that regard? I don't think the distinction it in social science and the STM domains is that very great I think the same kind of debate discussion Amundation small corrections and that kind of stuff which is must go on Is important Thank you Jean-Claude. Okay. So Let's move on now to a more political or directly political question Because for the moment most of the reviewing process is under the control of publishers mostly private or commercial publishers in fact It's the most important part of their added value and power and the communication system They because they organize the evaluation It doesn't mean that they do the evaluation because scholars do the evaluation But they organize and it's the root of their power on the scholarly communication system their ability to organize the reviewing So now the challenge is how and why To descend descend on the goal peer review from journals or publishers or private platforms and Try to organize it around the next generation of institutional open access repositories With maybe overlay services that was your question pandelis So the question is what kind of infrastructure we need in this perspective And how we can make it sustainable and how a repository based author and reviewer We wrote system could work pandelis you have to answer your own question You Shouldn't say that it's our question So, okay If we think about what kind of infrastructure we need I would say that what the researchers need what we need is just the space to meet and And that should that should that should be all We should be able to meet at a common space and debate about each other's works And this is what we do at conferences. This is what we do when we actually meet in in person and Now I mean we have the we have the the possibility of meeting Cyberspace and it's easy and it's and it's it's free. I mean space in On the internet is comes almost for free. So this is the basic infrastructure and Of course around this this space if this space is provided then we can have we can have When we say that the preview is being organized It doesn't mean that it will be there will be no organization whatsoever. There will be chaos, of course Academic societies exist and they will continue to exist People are being trained in specific research groups. These research groups have networks have connections They meet every year in their conferences. So these people can when they meet in this space they can array arrange to Put their chairs together and start discussing with each other the important thing is not to build walls around them Which is what we are doing now with the current infrastructure being not on the same place and being Controlled by specific people who decide who comes in and who comes not so what is important with this space is of course This societies and these groups will continue to do their work And but they will have also some free seats for people from the audience who would like from other disciplines from other rooms That will be able that we want to come and sit on the same table And so How this could work? We already have this space. I mean, that's what the repositories have been doing for four years now They provide this space, but they don't know how to take advantage of the space they don't know how to to to show the researchers that this could be the space where they could meet and there's a problem of course the of Diversification that these repositories are owned by institutions. They're governed by institutions and there is no actually There's no common place where they where they can where they can meet and this big but this can be solved with With standard interoperability protocols that make irrelevant where the meeting is taking place Whether I put my research and my research is being commented on my repository It would not be it would be immediately Harvested by a global platform where everything would be what the discussion could could be discovered by by other researchers. So what we're talking about is We can use the metaphor of having a cloud having my my music on the cloud and then I'm using various devices to get access to the same The same the same space the same server where everything is stored and I can and everything is synchronized So I may perform some activity on my institutional repository, but at the same time all the all my activities signed and and transferred to the cloud so Repositories remain retain their individuality retain their their infrastructure and their as a portal but they function as a portal in a common Infrastructure that is as I said we have to make sure also Issues of governance of this infrastructure when you create this space. Actually, there's no there's no need to pay anyone There's no money involved. So all this can be all this can be free all this when you don't Establish the societies when you do not Try to organize this peer review or say that you're organizing this preview while in in fact Just they're already academic societies who do that for you Then there's no need to get any to get any any profit from the process So this is the big strength of designing this kind of infrastructure because you don't you're not asking for money from anyone So Stephanie do we need to disentangle peer reviewing from the publishers? And so in terms of kind of separating peer review from journals I think they have they have already been some initiatives looking at doing that so period of science and Axios review Conducts the peer review separately to to the journal. So I don't I don't think peer review has to be Necessarily integral to the journal I do think it's important though that we continue to make sure that the peer review is meeting the quality is meeting Equality that we want and I mean my my area is Working in our in our medical journals, and I think it's particularly important with medical research where an article is Going to go on to inform clinical decisions that that will be made on patients that that report that that Manuscript has been reviewed by peer reviewers who are able to assess the methodology able to assess the statistics and that we've got those full areas covered And I think however we do that we need to make sure that we are covering those bases and I think at the moment that that is a role that an editor can Can play well in that as an editor I will make sure that I've Got peer reviewers who cover all the important aspects of the manuscript between them and to make a decision On those before before that article and is is published and I think the other The other issue is and how to incentivize and people to provide These reviews these these reports we we find that that if manuscripts are particularly interesting have a really exciting result It's really easy to get peer reviewers to agree to review that article But there's a huge amount of the sound science Important valuable research that's going on that actually isn't as exciting and to review And I think we need to make sure we had systems in place to ensure that that was also Going to receive adequate peer review So Alexander you worked at many different publishers place So I'm very curious about your opinion on that and I have a specific question then from the previous discussion if Stephanie you agreed that it's not compulsory that the publisher Organize and have the control over the peer reviewing process and get be done elsewhere you you said But if the publisher goes into this direction, what is is had that value them? Yeah, I can give a clear answer There's no need for any publisher or for a journal in principle if you look very yeah carefully into the history how journals Developed and yeah, how peer review the peer review workflow. It's an workflow technically spoken was developed It made perfect sense in the age of printed journals bonded issues Distribution by surface mail or AML if you're lucky, but we have the digital age since two to two decades and That's why we could consider a completely different workflow and this is usually done if you're working Yeah in a big company or a smaller company company in industry You should at least from time to time look very carefully to your internal workflow Of course, we have done this and we are doing this at the university several times also with my students and bachelor Theses and master thesis and yeah, we found out that's in principle. There's no need to to do that Journal based or print don't print a journal based workflow anymore And there's no added value you mentioned in the beginning of that question the added value is managing the peer review process and providing a technical interface For for doing all that stuff since the reviews are done by us by the scholars Anyway, that's what you said and if you know that areas or Thompson Reuters those two vendors these two vendors are Yeah, offering that system that technical system to the publishers It's not the publisher who is supporting Scientists doing this technically it's areas. Have you known areas or yes? And so you could also set up your your repository or use a repository or use your local Repository at your institution as Thompson Reuters for license, which is not so costly for a single journal or for a single Whatever you want to call it collection of papers and then you could start doing everything yourself And so we we must agree that there is no longer any added value at this Specific point from publishing industry and also copy editing and language editing. It's done by by mostly by Indian Wenders, it's not done in the publishing houses anymore for most of the journals I have in mind now and Having said this I like that idea simply look at Repositories which still exist as for example the archive with more than one million papers In physics mathematics statistics computational sciences for example and use an overlay structure That's how it was called by Timothy Govers a field medalist Which said two years and three years ago in a blog post? Which was heavily a frequently discussed afterwards Just let's use this overlay principle Use a structure of communication of quality assessment of discourse That was the expression not peer-review discourse to do that what we have said in the beginning of that panel discussion Adding value adding value to a manuscript Supporting the author to improve the manuscript and this could we this could be easily done But those means That's the situation. Okay any comment. Yeah, I Think these ideas are I work myself eight years at Thomson Reuters before so so I think the things what you're all saying is very well But as I'm still working for one science, which is a commercial open access I just want to hear a little bit more about incentives Financial models, I love the philosophy everything is for free and it's easy to make I can tell you to build scholar one or areas take some consistent money to upgrade it so my question to you is Keep told tell me a bit more about the incentives because my con not my concern, but my thoughts were When a certain article and a reviewer is named and he has to really start to review and it's open Will the cost not rise for that incentive? So I would just like to have a little bit more on the financial side the philosophy great Technical there's no issues there. Let's talk a bit about the cost And the second thought is what I'm missing, but you mentioned it is society publishers Because on all fairness you can't say a publisher. There are society publishers like American Heart Association Will they be happy when they lose five million dollars revenue per year? So that are the two little remarks I wanted to make The last question maybe is not completely linked with the previous one I think but I agree with the your question about the financial cost of Taking charge in fact the peer reviewing and the editing process so To the panelists, maybe Yeah Okay, the financial question So I think the people in this group. We are enough to support this infrastructure by ourselves I mean paying like 10 euros every month and this would be enough Why because one person has been able to create a platform that already exists and it's as JS You can already join and start doing this process. The process doesn't have any costs There is no team that will maintain the articles that were organized review There's no there are no experts involved Nobody will look for reviewers authors can do that themselves and they're doing it actually it's Yeah, so that's are you satisfied with the same financial The answer to the financial problem. That's an important question. I'm still missing the incentive there. Okay, so financially we're fine Since from the business perspective I completely understand your question So to be fair of course, they are costs also if you want to create an environment And we are not charity publishing or doing this stuff You need money from a funding or from the customer and Yeah, or from the you or from the society which would be a part of the answer to the last question But I think it's fair to say the added value could be Yeah, a selective model where the author can select some service options from yeah Greater or bigger what a variety of services options like Copy-editing language editing consider the fact that maybe in the next 10 years 40% of all publications will come in STM at least from China mainland and They are ready to pay since they have the money to pay extra money for language editing and copy editing to improve their Article their language of their articles Of course, they are not ready to pay some thousands bucks for that But it's fair to say if you know the pricing the priceings of the Indian vendors you could say, okay It's 10 bucks per printed page. I think that's 50% more than the publishers are spending at the moment per page So that's a fair model you click yes I want to pay 100 euros for 10 pager for improvement of quality or not and so on and then you will create a sustainable model and I think most Initiatives suffer from that fact so far that there is no sustainability That's why I agree completely with your question And that's if we are looking in the future into the future We have to consider sustainability for all initiatives for all ideas Otherwise otherwise they will sink to the ground as a bad paper. Okay, so Really very short just a quick comment to come back to it So I don't think there is any indication To see that neither open access nor open peer review would raise costs on the contrary There's a lot of reason to think that it should lower costs and enhance efficiency and nevertheless Even though I have absolutely no vested interest in publishing never made a single euro out of it And I'm proud of that actually so but still I think there is a place for for Professional and also commercial publishers when they act as service providers, right? Which they do partly the traditional publishers are partly service providers But mostly they deny services that we want namely open access And if that changes there is space for everyone and also for others So I think it's important one big institution is good for the like the streets and whatever right or the waters piping system But I think you do need actors to to maintain efficiency I think a functioning market which we didn't have in the subscription system But a functioning open access market is a good thing to make keep things innovative and and flowing Yeah, very short. Yeah, just just to realize that these services are already being offered outside the publishing Circle so for example when I'm that we have translation services if I'm not good with English then I pay for translation my group My research group pays or for or to produce figures or to all the or for typesetting actually So all these things can be totally independent of the actual process that we're describing They can be outside from the from the cycle so it doesn't need to be provided by the By the infrastructure itself or it can work in some kind of overlay Manor, but this is already costs that exist and they're not calculated in the actual model translations are not calculated for example or editing their professional editor almost every Decent research biomedical research group has their own editors and they're paying them and that's not related to the Revenues of the publishers. They're outside this Okay, so we have solved the problem with the publishers. That's good No, because there was a convergence in fact through the discussion With this idea of the publisher as a service provider and open peer review could be one of the services or can be one of the services But it's not compulsory. In fact, that's interesting so RV then Tony don't be afraid. I have an eye on the clock and I'm going to now to merge the two following question Which is around the question of the uptake and the adoption by the different communities scientific communities of open peer review practices. So is the question of incentives, but for the researchers this time Stephanie could you could you give us your your advice your opinion on that? What are the different level of adoptions inside the different scientific communities and how we can work with that? Yes, so and as I as I mentioned in my talk at the beginning and While I think well I think the benefits of open peer review are very obvious and in this room We're all very much in in agreement on open peer review When you go out and speak to researchers, that's often not the the response you receive They're often very anti wanting to sign their reports put their name to to their To their report and I don't have an answer to how to solve all this But I think there are a number of things that we could do to try and get to a position where Peer review is a much more open process. And so I think I think more research into Peer review into into the quality of peer review We can really sort of definitively answer the question as to whether Open peer review is better quality peer review not just looking at the quality of the reviewer reports But I think it would be really interesting to do some kind of study to look at whether the final articles Are better quality because that's essentially what we're striving for is not the quality of the report itself But what peer review is adding to that article so That I think that would help and because obviously researchers like evidence so and we could use that to to try and engage people, but I think also Getting out there and kind of working with communities and talking to researchers about and our experiences of open peer review and that The concern that no one will agree to review isn't founded because we've met we run journals on open peer review and people do Agree and to sign them and I also think possibly and there is an argument in fields that are very Not used to openly putting their name to criticisms to maybe look at a more stepwise Process so maybe normalizing publishing the reports and with with the option to sign sign the reviews to then Hopefully evolving to have a more open system because I think as much as we believe in the values of open peer review We can't impose it on researchers if that's not what they want We can't tell them that that's how they have to do it Which I think brings me on to the second part of the question which I think was to do with incentivizing peer reviewers and Peer reviewers are providing a really really important valuable service for free giving up their time So they need to be happy with the process that they're reviewing within and but they also need some kind of Recognition and I think that's an area that's becoming increasingly recognized that we need to reward peer reviewers and for their efforts and I think open peer review definitely and Enables that Reward because instead of just saying oh, I reviewed for that journal, but you can't see the report Can take my word word for it that it was a good report Your report is there in the open with your name on it and people could go and look how good the quality How good your report was and and I think and and also things like being able to assign a DOI to Reports on open peer review also allows and the next steps of credit to then come in and I think perhaps and that might then help Researchers and gain recognition from their institutions for the time that they're spending On peer review as well as the time they're spending on their own research and the other activities that they're engaged in Because of in your opinion for the moment, it's not the case at all. No institution do recognize. Do you have an example of an institution? Which tried to recognize open peer reviewing practice for example in the assessment of research researchers? No, it's completely out of the blue for the moment. Yeah, okay, so and Would you react? I would be happy if there would be some incentive for a sense of your setting on a report You're spanned easily half a day or so if you do your job quiet accurate and But if you are summing up finally at the end of the year at yeah, the report for your faculty That's what I'm doing every year. No way. I can say I review 20 papers or one. Nobody cares about since I cannot prove it I think that's the key issue and the other question is there were also ideas about Reward system for reviews which said okay, let's pay them Okay, and it's an easy calculation if you consider an STM only two million papers Published published every year four million papers submitted times at least two reviewers 2.5 at average so ten million Payments of let's say one hundred bucks one billion U.S. Dollars per year that's quite a lot of money and that's also what publishers said who that's a lot of money Brackets in brackets, of course, they are earning seven to ten million a billion Every year from the classical subscription system, however, it's one billion and that's why it was Yeah, it was thrown from the table then and no longer considered But I think having a DOI as a reward or incentive for the reviewer a sightable Something sightable for myself as a reviewer that would be a great advantage And I think that's only possible if we open up The reviews post the identity of the reviewers there Otherwise, I cannot prove myself as a reviewer of that specific article who contributed contributed to the quality in the discourse I think that's the only way that's why Okay, so we tend to always when we speak and when we think to Say that the reviewers are being selected by someone and they're doing it for someone and this someone needs to recognize them So it's like I think we should aim towards a totally shift in this in our in our way of thinking that the reviewers Have to be the ones who select what they review and this is where the incentives come I choose to review this work because I believe that I can improve the Improved its its quality I think that I have something important to say to the authors and if I actually do that and if I'm correct in my Estimation then the authors will be delighted that somebody is coming and is helping them To improve their work and most probably the next work will be in collaboration with them so This this there can be no no other no no better incentive that to open your research network get recognized by your by your peers and Bring new new quality to do research So this should be the incentive then you don't need doys Then you know you just do what you're supposed to do You're you're doing your your research and you're sharing it with your with your others either from the point of view of the author of The reviewer of the editor it becomes it Gets mingled. There's no distinction between authors and reviewers. It's just people getting together and improving each other's work This is it's a community process. We don't we don't have the culture and we don't have the infrastructure How can it be the incentive? Science open the platform we launched you can use yeah You can take any out of the 1 million archive papers and say I'm doing peer review right now Or since last week you can also choose among all these Cielo papers from Latin and South America to do this So I think we have the infrastructure. We have the infrastructure in terms of the repository Or the classical journals and we have the infrastructure For for example the platforms the win over or science open to do this. It's working. It's in place Just use it So the question I I give to the audience to to close this discussion about the incentives Is that we have a discussion about do we do researchers? Do their research for their career or for the improvement of science or maybe for the two of them at the same time What is your opinion regarding the incentives to participate to open peer review as an improvement of quality? Just one insertion about that. Yes, of course. There's the we know where their science open There is SJS when what what are these what are these their private initiatives trying to convince people that they should use them And that's what normally we're trying we're trying to do But that's why This is this is the problem. That's why I'm talking about governance That's why I think that it should be a federated effort a community effort It cannot be somebody trying to convince others that this is the this is your space now This is where you should do it because they can be as many initiatives as money as people who can gather around So we shouldn't we should come together and do this in a confederated manner instead of Ten years ago Where's the where's the black? When can I do it? Yes, no, that's Sorry Yes, I think it's also Yeah Doing it Josh Josh Largely not by choice, but by necessity. That's the only way we could get money We went to all the foundations said hey you want to do this? No, who are you? No, who are you? There's a lot of money, but it's not easy to get it Whereas if you kind of provide some financial incentive to maybe your backers down the road you can create something And so we wouldn't even exist. I know we could wait forever kind of thing Yeah Okay, Jean-Claude I'd like to go back again to that distinction between commentary and review and this time Link it up with two two elements. One of them is when you have Disciplines where the rule is that you have dozens of authors It seems to me that when these dozens of authors decide to go public There is already a good deal of peer review that has been going on inside that team That's the first thing the second thing is that when they go public they go public through some channel if the universities or research Centers essentially gave the okay to go public Putting into the the way to that the good name of the institution is at stake And that if you don't do the right work, you're going to be punished locally by the institution somehow It's going to enter in your evaluation Then you have a system in which people are going to be very careful about what they what they're going to be putting out And at the same time this would be a corrective for the present Institution system to just do counting the counting of a publication, which is nonsense You know get into sausage sausage slicing sort of situations And then after that you've passed in the sense the phase of review and you can go into the commentary phase Okay, so speaking of sausages I am Terribly sorry to tell you that We have to close this this conversation And I think we have enough food for thought to continue the conversation with the sausages now I'm terribly sorry that we didn't have time to To go into the topic of open science and open peer review, but it was too much So maybe we should organize another discussion about this specific topic because it's very important. I agree with you I know Thank you to all our speakers for this conversation and all the ideas that you get. Thank you very much