 Thanks very much for coming along to this session for meeting the editors we're trying to let the community know what we're doing and provide a forum for answering any questions you might have. So yeah let's just get started and see what we get to. So we're just gonna I guess you'll get a chance to meet four of the E-Life's editors and then I will take you through this new big thing we're doing at E-Life trying to get published and review model working or lay out the steps toward the published and review model and then we'll have a 30 minute Q&A at the end. Cool. Thanks. So maybe Maria you can take over for this slide. Sure so just before we start a few points on participation guidelines so you can post your questions in the Zoom chat to the panelists. We will invite you to ask your questions during the Q&A part of the session so we'll have roughly 30 minutes for that and we can unmute you if you'd like or we can just read out the questions for you. We also want everyone to enjoy the meeting so to ensure that everyone has a chance to contribute we ask that all participants abide by E-Life's conduct. So some examples of behavior that we consider will contribute positively to our communities including showing empathy and kindness towards others and being respectful of different opinions, viewpoints and experiences. Examples of unacceptable behavior that we will not tolerate today include for example making it difficult for others to speak or participate through repeated interruptions or disruptions. On the background we also have my colleague from the E-Life staff team Ania Therese so if you need any technical support please direct your questions directly to her and for your reference this webinar is being recorded. Brilliant thanks very much so maybe we'll just move on to introducing a few of the people who are here today. So I'm Tim I'm a deputy editor at E-Life I'm a neuroscientist at Oxford and I've been at E-Life since it started so a long time now but I'd like to introduce three other editors who can tell you some of their experiences at E-Life and what kind of manuscripts they handle and what they do so maybe start with Laura. Hi I'm Laura Colgan and I am in the Center for Learning and Memory and the Department of Neuroscience at the University of Texas at Austin. I'm a senior editor at E-Life I've been at E-Life for a few years I started out as a reviewing editor and I sometimes will still act as a reviewing editor if there's a paper that I feel strongly about that I think you know deserves a shot. My research I'm a behavioral neurophysiologist so I record from freely behaving rats I'm focused on learning and memory and I'm interested in the dynamics of neuronal ensembles during learning and memory operations with a focus on how rhythms coordinate neurons and looking at place cells and other cells in the entorhinal hippocampal network like grid cells. The kind of papers that I handle at E-Life I actually handle a really broad range of papers so if you see me as your senior editor and you think what does she know about this topic please don't be alarmed rest assured that your reviewing editor will be an expert on the topic and I would never make a decision about a paper on my own it's always in consultation with reviewing editors who have expertise but at the same time as you are all probably aware at E-Life we are trying to understand the broad impact of papers so I can see things from a broad perspective so I can handle anything from slice physiology papers up to even some brain imaging papers or human EEG with oscillations I sometimes get those types of papers and yeah that's it for me. Maybe Denise can go next. Hi I'm Denise Kai I'm a neuroscientist in the Department of Neuroscience at Mount Sinai in New York. My lab is really interested in how we accumulate memories across time and we're particularly interested in the intersection of memory or the tug-of-war between memory stability and memory flexibility and one of the primary ways we look at that is visualizing how cells in the brain you know in the patterns of activity of the brain underlie the memory representations and then using we use miniscopes to do that and freely behaving rodents and then we use optogenetic chemogenetic ways to manipulate those circuits to see how they contribute to memory. My lab also is committed to developing open source neuroscience tools including miniscopes and analysis packages that are freely available and I've only been reviewing editor with eLife for this past year and I actually had another experience as a reviewer feeling very frustrated wanting to have a discussion with my fellow reviewers about points where I disagreed with them on this manuscript I was reviewing and I just thought you know I really like the way that at eLife there is consultation amongst reviewers there's accountability there's opportunities for discussion and so I immediately emailed Michael Eissam I was like please like can you one day consider me as an editor I want to support the efforts of kind of this open review process and that's how I got involved so that's it for me oh I handle papers mainly on the topics of learning and memory and viva calcium imaging as well as open science or open source tools thanks. Fantastic I'm wondering. Yeah hi I'm Andrew Prashinsky I'm an associate professor in the Department of Physiology and Pharmacology at Western University which is in London Canada just outside of Toronto. Yeah I am an electrophysiologist working in humans and macaque monkeys and also recently trying to move into calcium imaging in marmoset monkeys. My research centers around the neural control of movement specifically how somatosensory inputs from the skin and muscles play into the control of the of reaching and grasping and object manipulation so we are very much closely tied to the to the peripheral apparatus recording from anywhere from the peripheral afferents themselves tactile afferents or muscle or muscle spindle afferents looking at reflex responses as well as going up to the to the motor cortex and maybe sometimes to the premotor cortex as well so my the kinds of papers that I handle at at elive revolve are all in the domain of systems neuroscience usually in the domain of sensory motor neuroscience either sensory processing somatosensory system sometimes a bit further afield in the visual system or auditory systems and certainly motor control studies in a variety of humans as well as in a variety of model animals so the my experience there has been very exciting as well and I like Denise said I've been there for a year and a half and have really enjoyed the the the community with respect to the reviewers and the reviewing editors and and learning about you know these these these fields and having a good back and forth about the about the science as it as it comes in fantastic thanks very much indeed um so let's move on to the next slide uh cool so now I'm going to tell you about something you were doing but first I'm going to give you a little background to elife uh and its history so and why we're doing it so our job really is to change our publishing works that's why we're funded uh we're funded not just to be a journal that publishes good science but to be a journal that innovates that tries to break or or transform a system of publishing that everybody knows the problems of and uh but there's a sort of um cycle of of uh that seems impossible to get out of of people requiring publication um from from certain places to get jobs and therefore those places becoming even more uh prestigious etc etc that cycle is hard to break out of and elife is funded partly uh to think about ways of getting out of that cycle and to make a new more transparent more open more generous way of evaluating science and we have a history of doing this so we've um got a history of increasing transparency of of um pay review whenever we can as Denise and Andrew have mentioned this consultative review process came out of elife and is now in many journals uh it leads to clear and quick decisions because because the review team can make consensus decisions about what you need to do to get published uh and everything uh where we can is is open so open access open data where possible or a big supporter of open science and also we've made technical innovations things like executable research articles which are fantastic so you can get in articles where you can go and and like run code from inside the articles change the change the statistics or change what uh how the figures look and play with the data inside the articles so we've been trying to promote those more and more as well so we've got established ways of doing that and every time we make we make new publishing software or infrastructure we make sure that that's publicly available and shared by everyone so other journals can take can make the most of it if but um if and when they want to so so we're proud of that record but somehow we haven't done really what we've been what we were funded to do or what we or what we want to do which is to really dramatically change the the shape of publishing we're still in the game even with elife uh the journals are gatekeepers and enormous effort is wasted in that gatekeeping process so you might submit to many journals years to publication and reviewers can control not just how your paper is is is seen by the by the world but also but also even whether it's published i mean which which is um which is this this old-fashioned way of publishing that's that is we think is a worry and equally an equal worry to that is because journals act as gatekeepers the only real form of evaluation that a paper gets publicly is just the journal's name that's it there's no other public form of evaluation of a paper or other evaluation is privately within labs etc so so we think that we really need to move to something radically different so maybe next slide um and so we think that there's now this amazing opportunity to move to something radically different because of what's happening in how people communicate their science which is just an extraordinary change over the past five years this uh massive takeoff of off of pre-printing where there are now something like 4 000 articles a month published on bio archiving meta archive catching up um which gives us totally transforms what it means to be a publisher in our view in science so next slide so we we we think that we aren't really publishers anymore journals are not publishers anymore they're curators of a literature that's already published and so i can tell you now that something like seven even even before any of these new policies i'm about to tell you started something like 70 of the articles that elive published were already published by bio archive before we published them like so so we weren't publishing these things that people already knew about them um instead we were just sort of giving them a stamp of approval and if that's what we are if if journals really are stampers not publishers then why do we just stamp with a single label that says elive and not with a rich interesting stamp that tells the people what's interesting and good about the paper not with uh uh a why does only one person get to stamp that paper not many people that find it interesting why for example if you're trying to evaluate an interesting medical thing that requires a complicated statistical tool why isn't there like an interesting medical things journal and a complicated statistical tools journal both get to have a say about whether that's an interesting paper it seems it seems archaic to just have this system once everything's online where one set of three reviewers get to give the only stamp uh to a paper and so we think this is an opportunity to totally change that this rise of pre-printing is an opportunity to totally change how we think about evaluation of science so we want to we want to think broadly about that and what I'm going to show you today is the is the results of our of our first thoughts and our first policies but obviously we're open to to to continuing to think about it and change these policies but we what we like to do is to create a new type of manuscript a pre-print that is that is evaluate that comes with an evaluation that comes with an evaluation from as many people who want to evaluate it that is and that and that is open and available to scrutiny and discussion an evaluation that doesn't just end at one point in time and we want to therefore be able to develop to to plan how that looks what does this new evaluation look like both in terms of the infrastructure which we're trying to build so we've got big teams of of computer coders coding up infrastructure for for for building new tools for for for for present disseminating science as well as but but but perhaps more importantly we need to as scientists work out how this system might work so that we can be responsible respect for in in in our comments in public but also make things that are that that perform the role better than the current journal journal system so make things that really are useful to the funding agencies useful to the hiring committees useful to the journalists and the other key stakeholders that need the the validation of peer review okay great maybe the next slide will be good so this is the kind of thing where at at the moment so on but so we're making these things on bioarchive we also have a separate website called society which is nascent but it's another way of looking at these things and will we hope become the primary way of consuming the whole referee pre-print but on bioarchive right now you can see what we've we write we write a little summary about overall this is what we think of the paper both in terms of how interesting it is how robust it is whether or not we think it's it's conclusions are supported and then we'll write some public reviews and those public reviews won't be like a normal private review that is written for the author with a list of detailed problems they'll be written for the for for the reader and not for the author so they'll try to highlight the key strengths of the paper as well as some weaknesses of the paper they try to they'll try to make a document which helps the reader understand what the manuscript has achieved and what it hasn't achieved cool okay let's move to the next slide so i just wanted to give you a couple of examples of these evaluation summaries we need a particularly good ones but just to just to give you a feel for for some of the things that we might do uh to replay that might eventually replace something like the journal name Eli for or nature so instead of tagging it with a name we'll tag it with something like this so this bit in red this study is of broad interest across a diverse range of neuroscientists studying sensory systems for walkout of behavior and inter-regional interactions it's a unique dramatic and important demonstration of the specific interactions between two areas etc etc or this study is a tour de force that makes a major contribution to the field it provides a massive amount of information about the connectivity of drosophila factory system identifying a variety of interesting features etc etc so that kind of clear concise statements about what it's achieved what what is achieved in the context of its field that could go in your cvs or could go in your in your grant applications in the place in the place where you currently put your your journal name we think that the funding agencies and hiring committees will quickly learn that that's a more useful thing but we'll only do so if the whole range of those things are possible so those things need to acquire the respect that journal names currently have amongst the other community there are the as we go forward just as we go forward there are going to be other ways that we're going to try to provide rich evaluation for example we'll try to provide clear statements about about the quality about about like badges of saying how strong the data are or open data whether the thing is has been pre-registered etc etc there are like clear statements that you can say about important things that the that the that the paper's done but and we're still working on new ideas for to go into these rich evaluations we're calling them okay next so and so now as you go to Eli as you as you if you submit to Eli this is what's going to happen to your papers this is one half of what's going to happen to your papers i'll tell you about the other half in a second so one half is this we plan to from actually next month July we are exclusively reviewing papers that have been posted as a preprint you can only submit papers to Eli that are preprinted and our we're turning the focus of this editorial machine we have six seven hundred people instead of making binary decisions we are asking them to concentrate on understanding how to make really good public review and how to curate the literature that's now we think our primary job okay next slide please however we know that right now though these refereed preprints aren't yet valuable to you guys you still need our name to get funding to get jobs etc and so whilst we try to increase these referee preprints as a currency and increase their currency at the same time we're going to be we're going to be making our traditional binary decisions and publishing papers in the same way as always so now if you submit to Eli perfect you get both things you get a set of public reviews and a richer evaluation and you also get a published paper just like Eli's mark one which means we have to make these binary decisions too next slide please okay so of course we like a lot of the things we we did on the way and so we're going to keep we're keeping all of the good bits like the consultation the clay decision making the limited requests and rounds of revision but now you'll also get in your reviews this separation of the public review which is this thing written for the reader which highlights the overall strengths and weaknesses of the manuscript along with some separated private recommendations which are about not about the the preprint but are now just about what you have to do to get the paper into Eli and so that's your that's like what the old reviews to look like and then obviously then you'll get that evaluation summary that i've just told you about and and those those public reviews are going to be posted to the preprint server i'll tell you a little bit more about that in a second carry on so yeah so as i say you get this this preprint with this this referee preprint and evaluation summary and critically we're going to want all public reviews to be to be published but we don't want but we are still rejecting papers for me life and we know that you guys still want papers published and we don't want our public reviews to prevent you from getting published in the future and so we are going to allow you to control when you choose to publish those reviews and so you can you can if you choose to publish them after your your papers being published in another in another journal so it can't prevent you from getting published elsewhere oh next slide please that might be the end oh we think that there are lots of advantages to this you guys control when your work is shared there's no gatekeepers to publication anymore we by divorcing the evaluation from publication we think it can be richer more transparent and it has many opportunities for things that you can't do if all you are as a gatekeeper and we think that we can use this to make peer review more respectful more and i mean critically right now a pair of viewers job a pair of viewers think that they there are that they are working for the journal to try to keep the journal's impact factor high oh i've got to protect this journal from this paper i mean it's a ridiculous state of affairs right what i'm trying to do is protect this journal from the paper what what the peer reviews should be trying to do is constructively contributes to the to the to the scientific dialogue and so we hope hope this will allow that to happen thanks very much indeed thank you team i wonder if we'd like to invite Laura Denise and Andrew to also share their thoughts about this new model before we open up the Q&A yeah great that's a great idea yeah maybe i can start because there's something that i'm really excited about with this public reviews and something that i think a lot of people in the community are really excited about and that's that it's going to provide the database for researchers to do scholarly research on biases in the review process with this which is you know something that is another really important part of the mission at elife is to try to you know we're all human but to try as much as possible to to make sure that biases in the review process are minimized so i think that that that's really another aspect that i'm very excited about yeah i i can comment too i think all as tim said at least in my world of systems neuroscience most of the papers i read now are bio archive papers anyway and you know i think that it would be so nice to have other scientists already kind of you know give us the highlights or things that we should think about as we read the paper to begin the discussion i feel like that's the whole point of science right is putting our science out there so that we can engage in scientific dialogue and learn together and i think that having this kind of open publish and review and as well as having curating comments from respected experts in the field i think we'll promote that kind of open dialogue and accountability as laura said yeah i wanted to add i've reviewed now i think maybe two or three papers using this new approach maybe two but what i've noticed is that over and above the the fact that before this when elife had this consultation process which read to which led to this consensus sort of review that you got which i think is to me really the the the most amazing thing about the elife experience relative to other journals though other journals are picking that up now when you introduce this uh these reviews that are going to be posted on the preprints and you instruct the reviewers to consider that they're writing those those parts of the reviews for for the audience for the readers of the paper rather than the authors it really does change the the further enhances this consultation that happens with respect to your paper when it comes back with the with the comments from the reviewers so i've seen an even it even amps up in terms of you know having the conversation about like what are the real key positive aspects of this paper in a really in a really good way i think overall that that leads to uh comments that are that that helped the paper get better and without being sort of obstructive at the level of you know the the sort of most detailed aspects they get talked about too but only in so far as they're sort of framed within the in the more general advance being provided by the by the paper so i think you this really adds to that thing that i think most people already like about about elife overall and can i just add one more thing the enterprise which is that you know my experience at elife as both as an editor and also as someone who's submitted work there is that because there's this accountability amongst the reviewers to have to discuss and decide you know what are the important points and you're revealing yourself to your other reviewers and i find that the comments are very constructive and respectful and and so i think that kind of accountability really lends to constructive reviews rather than just someone who can hide in their anonymity and be an unfair reviewer yeah and i would also like to add that um sorry it's not you know my door i would also like to add that of course this option is rarely used but we do have a rating system for reviewers where comments can be added and if somebody comes into the consultation process and they're you know just trying to push their own agenda or they're not um being constructive they're not you know they're we're asking them okay but what are your specific reasons for feeling that this paper should be rejected and somebody is not engaging in a constructive discussion with their fellow reviewers and editors then we can actually rate them as disappointing and and then write a comment to say if someone's going to choose them as a reviewer in the future you know hey this person wasn't really willing to listen to the opinions of others and to um to have their opinion changed and and then you know that would make them much less likely to to have the opportunity to review somebody else's work in the future so if you have questions put them in the zoom i think um uh rikardo is asking um how are the public review reviewers recruited are they voluntary or recruited by elife or by the authors so i i mean any of us could answer that maybe lori you want to have a go oh yeah so it's the same reviewers as review the paper so all reviewers now are required to write a public review but we would invite you all to to share your questions in the zoom chat or if you'd like to raise your hand we can also just invite you to share your thoughts and i wonder if a question that we frequently get from from people is also whether the identity of the reviewers will be posted on the public review team would you like to say a little bit about that so um we would they will not be they will not be for various for various reasons but the critical reason is that we think that that would exclude a large portion of the community from from performing that role because people are not everybody is happy to reveal their name as a reviewer and we don't want to exclude a large portion of the community from from from this system particularly because the portion the people who we would exclude would be biased in the toward in in in a way that wouldn't be productive to anybody so that it would be more vulnerable people would be less likely to sign their reviews so we've taken the decision to publicly present the reviews anonymously so that elive the journal takes responsibility for them not the individual but nevertheless privately as always elive if reviewers would like to reveal their name to the authors not to the public but to the authors then they can do so um and we will we're aware that that's a possible conflict so we will then report publicly uh which of the reviewers have told the authors their names and so it's a uh that's so that's what we think is the best compromise to that complicated question and also i think just in case people are generally familiar all of the reviewers and are are revealed to one another which i find is sort of like to me it's sort of a goldilocks zone of keeping some you know uh professional responsibility to one another without necessarily forcing people to reveal themselves to the to the original authors and i think that that probably mops up a lot of the things not all but a lot of the things that people talk about as positives when they discuss revealing a reviewer identity excellent great um so we got a question from afishak um he said hello as an author when we write the response to the reviewers should we reply to the public reviews or only to the recommendations to the authors what if there are points raised in the public review that are not present in the recommendations so we hope uh i mean any of us can answer that as well but i mean maybe i'll take this one so we hope that your reviewing editor has been extremely clear telling you exactly which points you have to address at the top of your review there'll be something called essential revisions we hope that you will be able to understand from that section exactly what it would take to get at your paper published into e-life my experience is that that's almost always the case but there are some reviews we send out where that isn't the case and people have to troll through the reviews to figure out what the important points are but we try wherever possible to be totally clear about for revision decisions about what it would take to be published and then we hope that your job is only to address the questions that are in the essential revisions and other things uh should be um should be addressed up to you whether you address them or not it's not always the case that that um that reviewing editors are as clear as they some perhaps should be um so maybe you have people have seen examples where that isn't quite as good but i think in a lot of the cases it's like that there it's very clear that in the essential revisions the things that you have to do to get published are just sitting there in the central revisions right at the top of the review and i would just add a little bit to that as well that the essential revisions is something that comes out as a consensus across all the reviewers after the consultation process whereas um unless there's something that for whatever reason the group decides should be edited in the public reviews the public reviews are what the reviewers submit before they enter into the the consultation thank you Laura a question from Ayuno Nakahashi so there will not be a record of the rounds of reviews that are currently available on e-lifepapers i don't understand that i wonder if the question is also whether when we post the public reviews if there if there will be any record to how many rounds of review or revision that paper has gone through i think that our answer is really just to say that the public reviews will only be posted to the version of the pre-print that has been submitted to go through pre-reviews that he likes i let me just be try to be clear about what i think so i think the answer to this question is we will publish the the peer the uh publish the reviews for the first version you submit to us and then we will publish a statement at the very end which says which updates the evaluation summary and says which of the criticisms have been addressed whether they've all been addressed or whether there are some outstanding ones that's so that's what i think is the situation and we will not publish all the intermediary rounds of discussion that happens between the between the reviewers and the authors thank you Jim another question so if i am acting as a reviewer for e-life is an editor expected or can they edit my review if changes are necessary and if so what's the process in terms of seeking my permission you want to take that i um i'll speak on this i mean so even as a reviewer i have made errors and it through the discussion and the consultation people have brought my errors would like to me where i was able to then update my my review but also i think reviews can take at least two forms so one which is after the discussion the reviewing editor might rewrite kind of a you know a real summary of the discussion of the main points and then kind of list also maybe some of the minor points and some and other times it's not a rewrite the whole thing but that they will post the individual requests from each of the reviewers but certainly i've seen that the reviewers can update their comments especially if they've made an error so that's true and so in terms of uh reviewers re-editing their things off the the consult that that's exactly right and then the other thing that sometimes happens is that uh there's a little bit of editing happens after it's left that consult by either the senior editor or by me or by the staff and we will so there's two forms of editing that might happen uh we might edit stuff that we think is offensive or is um inappropriate um uh or we might just reword little things to make it clearer that's one form and the other thing we might sometimes do is shift things from between the public and private reviews if we think that what that it's more appropriate in one place or another and we will not ask your permission to shift things from the public to the private review but we will ask your permission to shift things from the private to the public review so anything that's going to be made public we'll ask your permission for yeah and i'll just add to that and say that um if we make if we think that there should be substantial changes made to the decision letter where or to the public reviews or any of the reviews i mean we try to keep it as authentic as close as possible to the original reviewers obviously um but if it's something substantial then we will as a senior editor then i would bounce the letter back because the the decision letter that is written by the reviewing editor has to be approved by all of the reviewers and so if it's substantially changed then i would bounce it back and say hey you know this was listed as an essential revision but to me you know from the elife mission this actually seems more like a minor point like if it's just some you know piece of speculation in the discussion section or something like that that's not essential to getting the the paper for the claims of the paper to be robustly shown then um you know that type of thing i will run it past the reviewers again and just let them know or ask them if they're okay with that yeah and i think it's just worth reemphasizing that you know when you're in that consult for those that haven't been elife reviewers when you're in that consultation you see sort of the letter that's made right so one thing that the editor can do of course is write the essential revision so there they can pick and choose which of the comments they think are the critical ones in consultation with the with the authors of those original reviews and formulate that but of course the the reviewers themselves can feed that back and say no i disagree or so on and so forth and eventually you come to some consensus on what those on what those are so there's you know there's the editing of the actual reviews that would be probably i mean from the from the reviewing editor's hands in my experience i wouldn't go in and edit someone's own reviews they might do that themselves if they discover they're wrong about their second major point but where i would exert that would be with respect to the essential revisions that i that i point out at up top yeah we do sometimes ask us to mention to edit it if it's some really like kind of histrionic language that's too emotional and we feel like reading it that's going to maybe be a little bit too strongly worded to be critical of the authors you know unnecessarily and and sometimes it's just people don't realize when they're putting something in writing how it sounds like if somebody else reads it but again if anything you know we've run it past the reviewers again and just let them know if it's anything substantial as long as we're talking about different article types i might just chime in about a different category that i think a lot of times authors are confused about and that's the difference between research article and tools and resources so oftentimes people will submit something that's more appropriate for the tools and resources resource category as a research article in a research article it has to present results provide you know major new mechanistic or biological insights and tools and resources don't have to do that but i also think another misunderstanding with tools and resources that we often see is that it's more just like a traditional methods paper and that's not really the point of tools and resources either because tools and resources is expected to provide a tool that's going to make a really broad and major impact so it's going to allow people to do experiments that aren't currently possible or it's providing you know some open source tool that's going to allow people in a lot you know a lot of different areas of research that are not currently able to access a specific technique it's going to allow them to start using that technique you know things of of that nature but i think that that is a really um i i would say shortcuts me personally i don't really handle very many of those and i would agree already with what tim said i think they do have a higher bar and yeah but i feel i get a lot fewer short reports than i do tools and resources is it which is a category that i really like yeah so my take on a tools and resources paper is this if you've got a methods paper which other methods paper are going to people are going to be in which other methods people are going to be interested in then it's not an elive tools and resources paper but if you've got a methods paper which biologists are going to be interested in because it's going to let them do something new then it's an then it's a elive tools and resources paper yeah i also just want to comment on the tools and resource um article type because i think it's so cool um one of the things is that it doesn't have to be an entirely novel method but it has to be impactful and usable and have a broad audience and broad user base um and so i think that's something called that's maybe a little bit different than um some of the other methods journals um and i think the other thing to keep in mind is that because there's no you know word limit page the limit like please make sure to add enough detail um so that the the reader can know how to deploy this method or system you know in their own hands in their own lab for their own research um which i think is one of the goals of um this article type thank you the news um we have another question from a fish check what is the general position on studies in which part of the figures are replication results uh replication is important for science but difficult to publish so we've got a similar question this morning um i wonder if Tim would like to to start the conversation this topic yeah unless somebody else wants to i mean i i can say my piece on it um uh so elive has um historically uh been um a massive proponent of replication research so we have we have even drives part of which we fund to organize big replication efforts in cancer biology and various other places as well um we um uh agree it's an important part of science uh we publish replication studies and failures to replicate uh but we still require them to be interesting um and so we think so uh and you and maybe it is true that um that um i think i think um in a situation where where there was some doubts about whether some piece of work would replicate and if some group comes along and says it replicates we would certainly publish that or in a situation where some established thing that that is influential in the field fails to replicate we would certainly publish that as well but i don't think but but it isn't the case that we will publish a replication just because it's a replication i think that's that that's the uh uh the game but we are happy to publish we're not in i think the the best way of thinking about this is we are not in the game of chasing citations and so we're in the game of being making the scientific community better that's what that's the game and so if it's useful to the scientific community to have that replication like really useful in a way that any paper should be really useful then we will publish it even if it even if we think it will never be cited that's something that nature won't do yeah exactly that those that that's that's kind of the reason we're still looking for stuff that's impactful on the community but not we don't care whether it's helpful on us and i think i think it's worth pointing out what happens behind the scene well i don't know what always happens behind the scenes from when a paper comes in but the way from a reviewing editor's perspective what happens is there was presumably some a number of submissions i'd be interested know from from tim or laura how many of those then they immediately just don't send further down the list but eventually they recruit some number of reviewing editors to discuss that paper right so even the the the the decision whether to triage or to send out for review is is happening i think typically in a consultative manner amongst the reviewing editors so amongst the reviewing editors there would be a discussion about whether that replication is important for the field or not and if and if they were deemed to be the case then like any other paper it would be sent out for review that's not to say that every clearly not every replication is going to be get that sentiment i can't say if the bar is going to be higher or lower it's it's going to be different kind of different kind of thing i've certainly seen papers come through that as the question asks includes some aspect of replication and that i think can only be seen as an advantage if it's within a paper that then goes over and above those original results so yeah i think sometimes it's missed that there is some discussion amongst the reviewing editors to make a decision about the importance of those papers and so that of course is a scientific discussion amongst your colleagues about whether it deserves to get the dump deserves whether we deem it you know such a level to be sent out for peer review and it's also different than i think a lot of other journals in that the bar is that if i have you know three reviewing editors that are like i'm not really that excited about this paper but i have one who's like well i disagree and i think this is exciting and i'm willing to handle this paper then it's going to go out for review so it's not uh i think that the the triage process you know as far as what andrew was saying about what happens with me um i mean i would say i would rarely probably never just look at a paper on my own and decide to triage it there's always a consultation with other people and if we don't if i think if i can't find people to consult on it within our editorial board then we also have the option to go to outside experts and ask people in the community what do they think about it and you know they may even be willing to act as a guest reviewing editor which is something else that that we do so so i i think the difference the difference in industry differently i think some so i i'm a bit of a unique case because i see some papers that are just wacky um but um so i definitely do triage stuff without consulting anyone if it's about sort of extra sensory perception or or like Martian mushrooms or something um um uh or quite often new um quite often new uh fundamental laws of physics get submitted to us um to beat quantum mechanics and relativity that kind of thing doesn't often get past me um but um i i think also i would sometimes if i think that i'm as qualified to look at a paper as any of the we're reviewing editors and i think it's clearly not up to the bar i will i will uh sometimes triage papers by myself i think that that varies between senior thumb some as many as 20 percent some do none by themselves yeah exactly i mean i would also say i do if it's something that i feel that i'm very qualified to evaluate it and it's not you know any chance that i would have a personal bias against it um i would maybe do that but the decision i think it is rare for me probably to triage things on my own i don't personally feel that comfortable in that position it does make me feel like kind of a gatekeeper where i'm more of like a collaborator type of personality um but um it will say in your decision letter then i would say you're a senior editor you know i'm not going to lie and say i consulted with reviewing editors aka me and my multiple personalities no thank you all um we we still have a few minutes left that there is another question to say um bioarchive supports discussion and comments from the general audience woody life reviews and editors take that kind of discussion into account for a final decision on accepting or reading the paper uh who's that for i can take that if you want um so this i i think the answer so the answer question is no not right now the the you you're right to say that this new mechanism of prc public review curate potentially allows for wildly different ways of thinking about the evaluation and we are thinking of wildly different things like for example just telling everybody what papers are under review i mean they're already published why why don't we just tell them what's under review and then anyone can submit reviews to us that'll be one way of of uh of proceeding and those things i think need need care and thought because for example your mates or your enemies could could just submit reviews if they saw your paper was online they could say i like this guy i'm going to submit him a good review or there are all sorts of intricate dangers and potential biases that would emerge if we're not in control of who who evaluates the paper and so at the moment we're not doing any of those things but we are doing research and thinking amongst the community of editors about how such things might be done without bias because it is really exciting because the problem at the moment with selecting reviewers is that it feels unbiased and careful but actually quite a lot of the reviewers are like Andrew's mates whenever Andrew is the editor or Denise's mates you know and so there is a bias there there is a bias and we know that there's a bias towards North America away from Asia away from South America there's in our reviewers and we know that that affects the acceptance rates the acceptance rates are much higher from North America than they are from Asia and so there's really there's real opportunity with this new system to try to counter out those things but we don't want to do it in a way that puts everybody off the whole system allowing new biases in there and so that's what i think where we are with that right now i just want to comment about you know choosing the reviewers and one of the things i really appreciate about Eli there's two things one is that during the consultation with the reviewing editors and the senior editor whether you know deciding whether to send out for review or not all of the reviewing editors consulting can suggest reviewers so that it's not just Andrew's mates or my mates and so that's something that's been very helpful especially as an editor to get you know go beyond my mates and circle of influence and and to get a wider broader you know group of people to review i think the other cool thing is that there is an early career reviewer a curated list of you know people who are postdocs or assistant professors or whatever and they want to review and they've been vetted by you know other people in the field and it's been amazing to like work with these early career reviewers to get a different perspective as well and so that's also been very cool about the Eli reviewing process all right i think we don't we don't have any more questions for the benefit of the attendees i'll just put a link into the chat with information on how people can apply to join our early career review pool for those of you are still young investigators but otherwise i think we we can finish so thank you very much for for your time thanks the editors for lending their time and the expertise and to all of you who joined us today as well and if you have any further questions please feel free to get in touch with any of you of us via email you can use the email address on them on the slide but you can also reach out to to our personal email addresses or institutional ones and we would encourage you to also take a look at the the newer science selection of articles that is being published at Eli's. Thanks so much for the rest of your day. And thanks to the editors cheers guys. Bye everybody. Bye.