 As much as we all like to complain about peer review and reviewers and where the dark places that they have their heads shoved up because they just don't know what they're doing, or did they actually read the manuscript? Well, peer review is actually a pretty critical part in my mind of conducting reproducible research and getting the most robust, readable, and professional document out into the peer reviewed literature. If you recall, and if you've been watching previous episodes, you know that over the winter months, summer to winter, I actually wrote a manuscript here on YouTube, sent it off for review at mSphere, and the reviews came back. I actually haven't looked at the reviews, because I wanted to do that with you all. So I'm a little bit nervous about what they say. So in today's episode, we'll look at the reviews, and I'll tell you my process for how to deal with peer review and respond to those reviewers comments. Hey, folks, I'm Pat Schloss and this is Code Club. Yeah, I really have been sitting on these reviews for far too long. The reviews came back maybe about five to six weeks after I submitted the manuscript, and they've been sitting in my email, and I actually haven't opened them to read through them because I wanted you to get my first reaction so that you could see, you know, an honest assessment of the reviews as I see it. I have had over 110, 120 papers go through peer review. I've been through this many, many, many times. I've been doing it for the last, I don't know, 20, 25 years. And to tell you the truth, this is the most nerve wracking part of my job is getting reviews back or getting feedback back from people. And you'd think that after this time, I'd be used to it because, well, for a couple reasons. So first of all, we've had a really good success rate in getting papers published. I have great confidence that if I write a manuscript, I know I can get it published somewhere. So in some ways, like sticks and stones may break my bones, right? That the reviews could come back and be totally off the mark. But I'm still going to get the thing published, right? I just have that confidence, right? And at the same time, you might say sticks and stones are idiot reviewers, what are they thinking? Did they read the paper? In general, I would say that they've all made the papers far better. And I have actually learned a lot about writing through the lens of my reviewers. The other reason that I wonder why I'm so nervous at getting peer review is because I know it's coming. And I know that they're going to have something to say, right? That's why we submit it for peer review. I've only ever had one paper that was accepted without further revisions. That was the very first paper I submitted, believe it or not. And actually now after all these experiences of peer review, I have a pretty negative view of that journal because did they not see the holes or the flaws in that paper, right? Anyway, I'm sure it was an okay paper and I think my research has gotten better. And again, as it gets better, the comments keep coming. And again, this is part of the job. And as nerve racking as it can be to see an email pop up in your inbox, it's part of the job, right? So here I am in my inbox. I've got my two emails. Don't worry, they haven't been sitting in my inbox forever. But they kind of scroll them away into another directory. So I don't have to look at them. I'm somebody that when I look at my inbox, if I see stuff in here, it just it makes me nervous. So anyway, I've been hiding it away. So I don't have to think about it as I'm doing other things. So a couple of things that we're going to do as we open up these emails and read through them. First of all, we're not going to get mad, right? Right, right? You're not going to get mad for me. And I'm not going to get mad myself. I'm going to assume the best of intentions for my reviewers. I am going to assume that they want to help me make my paper better. They have a vested interest now and making sure that anything that goes out into the scientific literature is robust. What I'm going to do then is I'm going to read through the email. I'm going to take a deep breath, not going to get mad. And then I'm going to set it aside for a few days, right? And so I'm just going to allow myself to chill out. Maybe a few things will stick out to me and I'll kind of mow them over in my head. And then I'll come back and start working through the review. Okay? So that's the deal. That's what we're going to do. So let me start with the first email. So I realized that I forgot to remind you all what this paper was that we worked through here on YouTube for a few months. It was looking at the propensity of Amplicon sequence branch specifically, but also O2U definitions in general, the propensity of them to split taxonomic groups into multiple bins or to pool taxonomic groups or for the propensity to take an individual genome and split the six sequences in an individual genome into multiple bins, right? Whether that's an Amplicon sequence variant ASV or an operational taxonomic unit O2U. It was a short form paper. So it really wasn't that long of a paper. And that's again what we were looking at. So you can see here the email that came back to me from the editor. Thanks for submitting your manuscript to mSphere. We've completed our view and I'm pleased to inform you that in principle, we expect to accept it for publication. Yes! However, acceptance will not be final until you've adequately addressed the reviewer's comments. What a relief, right? So something to know is that the editors and journals in general are very reluctant to tell you anything about the final decision of your paper. Unless it's accepted, or in this case accepted in principle, or if it's rejected, right? If it's in between, then they kind of hedge, right? And so I have seen some decision letters that come back and they look pretty negative. And so people think, oh, I got rejected. And then I read over it and like, they didn't say it's rejected. They didn't say that they don't want to see it again. They just said that there's like substantial comments to respond to. So get on that, right? All right. So then the editor said, I had trouble obtaining a second review, but in the interest of time, I decided to just go with the one. They are one of my favorite reviewers and I trust their judgment a lot. You'll enjoy reading the review. So this editor is doing their job, right? They got the comments. They are a professional in the field. They know the field themselves. And they said, you know, this reviewer is right on. I'm okay with getting one decision. So as we see from my inbox, they actually got a second review. So this will be pretty good. So the reviewer won comments. Hardly ever does a reviewer read a manuscript or he finds himself muttering, told you so. And just as I always said, again and again, oh, this feels good. For me, this is such a manuscript, right? Dr. Schloss makes a fine case by Emperor's new ASV are actually no improvement over the shoddy O2. My crudely ecologists have been stitching together from partial six sequences, right? So the second paragraph they had was that one important aspect that has not been mentioned are the errors introduced by PCR amplification sequencing. And that's right on because I got the data from the ribosomal RNA number database, RNDB, right, which got their data from assembled genomes. And so we're trusting the sequences to be perfect and to not have any errors in them. Sorry, I'm still excited I got this accepted in principle. And so they're making this as a point that might be worth adding, right? And so that the variability that I'm seeing here is real, right? But when you're doing this in the field with actual sequence data, there's still going to be some residual sequencing errors. And so that might be something that we might want to add into I think that discussion paragraph, the last paragraph of the paper to kind of highlight that again, this is a best case scenario. So generally, when you get a review back, the first couple paragraphs are kind of the big picture impression that the reviewer had of your paper. And, you know, if there's major concerns, they'll generally be in here, right? So I would take this, this seems to be a pretty positive review. So I would take this second paragraph here, this one important aspect as being kind of their major concern if they had any concerns at all, right? And so this would be something that I would need to be sure to address that if they ask for something here, then I probably need to kind of bend over backward to address this. They then have this series of minor points that generally aren't critical to a decision. But that should be addressed in the revised version of the manuscript, right? And so, you know, things about wording and that I kind of, you know, maybe gave the wrong impression or they read it one way or it was confusing, things like that. And, yeah, so I think other comments through here as well, things about like the Y axis legend and so forth, right? So there's some some things in here that they encourage me to think about and to address in the revised manuscript. So all in all, this is a very positive review. It's also a short review. So I'm grateful for that. But for kind of demonstrating things to you and what reviews generally look like, this is a pretty abbreviated review. Let's go look at the second review and see if the same thing holds there. So the second review came in, just received another review from manuscripts, please see the comments below. Again, this first paragraph is kind of a summary of their overview of the paper. And then they have this second paragraph that is perhaps their major comments. So let me look through this here. Again, I haven't read this before. So it's a provocative topic of ASVs versus O2s as well written to the point informative. Thank you. Thank you. Thank you to whoever reviewer number two is. Thank you. So this looks all good. Let me just kind of look at this. Again, they're summarizing what's going on, the contribution of high impact as the data scripts are available via GitHub and the conception analysis on YouTube. Good. They watched. They noticed. It'll be a superb teaching tool in the lab in addition to the findings presented in the manuscript. Thank you. My only comment for improvement is that there could be more emphasis on why the risk of splitting genomes using ASVs is of greater concern than clustering multiple species into the same OTU. So again, this is a great point. And so that is their kind of major concern paragraph here. So accepting that genomes are artificially split and this reflects in inflated cluster numbers and diversity, who cares, right? You know, what could be better emphasized or communicated is why this makes zero ecological sense and how it can impact interpretations, right? From the devil's advocate perspective and ignoring biology, doesn't matter if diversity estimates are inflated, if everything is relative. So I feel like both of these comments, this reviewer's comment of like, can I strengthen the argument for like why we should care that we're inflating numbers, right? So that's good. The other comment then from reviewer one, as I'll call them, of dealing with the fact that we are looking at, in a way, artificial sequences because they're perfect and they don't have sequencing or PCR errors and how would we address that in the rest in the manuscript to kind of help solidify those arguments. So again, I'm seeing this for the first time. And so I'm kind of excited about this as it probably comes through, I hope. And so again, what I'm going to do is I'm going to set this to the side. And I'm going to think about these topics. And again, these are very positive reviews, I'll say, I'm grateful to that. But at the same time, I'm disappointed because I'd like to share with you kind of what negative reviews look like. And so I think if you ask around, you will find that these are actually very positive reviews. And so thank you, thank you, thank you to the editor and the reviewers for getting me these. And I feel really stupid now that I've waited like two months to get the to be looking at this and to be working through the reviewers comments. So having set the reviews to the side for a bit to kind of calm down if needed. And to kind of mull over some of the ideas in your head. There's a couple of things that we need to think about as we proceed in responding to the reviews. So first of all, the thing I didn't look at at the end of this initial decision letter email were the comments from staff. So they kind of tell me where I should go to resubmit the revised version that I will need to provide a point by point response to the issues raised by the reviewers in a document called response to reviewers. I will also need a copy of the manuscript that is a marked up manuscript that you know, you might think of using like track changes with specific page and line numbers where visions have been made in the marked up manuscript, right? So if I add a sentence or two to the manuscript in my response to the reviewers, I'll then want to say as you can see online, blah, blah, blah. And then, you know, that's where we address this comment, right? Okay. Also, they have an important paragraph here, which is the deadline, which I think I've probably gone after gone past, that they have a typical 60 day deadline at ASM journals. So a couple things about that. So again, they say that because of the pandemic, don't worry about it. Don't stress out about the deadline. They hope that we could resubmit it soon. But you know, I'm a busy PI with family pandemic lab, other things I'd rather make videos on visualizing microbiome data than revising or rebutting manuscripts, you know, I can procrastinate too. Anyway, you typically get 60 days. So what are you doing in that 60 days? Well, hopefully, you're reading through this, you're perhaps having to do additional experiments, and perhaps rewriting or reworking some of the analysis or the text. They have made the assessment that it should only take me 60 days or less to respond to these comments, right? And so if it's going to take longer than 60 days in kind of non pandemic times, then you are totally a okay to email the editor and say, Hey, editor, would it be okay if we had an extra two weeks or an extra month, right? They just want to know that like, things aren't going to drag on forever. And so the deadline, the 60 day deadline that they give you in my experience has not been written in stone, we've had things where we've needed like another a week or two. And they've usually been cool with that, because they're excited to get something back, because they've already invested a lot of effort in doing the peer review and reaching a decision. So keep that in mind. And that was kind of a bad thing of me to sit on these reviews and not look at them because whatever. But know that you generally do have a bit of a deadline, but it's not like set in stone, you're more than welcome to ask the editor for a little bit of leeway there. I've gone ahead and created a blank word document that I'm calling response to reviewers. I could do this in markdown, like I've done everything else for this project. But I'll do this in Word, because sometimes that's just easier. And I suspect if you're watching this video on how to respond to reviewers that you're probably doing most of the things in Word anyway, if I come back to my reviews, what I'm going to do is go ahead and highlight and copy the comments directly from the email into my Word document. I'll grab the review from the second reviewer and plop that down into my response to reviewers document as well. And I will make this reviewer one. And I will then make this reviewer two. And again, what I'm trying to do is create a document where I can respond blow by blow to all of the comments from the reviewers. So I'm going to then go ahead and bold all of the text from the reviewers, so that I can come in and go ahead and add my own comments to these reviews. Now, in general, as I mentioned, there are two types of comments that you'll get from the reviewers. The first are more substantive. So things like, you know, one important aspect that has not been mentioned are errors induced by PCR and sequencing, right? So that's more of a major comment. The other comment from reviewer two was that, you know, who cares, right? Add some text to indicate why we should care about the inflated number of clusters. So those are things that are honestly going to take me a bit more time to respond to, right? There's other things in here, these minor points from reviewer one that are fairly easy to tick off. So sometimes it's rewording a sentence or, you know, like this, I wholeheartedly agree with this statement, right? Like, I don't really need to address anything in the text. So what I might do here is say something like, we are grateful or I am because I'm the only one writing it, right? I am grateful for the reviewer supporting this assertion, right? And so what I'll do is indent it, you know, a tab space and make it plain text. Now, if I have to write multiple lines for my response, then I will indent the left side of that whole paragraph over. So it's easier for the editor and reviewers to see what I've said. So the life cycle of this is that the editor for sure will see my responses as well as my marked up manuscript. They will generally make a decision whether or not to send it back out to the reviewers for the reviewers to give it another go. And so I want it to be easy for the editor and reviewers to see what I did and how I responded to their work. Now, this being basically accepted or condition accepted on in principle, it's unlikely to go back out for re review. But I want the editor to see kind of what I said and how I responded to the comments. So I'm going to go ahead through the rest of these comments, addressing them one by one. I will do that in my are markdown document to generate a new document. And I will then show you what my response document looks like, as well as how we then go about making that track changes a version of the document. So I've been going through and editing my manuscript and addressing the reviewers comments in parallel, I use the reviewers comments to kind of, again, go through the manuscript and identify those places where I can clarify the text or add information that they are looking for. As you look through my comments, I do try to be very gracious to the reviewer and grateful to them. And again, these were very positive reviews. So I have no problems with the reviewers. And that's I try to be gracious, even if the reviewers are idiots, right, even if they're saying things that just make no sense and are totally off base. I know people have horror stories about reviewers. And, you know, maybe I've had horror stories, but like I said, in general, they're, they're, they're doing their best to be constructive. And this is not easy work. It is totally, totally underappreciated, right? So anyway, be nice. Don't be a jerk. If they're a jerk to you, or you think they're being a jerk to you, or they're being kind of short drifted, it gets you nothing, it gets you nothing to be a jerk back to them, right? You're just going to piss them off all the more. And that's not going to help you get your paper accepted, right? Remember that the reviewers, very likely, will see your responses. And so if you're kind of a jerk back to the reviewer or dismissive of the reviewer, you're just going to make them mad, right? I have had that happen to me where I'm a reviewer and I get responses back. And the authors are just like jerks, right? And that just, that makes me mad, right? And as much as I try to not be mad as I'm reviewing, it just makes me mad. You do not want mad reviewers. That's a that's a pro tip for Pat. I'll give that one to you for free. Anyway, again, some of these comments were seeking clarification. And one of them was like, where had the references or citations the wrong sentence? And just kind of things where they, you know, they said they agree with me, or just kind of saying, you know, this bears out with my experience. And I think that was pretty helpful. So I didn't have any situations really, where I said no to the reviewer. There was one case where the reviewer once said it seems weird. I don't know where it was in here, but it seems weird to have the first figure mentioned to be a supplemental figure. And I was like, yeah, I get you, you know, but for this format, I only get two figures. And I felt like that first figure kind of states an obvious fact, as you increase the number of genomes per species, then the number of ASVs will also go up. And so yeah, I think that's an important point. That's why I made it a figure. That's why I made it a supplement. But you know, if I only get two figures, and I've already got two figures that I think are pretty important. I think I'm going to put that in the supplement. Maybe I could make figures two and three, a single two paneled figure. But I'm not really excited to do that. I think I think they stand on their own. And while I think the supplemental figure one is important, I don't think it's so important that I should, you know, rearrange the other figures. So I'm pretty happy with that. And again, I think that the comments and suggestions the reviewers made really did help. And they've asked me to add a little bit of text. That's always a challenge when reviewers ask you to add text, especially in a format like an observation, where they try to limit the author to the total number of words that you have in a manuscript. So if they're encouraging you to add text, but not telling you what text to remove. Well, I figured that kind of be carte blanche to add text and to kind of go over the word limit. So I think we'll be in good shape. What you'll notice in here also is that I have highlighted to myself the points where I need to go back and add the line numbers. So the line numbers I'm going to put in here will be the line number in the marked up document. And so I will add something to this response to reviewers document to indicate that the line numbers refer to the marked up version of the manuscript. So I've been making all my edits in my are marked down document. My are marked down document is under version control, which means that I have the version that I submitted stored away in git, which I can easily get back to compare to this version. Again, I made all the comments here. One thing that I need to keep in mind when I submit the manuscript is that I'll need to submit a doc x version of the final manuscript, along with that marked up version. The doc x version should not include the figures. So I'm going to go ahead and remove the figures. When I upload the information or the files, I will upload the figures as separate files. I will go ahead and make submission manuscript dot tech. This will generate also the doc x and PDF versions of the manuscript takes a minute or two to run through and generate this. So again, this generated all of the files. You'll see that I have this PDF tech and doc x file. These are the latest versions of the manuscript in my make file. And again, I used make to keep track of the entire pipeline. You can go back, look at other episodes in this playlist for the paper to see how I've used make and how I've used to get at the bottom here. I've inserted a rule. I'm not going to go through and kind of recreate this rule for you. I copied it and pasted it from someone's website online. The idea is that again, the dependency to this is my manuscript dot tech file. The output is going to be submission changes track changes that PDF. I again have everything under get. And so if I do get log, I see all the previous commits the commit history of this project. And the last commit that I used to modify that file, I can then plug that get commit ID into my get cat file. And what get cat file does is it outputs submission manuscript tech from that commit. And then it outputs it as submission manuscript old tech, I can then use latex diff on that old version as well as the current version of tech to get a track changes dot tech file, I can then use PDF latex to then take that track changes tech file and output it as a PDF into my submission directory. And then I do some cleanup. So I'll go ahead and copy this and I will do make submission track changes dot PDF. This runs, I can then do open submission track changes. And so this is a PDF version of my track changes document. And nothing changed in my abstract or the importance section. And you can then see in here the red and the blue the red for things that were moved the blue for things that were added. Again, some things were moved around. You can see not a ton of changes. There were a couple sentences that I added for those kind of major comments from the two reviewers. And again, we can come down here and see I moved around some of the references. And I removed those figures. So I have a couple other things that I need to do. So I will give this another couple read throughs to make sure that everything looks good. Again, it's been a couple months since I looked at the manuscript. And so some other things might stick out to me that the reviewers didn't detect. It's short enough. And I went over enough times previously. I think it's probably in pretty good shape. So if there's no other changes, then I'll take this version as my marked up version with the track changes. I'll also take the doc x version. So if I do open submission, manuscript dot doc x, this is kind of the default formatting that comes to us as a markdown document. And there are ways to make a doc x file look a little bit nicer. I don't find them very satisfying. So I will come through here and manually edit everything to make it look nice enough. Basically, I'll make everything double spaced, I'll add line numbers, I'll add page numbers. And that's basically it, right? Otherwise, this is the doc x version that I will upload, and that they will then use to make into the manuscript that is eventually published. So I'm pretty happy for the most part with the way this looks. Again, I just want to give it one or two more looks to make sure everything looks good. I'll also take my cover letter and update that to make sure that that all looks good. And then we'll upload it. And hopefully, in a couple days, the editor will get back to us and say, why do you wait so long? Congratulations. This paper has been accepted. And then we'll see it shortly thereafter, as a manuscript published in mSphere. I'm optimistic that this will be the end of the story for getting this analysis published. If you have questions about the writing process, by all means, let me know down below in the comments. I'd be happy to, you know, take on some other questions that people have about the writing and scientific publishing process, because I know sometimes it's a little bit opaque to people who are just getting going in science. I know that when I have people in my lab that go through this process of looking at reviews, it's pretty scary if it's scary to me. And I've been doing this for 20, 25 years. It's certainly scary if it's your first time around. And there's just things that are obvious to me, perhaps, or people like me that have been through this a few times, that aren't so obvious to people that are just getting going. So by all means, ask questions down below. And maybe we'll do a final celebratory video when we when we know the final verdict on the manuscript. Anyway, let me know what you think, and we'll be talking to you soon.