 Hi there, and welcome back. I hope you enjoyed learning about GitFlow as a way to blend version control and documentation to help us to collaborate with ourselves in a more reproducible manner. I admit GitFlow is a bit of an advanced move, but if you feel like you've got Git pretty well under control, I really would encourage you to try to use GitFlow as part of your process. If nothing else, you could use it as a checklist to go through as you address comments that are made by collaborators or reviewers. I've also found that using it to outline the steps that I need to take to develop a project can also be pretty useful. I really like how we can use branches in the issue tracker to reflect the bush-like aspects of doing science. Too often we portray science as this linear process. We get to the end of a beautiful paper, but we don't see all the convoluted paths that we know happen along the way. At some point though, we'll want to move from collaborating with ourselves to collaborating with others. Today's lesson will discuss how we can use additional tools from our reproducible research toolbox to foster that collaboration with others. Baked into our discussion is an attempt to improve the reproducibility by improving the quality of our code. We'll discuss how we can use GitHub's collaborative tools to foster a view of our code by others and also solicit the contributions from others. Again, this is another advanced move that you may want to ease into. Join me now in opening the slides for today's tutorial, which you can find within the reproducible research tutorial series on the riffimonus.org website. So in the last lesson, we used GitFlow as a more systematic way for making changes to our project. Our first issue that we dealt with was simulating our PI noticing that our access labels on our NMDS ordination plot still had PCOA. Oh, later when we are looking at the output of our rendered main script document, we perhaps noticed that the figure legend said PCOA instead of NMDS. So as today's pop quiz, what I'd like you to do is use GitFlow to fix this typo. Go ahead and pause the video now and go work on it on your own. And then once you start the video again, I'll show you how I did it. I need to file an issue. So this is issue five. So we're going to go to our directory and we can do Git checkout dash B issue score 005 and we can open up our submission manuscript.RMD file. And I'm going to search for PCOA. So I'll do non-metric dimensional scaling ordination of Theta YC values relating the community structure, blah, blah, blah. Yep, that's great. And so we'll save back out, get status, add a change. We can do make.nwrite.paper. All right, make write.paper. And I'm going to go into FileZilla and log into my instance. And I just want to make sure that my PDF file looks right. And sure enough, if I see that non-metric dimensional scaling, blah, blah. Great. So that's all good. So I will go back to my terminal and I'll do get status, get add submission, manuscript.star, get status, get commit dash M. Correct. Figure legend closes number. Just want to double check that this is issue five. Yep, it's five. It closes number five. Get status. Great. Do get checkout master. Get merge master. Yep, sorry, get merge issue underscore zero five. That merges in. And I can get status. And I can now do get push. Very good. And I see that I've closed my issue and we're good to go. So hopefully that was a good review of what we did in the last tutorial. And again, as you see, we're slowly building to our toolbox, reusing a lot of what we've used previously, whether it's documentation, organization, version control. These are all great tools that we're building on to, again, improve the reproducibility of our projects. In today's tutorial, we're going to expand further. In today's tutorial, we're going to expand again to use GitFlow for working in a team. We're going to use ourselves, perhaps as our team for today, just to try things out or with an actual team. We're going to make a pull request. This is new jargon related to version control to incorporate new code. We're going to integrate an external review of our code into the GitFlow workflow. And then we're going to apply many of the principles that we've been working on throughout the series of these tutorials. We're going to continue to expand our skill set. So we have this annoying postdoc in the lab. And I think they have this great idea. Postdoc says, you're using ARS-based graphics. You should be using the tidyverse. You reply, eh, you know, I have a plot. It says what I want. It works. It's very functional. But it'd be really easy. You should totally change your code. You reply, make a pull request, and I'll think about it. So telling somebody to make a pull request is kind of a good way to get a conversation to end. It's kind of the way that you can bioinformatically drop the mic and get them to shut up and get them to do the work for you. So what is a pull request? Pull requests are a mechanism that allow others to contribute to your project, but that give you or someone you designate the power to accept or reject the contribution. The pull request can then be discussed between you and the contributor and allow the contributor to make changes before you finally accept or reject the contribution. So how do we think about this with GitFlow? Well, as a project gets bigger or more public, more people might want to contribute. And so we can delegate issues to different contributors. So remember, one of those steps was to take ownership of the issue so that others aren't going to be working on top of what you're doing. We don't want all contributors to have the ability to merge to the master branch. And so a pull request then gives you the authority to designate who can make those merges to the master branch of the project. Also, even on a project with a single developer, you may not want to be able to merge your changes to master without someone seeing your code first. And so some of the concepts that we'll be discussing before we're done with this is the idea of a fork, an upstream repository, a pull request, and code review. And so again, thinking about this as an extension of GitFlow, what we would do is to fork a repository into our own GitHub account, claim the issue from the primary copy of the project's repository. Alternatively, we will file an issue and ask the developer whether they'd be interested in a pull request on this issue. And so again, if it's a big project and it's not somebody you know, you might want to file a pull request. You might want to ask the original lead developer first whether or not they would even accept a pull request before you go do a bunch of work. Within your own repository, your own copy of the repository, you'll create a branch. You'll make the changes. You'll commit those changes. You'll then merge the upstream copy back into your local copy to resolve the differences. And then we'll push our copy to our GitHub account and then file a pull request to the original version of the repository. And then we can repeat these steps four through seven until the developers accept or reject your pull request. So again, this workflow has a lot of jargon in it that we're not familiar with yet. And so as we go through today's tutorial, these will hopefully make a lot more sense. So if you don't already have an account for your lab, go ahead and create one. So my group already has one that we use to support each of our manuscripts. And so that way, we have this organization Schloss lab. And so any manuscript that's written and has a repository, the final repository is stored within the Schloss lab account. If you're at an academic institution, you can tell GitHub that you're an academic or at a nonprofit. And they will give you unlimited free repositories. Just be sure you make them public when you finally submit your papers. So how do you sign up for a new organization? Well, let me go ahead and get out of the slides and go over to GitHub and show you how we do this. All right. So up here in the plus sign, there is an option for a new organization. Click on that. You can then click your organization name. And again, mine is Schloss lab. That's already taken because that's me. We can put our email address in here. So even if you're an academic and you're going to have a free account, you could put it in here. And so I could then say pschloss at blah, blah, blah, dot edu. And then we could choose our plan. So again, being an academic, click on the free option and then create your organization. So it's really that simple to create an organization. I would encourage you to talk with your PI, or if you're the PI, about setting up an organization and make sure that you have their go-ahead before you do this. But I would really encourage your lab to have its own organizational account. I always get nervous when I see manuscripts with repositories under the postdoc or student's name because that person may not stay in academia. And of course, the PI may not stay in academia either. But there's a greater chance the PI is going to persist in academia rather than a student or postdoc. So it's nice to have a kind of a corporate account to hold all of the projects from your research group. All right. So once you've done that, I'm going to go back to my Kozitch re-analysis repository. Find that. And I'm going to go to Settings. And what we're going to do is we're going to transfer the ownership of this repository to Schloss Lab or whatever lab you are in. And we're going to scroll down to the bottom to the Danger Zone. And we're going to transfer our ownership, click Transfer. And we need to type in the name of our repository, which is Kozitch re-analysis AEM-2013. And the username or organization that we want to transfer to Schloss Lab. So I understand transfer this repository. And I am going to click the green Transfer button to make that transfer happen. And so we're moving Schloss Lab, removing the repository to Schloss Lab Kozitch re-analysis AEM-2013. It might take a few minutes. So I'm going to go look for my Schloss Lab repository. You Schloss Lab. And so we see now that Kozitch re-analysis AEM-2013 is here. And I'm going to, similarly now, go back to my profile, look at my repositories, and now see that my Kozitch re-analysis is no longer here. So I'll go back, go back to my Schloss Lab version, and there we are. So what we've done is we've created an organization for our research group. Again, if you already had one, that's great. If not, well, now you do. We've transferred ownership from our personal account of this Kozitch re-analysis to ownership in Schloss Lab. And so it has been moved. It's no longer in our thing. And like I was saying before, I think it's a good idea to have the lab account hold on to the ownership of the repository. So this might seem like a bit much for projects, again, where you're the only researcher. But as I was saying earlier, it does create a structure where others in your group, that annoying post-election, the awesome PI, others can reinforce your code. And it also will help us to reinforce best practices for contributing to other projects. So what we're going to end up doing here is what's called a fork of this repository to make a new copy in our private account, our personal account. And so to make our own copy, we're going to do what's called a fork. And so you'll see this button in the upper right corner. We can click Fork. And I'm going to fork the repository to P Schloss. And so this takes a bit and it doesn't take too long. And eventually, it shows up. So now if we look in the upper left corner, we see we're in P Schloss, which is my personal account, in the repository Kozatree Analysis AM 2013. And it said that this is forked from Schloss Lab Kozatree Analysis AM 2013. So congratulations. You've forked your first repository. This is pretty cool. Again, what you have is a copy of the repository in the Schloss Lab project folder. And the copy now is in your own personal account. And so we're going to be able to make changes in our personal account. And we're going to hopefully then ask Schloss Lab to pull those changes into Schloss Lab. So we need to get a local copy onto AWS. So first, our local repository is already looking for the remote GitHub repository in our account. Because we had started this in P Schloss. We deleted it on GitHub. And then as we deleted it, we moved it to Schloss. But we still have that. And then we forked it back to P Schloss. But our AWS repository is still looking for it in P Schloss. So it doesn't know any of this has really happened. So that's cool. Alternatively, and probably the better way, would be to delete our Kozatree Analysis directory on AWS to clone it from P Schloss Kozats and then re-render the data. So we've done this a few times. We know it works. We know it takes probably about 45 minutes to an hour to run all this. I'm going to skip this for now. And for this project, we're going to use our local repository that's already looking for the remote. The remote is, again, GitHub. That we've got our local repository on AWS. Or on your laptop, perhaps. And our remote is what's being stored at GitHub. So what is the remote? As I said, we've been looking on our local version of the repository on AWS. The remote is what's on GitHub and is under your account now. We need a second remote, though, which we're going to call upstream that tells Git what the version upstream of the parent or the parent of our repository is. So we have a remote and then we kind of have a remote to the remote, if you will, that we'll call upstream. And the upstream repository, you can think of as like the official copy. And that's the copy in SchlossLab. We can look at the remotes by doing Git remote-v. We see that our remote is P Schloss Kozatree Analysis AM 2013. That's what we'd expect. And the name of that remote is Origin. So you might not remember this, but way back when, when we did our first push, we did Git push Origin. We were pushing to our Origin, which is our repository copy on GitHub under P Schloss. But we need to set our upstream remote. So to do that, we're going to do Git remote add. And we're going to say upstream as the name of our upstream remote. And we can go back to click on this link that we forked from. And we can get a copy, copy the address to this repository by clicking on the green clone or download button and then clicking on the copy button. Coming back to the terminal and pasting that in, hit Enter. And then we can do Git remote-v. And we see now that we have our Origins and our Upstreams. So one of these is to fetch, which is kind of to pull stuff down from GitHub. And the other is to push, which is to put things up. And we have a fetch and a push both for Origin as well as for Upstream. So you'll notice that we have a fetch and a push for our Origin and our Remote. And so we've talked about push already, where we take what's in our current repository and we push it up to GitHub. Fetch brings down the commits from the remote branch that are not in our current branch and stores them locally. Merge then will bring the stuff we've just fetched. And it merges it into the branch that we're on. So we can do that in one step with a pull. So when we do Git pull, we're both fetching and merging at the same time. And so that's why we see fetch rather than pull because it's breaking it down into those two steps. If you're not careful as you're doing your pulls, you might run into conflicts. And so that's why sometimes people prefer to do a Git fetch and then a Git merge rather than a Git pull. And so when we're working with our Upstream, we will do Git fetch, Git merge. So what does this look like in practice? We can do Git pull origin, which is already up to date. And then we can do Git fetch upstream. So Git pull origin is pulling it down from pschloss. Git fetch upstream is pulling it down from schloss lab. And so this now tells us that there are two branches, issue four and master, that it's brought down. And we can then do Git merge upstream forward slash master. So this is telling Git that we want to merge the master branch from upstream, which is right here, into the master branch that we're on in our thing. But before I do that, let me just make sure which branch I'm on because I don't know that I moved back. Yep, I did. So we are on master. So that's a good thing to always double check. So I'm going to do Git merge upstream forward slash master. And of course, it says we're already up to date. You could imagine, though, that if we had forked it and maybe sat on it for a day or two and someone else had pushed something into the schloss lab version, that our version on our repository might be different than the upstream version. And so it's good to always merge our origin version back into our local version to make sure that we're working on the most recent version of the code. So what we'd like to do now is to create an issue in GitHub in the schloss lab, the upstream version of the repository. So we'll go issues, new issue. And we will say convert NMDS plot to use tidyverse packages. And I'll leave a comment to say the postdoc thinks we should convert plot NMDS.R to from using base R functions to instead use functions from the tidyverse packages deplier and ggplot2. And so we might submit that issue. And we might let it sit. And we go away and we think about it some more. And we say, if we were to use deplier and ggplot, there's a lot of code in that plot NMDS that's just kind of doing funky things. If you look at that code, there's all sorts of little things in there that the more I think about it, I just don't like. And that it would be good to refactor that code to make it a bit more elegant, a bit easier to maintain. So I'm going to take on this issue. So I'm going to assign it to myself. And I'm going to add a comment. I'm going to say, the more I think about it, I agree with the postdoc. I agree because I think it will help to make the code easier to maintain. And so I'll comment here. And again, we can use assignees. We can use labels. We can add these to different things. We can add people on to the discussion to get their feedback. We can have a dialogue here. All right, so I have claimed this issue for myself. I'm now going to go back to AWS. And I'm going to create a branch in my local repository for the issue. I will do get status. Just to make sure I'm on the right branch. I'm on branch master. Can you get check out dash B? And I will then call it issue005. Double check that's the issue. Oh, nope, I'm sorry. It's issue6. Get status. We're on issue6. And now we want to edit our code. So we can then do nano code forward slash plot and mds.r. And this is our R file. And again, because this isn't an R tutorial, I'm going to do some copying and pasting from the slides. I apologize for that. Again, it's in the slide deck if you want to take a look at it in greater detail. But we're going to use dplyr and ggplot with our mutate commands and ggplot to build the plot. The inputs are the same. The output is still that ggsave. The function name is still plot nmds. So the dependencies and the output are the same. So I will save this and quit. Get status. Make dash n write dot paper. That looks right. So we'll do make write dot paper. And I will now go into filezilla. And I will look at my manuscript.pdf to make sure that all looks right. And we see our plot, our ordination for the ordination, using tidyverse, using the ggplot. And so we see that. And so that all looks good. Maybe our legend could be shifted over a bit. Maybe I'll go ahead and do that. Let's see. Where do I have that? So legend position 0.15, 0.15 is, I think that's right there. So maybe I'll put it to 0.5 on the x-axis. Make write dot paper, open this up again. I think what it's doing is it's putting it halfway across, not at the actual x-cordant. So I will edit that again to maybe make it 0.75. And make write dot paper one more time, and we should be good. Great. So our legend's out of the way now. So that looks great. I think we're ready to ship this back and commit it and make our pull request. Get status. I'm going to move this up. For some reason, it's made rplots.pdf. I don't want that, so I'm going to do rmrplots. This is not being tracked, so I don't need the get rm. So it's not tracked, so we can delete it. Get status. Get add code, plots, results, figures, mds, submission, manuscript PDF. Get status. Great. Get commit dash m, refactor code to use tidy versus addresses number six. Get status. Great. So we're on our branch issue six. We've made our change. We've made our commit. We now want to push our branch. And so we saw this yesterday that we can do get push. But before we push, we want to double check that nothing's changed on the upstream repository. So I'm going to do get fetch. Get merge upstream forward slash master. And we see we're already up to date. So good. So now we can push. So we do get push origin. So we're going to push to our version and our account under pshloss or under your version, wherever that is. Origin issue underscore 006. You may have to enter your credentials, I don't know. And so now we see that it's pushed it up. There's new branch. We come back to me and go to my profile and my repositories. And I see that I recently pushed up a branch issue six. And so I would like to compare and make a pull request. And so this creates a form to open a pull request that is going to use as our base fork. What we're comparing it to, what we're trying to, we're requesting that Schloss lab pull our version from pshloss. And so it says able to merge. These branches can be automatically merged. Because we did that get fetch, get merge upstream. And so I can now leave a message in here. And I can say, I've refactored the code to use dplyr and ggplot to functions. I feel better about the ability to maintain this code. I think I like this version a bit more than the original using the base graphics functions. So we've entered a comment to the maintainer of the Schloss lab version. I can then click create pull request. And thanks for a minute. It sees that there was a comment there that I added a commit that's here. And so it then says that this pull request could be merged. So I'm the owner of Schloss lab, which is why we see it gives me this nice green button. So don't push on that yet. So what we're going to do is we're going to ask someone to critique it. And so I'm going to again simulate this by saying, thanks for the contribution. This looks great. Before we accept the PR pull request, could you indicate in the readme file that the user needs to have dplyr and ggplot to packages, I'll say the installed before running the code? So we're going to push this back to pschloss. And I'll say comment. So we've gotten this feedback. And now what we want to do is we want to go back into our cosetree analysis. I'll do get status to make sure I'm still on issue 6 as a branch. And I will open up readme.md. And I will scroll down here where I had dependencies and locations. And I will add dplyr and ggplot2. And that's saved. And come back and do get status. It's been modified. Get add readme. Get commit-m. I'll say add dependencies to readme. Addresses number 6. And then I will, again, do my get fetch upstream. Get merge upstream forward slash master. It's all good. And so now we can do get push origin issue 006. It pushes that up. And now we come back to our pull request. And then we see added dependencies to readme addresses issue 6. And so we've got that commit is automatically added into our pull request. So then the developer, the owner of the repository, writes back, these changes look great. Thanks for your contribution. I'll do fireworks. Bam. And so then we will comment. We will then merge the pull request. And we will confirm the merge. And so our pull request has successfully been merged and closed. You're all set. The pshloss issue 6 branch can be safely deleted. And so if I click delete branch, it deletes that branch. And we can then come back to our issue. And we can then say close issue. Great. And so it adds to the transcript of this issue that the issue was addressed by the merge pull request from pshloss issue 6. So we want to update our local repository. And we will get back to master. So we'll do get checkout master. And we can then do get fetch upstream, get merge upstream master. And that's been brought in. Again, if we do get status, we see that our branch is ahead of origin master by three commits. So our personal version of the repository is three commits behind of the upstream as well as our local. So we want to fix that by doing get push. And we can also, reminding our branches, get branch, we could do get branch dash d issue 0, 0, 6, or we could keep it or whatever. So I'm going to go ahead and log out now. And we'll do exit, exit. And I'm going to stop my instance. So GitHub allows us to set guidelines to potential contributors for how they make those contributions. So we can state explicitly our community and behavioral expectations that there's no name calling, no snark, no being jerks. We can provide links to external documentation, mailing lists, or a code of conduct in terms of how we set those expectations. We can also, beyond expectations of behavior, we can describe what our expectations are for creating good issues or pull requests. And so Mozilla at this link has a great checklist for you to think about. So for example, a pull request should really focus on one thing. And so our maximum submission should be a maximum length of perhaps 400 lines of code, maximum function lengths of about 50 lines. Our code must be able to be automatically merged in, as we saw. And we were able to achieve that by doing the get fetch, get merge. If there had been a problem with merging, we would have had to resolve those merges before we then push to our origin and then did the pull request to the upstream repository. If you have tests, kind of like our make write dot paper, they must all pass. Functions should be tested. Functions should be documented. They should respect a style guide and indicate what issue the contribution addresses. And then finally, in the pull request, to summarize and annotate the changes. So we can stipulate these requirements with these steps for making good issues and good pull requests. When we look at our issue tracker, we'll see a link either up here or down to the side for how to make good contributions to the repository. Again, if we go into GitHub and say I want to file a new issue, you may notice that down here in the lower right corner, there's a link for helpful resources. And so here you see all those things that I had just mentioned. And so where does this come from? Well, this is coming from our contributing.md file. Another good resource that I have in the slide notes, as you'll see, is a link to the software carpentry page for their contributing.md file for what they expect from people to make contributions to their repositories. A big part of making these pull requests is having someone else look at our code. Just like we simulated someone saying, hey, it'd be really nice if the read me included dplyr and ggplot2 as packages that need to be installed. We need others to look over our code and make sure that we've dotted all the i's and crossed all the t's. You need to get over the quality of your code. That's the purpose of having a review. If you don't want people to look at your code, that's a bigger problem. And so we need to get used to people looking at our code, making suggestions, and that's really the only way we're going to get better. I know that my writing of narrative has gotten better because of peer reviewers looking at manuscripts I've submitted. And I know having people look at my code will only make me a better programmer. Think about what you want to get out of your code review. What is your goal? Do you want to make sure that you're writing code that people understand, that works, that is readable, all of the above? So think about that. And think about that as you ask someone to review your code. You can solicit reviews from your PI, collaborators, colleagues, random strangers, people on Twitter, wherever. Do it early and often to prevent code backlog to get the best feedback. So really, there have been studies that have shown that you've got to keep it under 500 lines of code, or it just gets to be too much for somebody to look at. And if you have to give someone 1,000 lines of code, then that might indicate that there's probably a problem with your code, that it needs to be broken up into more modular segments. And also just the psychology of getting this reams of paper or reams of code is not going to get a lot of enthusiasm from potential reviewers. So your review can be live, like in a lab meeting. So in our lab meetings periodically, we'll have somebody put a few lines, maybe 150, maybe 50 lines of code on the projection, and we'll go through it line by line and comment on the code. It could also be asynchronous. So it's perhaps be more like peer review for a paper where you ask someone to look at your repository and perhaps a specific file in your repository and ask them to review that code for you. If you're reviewing someone else's contribution, Mozilla has a really nice guide for thinking about code review. And so these are some of the questions that were taken from that guide. And so they break it down into intrinsic examination as well as extrinsic examination. So are the functions as simple as possible? Is the code efficient? Is the usage of each function clear? Is there documentation on how you use that? Have edge cases been considered? So kind of like things where, you know, dividing by zero, I think we talked about that when calculating Shannon diversity when we were doing our coding. Have those edge cases been considered in the function? Extrinsic examination, does the new code reinvent any wheels? Are there packages that could have been used instead that are already pretty well developed and tested and robust? Does the new code successfully address the needs of the project? And does the new code respect the structure of the project? Right, these pull requests should not be totally revamping the whole code base of a project. That does not respect the structure of the project, but it should instead work within what the goals and the structure of the existing project. I'd like to leave you with a few questions to think about. So based on what you've learned in this series of tutorials, why should the quote final version of your repository be in your group's organization and not just your own private account? I think there's a few reasons for that. Look around the GitHub version of your lab's copy of the repository. How would you add someone from your lab to review your code? So noodle around on those pages and see if you can figure out how you would add someone to be an approved manager or somebody that can approve pull requests to your lab's code. And finally, ask yourself, who could provide a review of your code? Are there people within your local community, within your lab, or that you know through social media or through meeting people at conferences that would provide a review of your code? The final thing I want to leave you with is a opportunity to get a quote virtual badge for completing these tutorials. And so on the home page of the reproducible research tutorial series, you'll notice that there's an honor roll. So you and your picture and your information can be added to the honor roll as an indicator that you completed this training by filing a pull request. And so to do this, you need to fork of this repository that has all these slides for this tutorial series. You're gonna, like as we did in this tutorial, create a local copy of the repository. You're gonna add a file that's called, that has your GitHub ID, whatever it is, so minus pshloss.yml. There's a copy or a template of what you need to use in the directory honor roll that's called template.yml. And so you're gonna add that file to honor roll and you're gonna complete the needed information. You're gonna add your picture and the picture needs to be 300 pixels by 300 pixels. And that needs to be named, again, your GitHub ID.jpg. And this needs to be also added to the honor roll directory. You're then going to add the changes to your copy of the repository, create a pull request and then we might go through an iteration or two to make sure everything's in row and then we will then merge your pull request into the honor roll page and that will then get your image to show up on the honor roll and will allow you to say that you've completed this training in reproducible research practices. I really hope that you take me up on the offer of filing a pull request to receive a badge for completing the materials within this tutorial series. This activity really serves as a capstone for the entire series, but hang on for one more tutorial after today's to finish the series. Have you ever had anyone review your code? We were used to having our science reviewed in committee meetings, seminars, posters, papers and grant proposals were reviewed constantly but have you ever asked anyone to take a look at your code? Have you ever looked at someone else's code? My research group does this on a regular basis as part of our lab meetings. And while we still could do a lot more with it, it's really been helpful to get people to learn to code better and to identify potential problems. Some groups have varying policies on who can accept pull requests and what steps have to be taken to approve those pull requests. At the minimum, people will push directly to the master branch without code review. Others require that the contributor cannot accept the pull request themselves. Still others require one or two or more people review the code before accepting that pull request. These requirements can get kind of onerous if you're the only one working on your project or if no one else in your group is familiar with programming. If this level of rigor interests you though, I'd really encourage you to use social media or your network of friends to reach out and ask someone to review your code to help you out. Similarly, you can also make the same offer to review others people's code. I think you'll learn a lot by looking at others people's code and comparing it to your practices as well. I have to admit that I use the tools we've been talking about in this series long before I started thinking about code review and branches and pull requests. Now I think of a fork as a bit of a firewall between what seems like a good idea to me and what is truly a good idea for the project. You can think of it as an added level of security for ensuring the replicability of your work. In the next tutorial, we'll finish this series by discussing the value of openness and transparency and reproducibility. We'll talk to you soon.