 Thanks for sticking around for the last one or coming back over here Talking about finding a home for the docs and and it is actually running live right now from the internet I was thinking I was gonna have to do this offline So if you go to that address you can actually have your slides like advanced if you actually had an internet connection So who am I well, I had a bit of an introduction, but I am documentation engineer at netlify which means that I Am in charge of the docs for all of our projects that we work on including our own docs for our service, which is There's like these fancy words we're not supposed to say but Basically a static site host with a whole bunch of other fancy features attached to it Which is good for documentation And in addition to our own docs I also manage ones for our open source projects including netlify CMS which is a Git-based Content management system for well system is kind of a big word for it. It's a wrapper for git functions That you can use to edit your content in git repos And then also our other open source projects We have several microservice apis that happen to be written in go that provide all sorts of Dynamic services to your static sites like authentication E-commerce and commenting But because we're also good for hosting docs we actually happen to host a whole bunch of open source documentation projects and so Because of that as someone who provides support to all these people who write docs I kind of have a front row seat on how a whole bunch of different docs are made So what are we talking about when I talk about finding a home for the docs? I'm talking specifically about where to sort the source of your docs So not where you go to find them when you go online to see the docs But actually the source of them if we're managing docs like code Where is that source code for the documentation? So and we're talking primarily about open source projects Mostly written in markdown and Jason Yamal and Tom all the this could apply to other things and it could apply to Not only static site generated sites, but also sites using like that fancy tool that kitty was talking about that puts it into Drupal But generally these are managed with git and stored in github they obviously could be stored at other good hosts, but Most of the ones I'm encountering are through there. So that's what I'm showing Now why does it matter where you store your docs? There's a couple things to keep in mind Basically, you want to be optimizing for all the parties involved in your project. So Your users you want them to be able to find your documentation and be able to use it and have it readable and accessible You also want your contributors to be able to find where they can make contributions and edits to the to the docs You also want to be something that your maintainers can maintain those docs and keep them up to date and handle contributions and Manage all of their repo and between code and docs so What are the options for where they're stored? There's a few that I'm going to be looking at The first one is Docs stored in a repo with the project code So you're not just in the same place where the code of the project itself the docs are in that same repository Another possibility is that the docs are in their own repo separate from the project code And then finally there are some hybrid solutions that kind of mix it up So we're going to talk about those three And we're gonna start by telling a story. This is the evolution of the docs for the Netlify CMS project and How that started pretty simply it was a repo and github and It started as Docs that are stored in the same repo as the code The way that so many open source projects start with a read me in github and that's where you viewed the docs You do it in the github UI Over time added a couple more added a docs folder and some more files inside of that Then that's when I came on just almost a year ago and added a couple more docs into that But still just reading those inside of github And then we kind of decided that we needed to make a real website So we didn't just have people going to get have to see the docs We needed to have a site that had other information so that you could actually navigate it without having to scroll past a whole bunch of files That has other pages besides Documentation pages like the community page on there is a separate page and Also has information and also can be kind of media heavy like for example this little Screenshot here is actually a video demoing the site so Those things can kind of add up in your docs repo So when we decided to make that we decided to put those docs into a separate repo so we created a new repo and We're built it with Hugo and anyone familiar with Hugo. It's a static site generator. So, okay So it's built with Hugo and the docs are all stored in markdown files Looks like the same structure We took basically exactly the files that were inside the code repo and then we moved them into We copied them. I should clarify. We copied them into a new repo So there they are still existing in the original code repo in the netlify CMS repo But also now copied into the netlify CMS www repo You can imagine where this becomes a problem because now we have two sources of truth And we would have people who are writing in pull requests all the time to the code repo and say hey I fixed this typo in there and we just say well, yeah, it's fixed there But you're gonna have to actually go to this other one to what fix it there too. Yeah, I know it's kind of stupid But that's just the way it is right now because people didn't want to give up being able to read their code Or read their docs inside the code repo But we also wanted to have our docs inside their own separate repo. So we were doing it in two different places at once That was not fun So we started to use a fancy new tool Which is called markdown magic This is an open-source project made by David Wells who works at serverless for the serverless framework and Basically, it's a thing a system for you can include it into your project You write comments inside of your files and those can get pulled into markdown files. It's Really geared towards making your markdown files inside of github's UI Do all sorts of fancy stuff that you wouldn't necessarily do What we used it for was to basically take The code that the docs files that were inside of the code repo and automatically generate them into the website repo so This is what it used to look like before we added markdown magic This is on the website repo and we've got our markdown file is in there You can read it inside the github UI. It has some Toml information about the title and the position But then what we did was we removed that and we took out that part that was in there So now if you went into the Docs website repo There wasn't anything to see inside a github anymore and if you went into the raw view of the file Then you'll see this comment in here So it won't show up in here It won't show up if you were to load it into a site But has this comment that says hey go to the code repo and get this information from there and pull it in here And this runs at build time so when we run the build on the docs repo site It basically goes to the code repo site and grabs all the docs information and brings it in there Which is all magical and wonderful except that Now we've got these docs in here and that's nice because everybody's happy. They're inside of the code repo They can see their docs in the same place as the code But we're also pulling it into the other place, but not managing the same content in two different places The trouble is then making magic is hard Especially when it breaks your other magic and The magic that we had going on in there. It was continuous deployment and continuous integration that leads to deployment One of the really cool things that the site's hosted on netlify that I've gotten really spoiled by is deploy previews So this is what this looks like is when somebody makes a PR to the repo What it will do is automatically run a build of the site Taking if that PR had been merged into master. This is what the site would look like and it puts it on its own Excuse me on its own address So if this person puts in PR 1072 then we get a brand new Link to a build of the site at deploy preview 1072 dash dash cmsdemo.netlify.com And you can set up these notifications so that it will give you this little check mark if the build succeeds And if you click on that then it'll take you to the link or you can get these comment notifications It'll take you to think that's a bot There are so many people though who think that I am just like really fastidious And it doesn't help that like my username is very thorough that they think I'm actually going in and commenting on every PR Say hey your deploy preview is ready No I'm always really excited about it, too, but But those are really nice however when we have markdown magic the trouble is that this gets triggered when there is a change on the repo and The docs repo if you've made a change to the docs on The code repo if you've made a change to the file on the code we made a PR there The docs repo has no idea that you just did that So it's not going to make a deploy preview and it also isn't going to be pulling in if even if you trigger PR and made a Deploy preview it's not going to know where to pull that from so you have to go to the Docs repo Set up how you're going to make that connected to the other one make a separate PR That's not to be merged, but only for a deploy preview and it's a total pain in the butt And So that led to me put filing an issue publishing docs in two places is no fun So managing them in two places was no fun either, but plus you need to do places is Is not really helping things because I really want my deploy previews to work I don't like dealing with the fact that like the links get handled differently in two different places So we had to talk about this this way is not working. We need to find a new way to store our docs and Also build them at the same time without running into issues And we had some different things we needed to think about so some of the things we were talking about in this issue Oh, and I should mention that most of these images Link to the page that they're screen-shotted from so if you ever wanted to look at them you can So some of the things we were thinking about were the ability to contribute to the docs code To avoid doc strips so the idea that when people are making contributions to the code We want it to be just as easy to be able to make contributions to the docs to go along with it So we don't have to like say oh go somewhere else and go do that Also, we have to consider that against managing the repo size and complexity that like you have a whole bunch of extra stuff That doesn't have to do with the docs Do you want to have that all in the same place as your code thinking about that? The other thing to think about is how to handle legacy versions and how to handle versioning in general with your docs to match your code So looking at that those considerations and weighing them against the options that we presented at the beginning Looking first at docs in the code repo. So some examples of that First off a few examples that I'm showing here using github UI for display Also tons and tons of tiny little github projects obviously that's how they show their docs to that they don't go and build some other thing But here's a few examples React static is a static site generator that builds it in react Markdown magic as it happens is also the docs are stored right there in github and that's the only way that you read the docs Go true our our open source Authentication library is also just stored that way because it's a very tiny little thing that doesn't have a website to go with it But some other ones where the docs are stored in the code website Code repo and built to a separate website are as it happens some static site generators Gatsby and Jekyll which it kind of works out that way because like if you have the code for a Jekyll site Because the Jekyll docs are a Jekyll site the Gatsby dots are a Gatsby site They're both static site generators kind of makes sense that they would have you know the docs built in their own tool But by having the doc site inside of the code repo you've kind of got a working example of how it even works So it makes sense that they're in the same repo So what are some things that are what really shine about this what makes this a happy path? To going to it One is it connects the docs with the code So that means that you can view them together in the same repo sometimes people like to be able to look at the Docs and the code at the same time in the same place and be able to go back and forth with them easily Also, you can just clone down one repo and have access to all of that at the same time They can also be Put together into the same PR so when someone makes a code PR they can also add their docs that it's in the same thing you can make that a requirement as part of the PR Then there's also the idea that I mean if we want to avoid Docs drift and docs that don't catch up with the code What better place to store the actual docs is in the code itself in comments, right? And it makes it really easy to do that when the docs are in the same place as the code One example tool for that is JS doc which takes JavaScript and Comments in there and puts them out to docs there's a plug-in for That connects to that called swagger JS doc which takes your JS dot comments and converts them into open API spec There's also a markdown magic plug-in that takes your JS dot comments and Puts them into markdown files that you can show in markdown magic tools, so That kind of takes that even further besides just having the docs inside the same repo you can have them inside the code itself But it's got limitations obviously Another thing to think about is that you can tie past versions to releases really easily so in the github UI when you go into a repo you can choose the branch or you can choose a tag and You can go and browse the repo at any of your previous version tags And if your docs are in the repo with the code then you go to that version tag And you browse to docs and you're seeing the version of the docs that goes with that version of the code So that's kind of convenient, but it also assumes that you've kept your docs up to date with the release So if they weren't then they're going to be a little off It could also work well with tag-based deploys so one thing that we do is Branch deploys so whenever you push up a branch to your repo you get a deploy of that branch We're also going to start enabling because one of the docs projects is actually requested this tag-based deploys They basically work the same way that when you've got a tag you automatically get a deploy of that tag All set to itself now Those are some pretty good benefits, but there's some other bumps in the road that we might have to worry about So we're balancing that against repo size and complexity And that can be an issue for example if the website has lots of non docs content. So if it's got a blog or Lots of video or heavy tutorials or things like that stuff that actually makes your code repo really really bulky That's something you might want to consider Which was the media heavy part Another thing to think about is if versions are stored in sub-directories So there's a variety of ways of saving your old versions one way is to use the tagging another one some tools will take your snapshot of the docs and Basically copy them into a sub-directory in the repo and if you do that then You're gonna have a whole bunch of copies of your docs Inside that same repo, which is not very efficient and you really don't want to be that inefficient Inside your code repo at the same time so Of course, you don't have to store it in some directories you could do lots of other ways But if you're doing it, I wouldn't do it if you're actually stored in the code repo Another thing to think about is that the docs can get ahead of the release. So as You're merging code and you wait until you tag your release that code gets merged, but the release version that people are using Doesn't have the new merged code, right until you tag the release. So if you have 1.1 and that's what people are using because that's the most current version and you've merged some new commits in there that are Going to be part of 1.2, but 1.2 is not out yet It doesn't matter that they're in the repo in master because they don't have The new version and then when you get the tagged version 1.2, then they'll get all those new additions Right, but if your docs are getting developed right along with that and getting merged into master right along with that Your docs are for 1.2 while people are still using 1.1 So that's one thing you have to worry about if you're just using tags to launch your code your docs So one thing you can do is you can publish everything to a next branch and merge that right at the time that you do the release But then you're not really automatic anymore Docusaurus is a tool that's an open source docs generator that Facebook built that Does the handling with subfolders and what it does is it has like a next sub or Subfolder and then when you publish a release that becomes the current version So it handles it, but then you've got that subfolders issue. So I don't know another thing video JS is an Open-source project that they have a script now that basically Stops their publishing until it's released tags. So they run a script and I meant to link it I'm gonna have to add that link to the slides That basically says hey if you're doing a build check first is this a deploy previous This is a branch deploy if so go ahead if this is to master and it's a production deploy And it's not released tag don't build it and it just won't publish them until it gets released tagged Another thing to think about if you're using the github UI with the docs in the code repo is they kind of look generic they can be unwelcoming to github newcomers and They can require a lot of repetition markdown magic can't help with that. I'm running out of time. So I'm rushing The last one is by far the most popular with really big projects So most of the ones that we have are actually done this way The two on the bottom ember and docker actually have multi repo sites So they might have like marketing in one site and their docs in another site and then their code is actually in like a dozen other repos This can be good because it separates your concerns You might have a whole bunch of different interlink projects that you want to document all in one giant documentation site You can't store the docs in the repo unless you're doing that crazy markdown magic sort of thing So this is what docker did they pulled all their docs out of their repos and put them into one giant docs repo And so if you're doing docs for swarm or docs for Moby or docs for any other different parts It's all inside of their one docs repo Another thing media heavy docs or multi-purpose sites. It's great because now they're going to be in separate from your code Subfold versioning another one that's better But branch versioning is probably a better way to do it because it's much more efficient in general And it also supports specialized maintainer teams So if you're a big project and you've got your code maintainers and your docs maintainers and they're two different groups of people like it's easier for those people to manage their two separate repos you can use code owners and GitHub now and that's one way to handle it all in one repo, but it's kind of nice to have two separate one thing Docs can updates can be overlooked the farther they are from the code the more likely they are to get Forgotten however, you can make that a requirement for merge So like in the end this is a people problem so if you make a docs PR a requirement in order to merge their code PR then You can make them do that over in the other repo Another thing is that versions aren't automatically tied to releases But that can have some advantages and you can recall you can make those docs go ahead and then say hey merge these When it's time to release Another thing that's an issue is that docs contributors don't count in the code repo if they're in a separate docs repo One thing that we use is all contributors, which is a spec created by Kent C. Dodds That has some tools to go with it that recognize all sorts of contributions in addition to docs bug reports and blog posts and tutorials and design and Organizing events and monetary contributions. They have all sorts of ways of contributing Counting those I'm totally running out of time. Sorry The hybrid yeah, I suppose so, but there's going to be another whole deaf room after us The hybrid approach was what we used to do Servalus, which is actually those docs were coded by the person who made markdown magic You can tell he's a fan of that sort of thing They pull their docs content into a site video JS They have their docs and their marketing sites separate So their docs are in the code repo and their marketing site and the other bulk that goes with that is a separate site And the docs one is just docs.videojs.org and the marketing site is just videojs.org So they can have those in two separate repos building two separate sites It keeps the docs close to the code, but it can be You can keep the heavier content separate those things are all cool And it doesn't have to be that complicated if you do like video JS's method where it's two different sites for two different repos It can also get difficult like if they are two different sites You have to manage the styles of both of those so that they have the same look and feel if you redesign one Then you're going to have to redesign the other or think about how those match up But you can also think about having a really complex setup when you're pulling in that multi repo content into a site Like we dealt with serverless has this whole crazy thing that all connects in different ways So it either requires a whole lot of work in the beginning to automate it Or it requires a lot of work to just maintain it if you haven't automated it So where did we end up we moved it back home again So the docs came back into the code repo We went back and forth a bunch of different ways and we may end up going into a separate repo at some point But they're now back inside. It's just the Hugo site is now inside the website path It's built the same way as before but it's in there We can have deployed previews for different things so docs gets its own deploy preview the demo site That's built in there gets its own deploy preview So that's kind of nice Okay, okay, okay. I should stop one clip tip though I'm going to say this and you guys had it in your kiddo thing too no matter where you store them Add an edit this page link to all your docs and then people know where to find them and those aren't that hard to add so That's it