 Hello, everybody. Thank you so much for taking the time to be here. Thank you so much, Carson, for sneaking in late. It's been a while. This is open source documentation and how to scale it. My name is Celeste Horgan, and you are all here at KubeCon Cloud NativeCon EU 2022. Hello, world. So as I said, my name is Celeste Horgan. My current job title is staff technical writer at Stripe. That is a very new job title for me. I only left my last job a few months ago. Prior to that, for two years, I was the staff technical writer at the Cloud Native Computing Foundation, where the portfolio work that I did was supporting all 140-plus CNCF projects with their documentation to the extent that a team of three can possibly do that. What that really looked like is we gave a lot of advice to different Cloud Native projects on how to do well by their docs. There are still other members of the team who are still there. Their names are Nathan Waddington and Pratish Shalin. You may have encountered them in your journey in the CNCF. And a lot of what we're talking about here are still services that they can provide, though I specifically will not be doing that for you. In the community, I still hold a couple of community titles. I am a member of the Kubernetes SIG docs group. And I am also a member of the Kubernetes Code of Conduct Committee. As such, please be wonderful to each other during this week and especially during this talk. That is a suggestion more than anything else. So let's talk about this talk a little bit. And I apologize. I'm trying to make sure that I remember how much time I have left. This talk is going to be about some of the common challenges that docs teams have found while we did CNCF docs assessments and while we ran CNCF office hours for documentation. We'll talk a little bit about what those are and how you can access them if you are a CNCF maintainer. This is also a little bit of an initial exploration of some of the broader issues that open source documentation faces that we can't necessarily address on our project by project basis. But most importantly, the core of the presentation is recommendations that the CNCF documentation team found itself making over and over and over again to different projects based on size so that you can have a little bit of a shortcut to the advice that we give. But this talk is not. It is not an exhaustive list of every problem documentation teams face. It is not an exhaustive list of every solution that is possible. This is not a checklist for projects looking to graduate. Documentation is one component of graduation in the CNCF process. And this is not a one-size-fits-all approach. So before we start, most of the content of this talk was generated from CNCF docs assessments and CNCF docs office hours. Docs assessments are a service that the CNCF team provides to CNCF projects where we essentially look at your docs holistically. And we give you an evaluation and a report based on where we think you are. It's super useful. One of the people that we work with once described it as the most useful service they had ever gotten from the CNCF. So if you are a project maintainer, I highly recommend you go to the CNCF service desk and request this at least once in your journey with the CNCF. The other thing that we do offer as a documentation team are documentation office hours. We run these once a month. I believe they're the first Tuesday or Wednesday of the month. And these are drop-in. If you've got a documentation question that your project is facing, we will put you in front of experienced technical writers who can give you an answer. Once again, I no longer attend those because I no longer work for the CNCF. But I loved doing these when I was there. It was such a fun opportunity to have casual chats with maintainers. So you can find all existing appointments for this on the general cncf.io calendar. It's not the greatest for actually finding stuff, and I apologize. But they're there, and you should absolutely drop by if maybe you're just looking to learn from other people and other projects as well. OK, so let's talk about the extra small to small size projects. I don't want to categorize these by how many maintainers they have because there are very large and very mature projects in the CNCF ecosystem that really only have 10 maintainers. So let's talk about how much documentation they have. Typically, what I would say a small project is, is they've got maybe one to 40 pages of documentation, and 40 pages can actually get on the big side of things. This loosely corresponds to the CNCF sandbox, if not CNCF incubating as well, depending on the project. At this stage, the major issues that we see projects facing are around tooling and how to choose a good tool chain. The move to what I like to think of as formal product documentation, so moving away from kind of readmes and people answering questions in Slack to compose documentation that somebody's thought about as a user experience. And the third thing that we usually see at this stage is content completeness. If you do these three things as an early stage project, you're going to be able to move on to being a bigger project, at least from a docs perspective. OK. So from a tooling perspective, one of the most common questions I get as a documentation person out there in the world is, do you have a best of breed tool chain for docs? What's the best static site generator? How do I set this up? The unfortunate answer in the world of documentation is there isn't a best of breed right this second, but I think we're starting to approach one as we go. The advice I have, especially for small projects, pick the least cumbersome tool chain that you can manage. The people who run small open source projects tend to be extremely passionate engineers, and it is wonderful to see that. And so they tend to try and solve the documentation problem with tooling. Now is not your time. There is a time for automation. Now is not your time. Here are a few options that we like at the CNCF. We do have a Hugo Netlify starter that we use to spin up projects. It's neither here nor there, to be perfectly honest with you. We like spinning up projects on Doxy, and one of our technical writers is actually a steering committee member on the Doxy project. We tend to go for Doxy because it's a little complicated at first, but it does leave a lot of room to grow. The company I now work for Stripe, literally last week, open sourced some of its documentation tooling called MarkDoc. MarkDoc, honestly, it's a little advanced for a small project. It's really best if you're working with a dedicated front-end engineering team. So it's on the table. I don't think it's your best option if you're a small project. The big other thing with tooling at this stage is do not get bogged down in automation. Engineers especially love looking at a documentation problem and going, can I auto-generate this from code comments? Can I write a spec and then auto-generate it? What if I, to pull an example from the Kubernetes release engineering team, what if we look at the comments in the PR and then we pull the comments into a release note and then we don't have to write the release notes? All of these are smart solutions, but not when you're a really small project. Focus on learning to write your documentation and getting the process around writing things first. Speaking of which, let's talk about content. To quote one of my old coworkers, sometimes I look for open-source projects to implement for my clients because I'm an engineer, I'm a solutions architect, and that's what I do. I implement for clients, and sometimes I'll find the perfect project that does things perfectly and has every functionality that I want, but the only documentation they have is a readme and I have to implement for professional clients so I don't choose the project. I need something that's more professional. So in terms of content, your biggest goal as a small project is moving from readmes and mark-down documentation in your code repositories to a website. That is what will convince people that you are serious enough and that this project is well maintained enough to use it, and that is a part of what you need to do when you're a small project. The second thing you need to focus on is not even necessarily content quality at this stage, it is content completeness. For everything that gets developed, you need to assess and write the documentation for it. At least say that you've written some. It doesn't need to be good, but you need to learn how to check that checkbox off. A simple process is every dev task always has to have a checkbox that says, did you do your docs? Yes or no? A slightly more complicated process that is very worth doing at this stage is something like the Kubernetes enhancement projects process or architectural decision records, where for every code decision that gets made, you produce a little bit of documentation up front saying, why did we do this? What is the structure that we're trying to create? And what is the intended impact for the user? I love the cap slash ADR process, especially for young projects, because it gets you used to the idea that you need to justify your decisions. One, two, when you actually go to write your user documentation, you've kind of written half of it already, and you can honestly copy and paste, and that's sweet, copy and paste when you can. Two, and number three, I think people underestimate how long lived open source can be. Log4j is like 20 years old, and that's nothing to say of Linux. And these things are still in production. And when you think about documentation and you think about open source on a generational time span, learning to document your decisions early and just keeping a little journal is incredibly invaluable, because you will not always be there, but somebody else needs to understand what you did. OK, medium-sized projects and their problems. So you can think of a medium-sized project as having maybe one to 100 documentation pages, or maybe they've just got more maintainers. It's a little fuzzy. This loosely corresponds to the CNCF incubating stage of projects. At this point, projects start to care a lot about content quality, about content organization. They start to think about community, and they start watching your engineers. You can start to think about automation, but don't get excited. OK, so let's talk about content quality. Broadly speaking, if you, as a medium-sized documentation project, maybe you're in the CNCF and you're starting to think about graduating, think about organizing content logically on each individual page. The easiest way to think about this is, does the page talk about things generally, and then more specific as you go down the page? If you can do that on a page-to-page level, you're going to be pretty organized. If you're organizing similar pages in similar ways, or you're, for example, following a template, that's also going to go a long leap towards your content quality. In the world of documentation, we call this parallelism. And it actually aids in users' understanding and retention of content when similar pages have similar content structures. Follow-tech writing practices, I will have a full slide on this after, so don't worry about, I'm not going to go into too much detail there. And the last thing is, it's time to think about adopting a writing style guide, but if you don't do it at this stage, you're not, you're not hooped. Content organization, so in the world of technical writing, we often refer to this as information architecture. At this stage, when your content set has started to grow, you need to start thinking about what is a person's path through the content? What is the beginning-ish, the middle-ish, and the end-ish? You could truly think of it as a narrative. There is an easy shortcut that works for almost every project or product that I have ever been on in my 10-year career, and that easy shortcut is organizing pages following the user's journey. So for a lot of things in the CNCF ecosystem, which are about cloud-native deployments, basically, what this looks like is there is an installation step. You're either installing to an existing cluster or a new cluster. So your first chunk of documentation is about installation. Usually, the next step in the cloud-native world is some sort of configuration step. Okay, you've installed the thing. Now you want it to do something for you, and you got to configure it. Either you're applying configurations via everybody's best friend Gamal, or you're like passing in a bunch of CLI flags. So what are the different configuration options? You usually have to lift them out and define them for somebody. Usually the best thing you can do at this stage, too, is provide kind of recipes for common configurations, like start to understand your user and start to say, if you're this kind of business, if you're running this kind of cluster, this is a good starting point of configuration. For cloud-native, the next step is usually deployment. How do I deploy this thing into the wild, and then it's maintenance. Like what goes, what can go horribly wrong that I, a cluster operator, may need to know. Simple, it works almost every time. I can do this for a non-cloud-native use case, too. If you just figure out what the user's journey is, as they're using your thing, that's your organization done and dusted. If there are branching paths, for example, with configurations, where if you're one kind of cluster operator, you might need one kind of configuration versus if you're another kind of cluster operator, you might need something different. What you need to do is provide a little introductory text to those branching paths and say exactly what I just said. If you're this kind of person, you might need this. If you're this kind of person, you might need something different. Simple. A couple of sneaky plugs. These are around technical writing best practices. The talk I gave last year at KubeCon EU, writing for developers, taking your project docs to the next level, provides kind of the very, very basic how to do technical writing in 20 minutes or less. If you wanted a slightly more in-depth piece of information, one of my former colleagues at the CNCF, current colleagues at Stripe, as well as truly some of the best people in the technical writing and documentation space, have written a wonderful book for you called Docs for Developers. It is truly some of the best content I've ever read on technical writing and the entire book is covered in pictures of corgis. So you should pick that up if you're interested in learning to be a better writer. It is wonderful. Okay, community and tooling. When you're a bit more of a medium-sized project, it's time to start thinking about how does the community contribute to documentation, not just how does the project do documentation. One of the best things and something we recommend almost every project in the CNCF do at some stage is start creating documentation good first issues. Documentation is rife with good first issues. Reorganize a page or rewrite a paragraph or correct the grammar in this paragraph. And people love documentation, good first issues, engineers and writers alike because they are low barriers to contribution to the project. It's I can get my commit in, I can get my feet wet, I can learn what submitting a PR to this project is without breaking the code base. And that's very appealing to people. So I think if you can always keep a backlog of five or six good first issues, you're in a beautiful state. One thing to note about good first issues is these are things that are often really quick and easy for the maintainers to do. And it's actually a little cumbersome to write the good first issue. But it pays dividends for the community and the community is what will get you to the extra large size. So do it. Scope your documentation tasks well, something I see a lot of in CNCF projects, Kubernetes included, is like, hey, yeah, we should have documentation on this topic. Yeah, we should have a little advice on how to configure to your deployment. And that's the ticket. That is not a useful amount of information, especially if somebody is a fairly new contributor. And as we've just mentioned, as your project grows and as your community grows, you have more new contributors and they will go to the docs first, looking for easy kind of low hanging fruit. So if there's documentation missing in your doc set, really take the time to scope out like this is what the page title would be, here are the topics that we're talking about, who would be good people in the project to contact to get more information about that. Finally, tooling, you can start to think a little bit about automation here, but don't think about it too much. There are natural language linters available. The biggest one is called Veil. And it might be time to think about is this something we wouldn't want to implement. I will warn you though, in every professional documentation environment I've ever been in, we have used Veil as a VS code plugin, but never as failing tests on the actual PRs because it just tends to annoy people. And people are worse at grammar than they think, and that's what Veil will try and fail you on. Finally, moving docs to their own repo is something we recommend almost every CNCF project do, not every project listens. If you want to keep docs in your code repo, we will support you in that decision, but beware that you are signing up for some pretty big maintenance overhead as you grow, and that will be very difficult to undo later in the pipe. So if you're going to consider moving your docs to their own repo, do it while there's still a pretty small set and why your project is not that big. And again, if you choose to keep them in the code repo, I warned you. Okay, a little bit of a note specifically for CNCF projects, and I sincerely hope there are non-CNCF maintainers here, a key component for CNCF graduation is attracting external users and new contributors. That is really the metric that they're looking for. So at this stage, seeking out specialized helps from experienced technical writers works really well. CNCF tech docs office hours and docs assessments, as we talked about earlier in this talk, are the place to go. And Nate Waddington is one of the people you encounter. He is an incredibly friendly, lovely human, and we will talk a little bit more about that later. Okay, let's talk about the large and extra large projects. So this really corresponds to late-stage incubating projects and projects that are graduated in the CNCF. At this point, documentation sets are one-to-infinite. To give you a picture, I think the Kubernetes project has about 400 pages of documentation, and that's in one language. And bear in mind that the Kubernetes documentation is localized into many languages. So actually there's tens of thousands of markdown pages there. The focuses here are really around community organization and process because it will not get done alone. It's really around content policies because at this point, the project is a bit of a magnet for marketing. And the documentation is a wonderful place to insert marketing. People love doing it. There's some big hard problems that crop up like localization and versioning. And we need to start thinking about advanced automation at this point. So engineers like rev your engines. This is the moment you've been waiting for. Here's where you can automate the docs. Okay, so before we talk about automation, though, we need to talk a little bit about ownership. When you get to a really large project, it is very important that you have dedicated ownership for documentation. So as I mentioned, I was directly employed by the CNCF to do documentation within the community. And as I also mentioned, I was one of about three people spread at 140 projects that is not a sustainable way for anybody to contribute to anything full-time. My personal advice, particularly for graduated projects, is you need to seriously consider staffing technical writing roles if you want to have good documentation. It will not get done without full-time help, first off. And second off, when you look at a documentation set the size of Kubernetes, that is not actually one full-time rule. That's five to 10. Documentation is seriously understaffed in open source and it will start to cause massive sustainability and maintenance problems. That's your solution. The next thing you have to do is have really consistent policies, particularly around reviewing and merging. As the project grows, there are lots of people who think they understand the project. And you need to get really, really clear about the fact that documentation is the truth of the project. And because there is no one company behind it, it is very important that the truth remain true, which means that as random kind of contributors off of the internet go like, wait a second, I know a Kubernetes thing, let me write the documentation. You need to know who can verify the technical truth that a random person has contributed, because that contribution is very likely valuable, but it is also potentially very harmful if untrue. Content policies. What happens if we deprecate a feature? Question mark. How do we communicate that to people? How do we communicate that in the docs? At what point do we actually remove it for the docs? You need to think about this. Betas and alphas. The Kubernetes project loves betas, and it got to a place as a project a few years ago because it loved betas, but it had no deadline for its betas, where probably like half to maybe two thirds of the project were beta features. That's a cross-cutting concern. That's something that both the technical leads in the project as well as the documentation leads need to understand because it creates a very poor documentation experience where this thing looks like super janky and shaky, but it's not up to documentation to decide on that, that beta and alpha policy. Release notes. As your project grows, people start actually caring about what you've done, and you need to actually explicitly say in version XYZ changes, YZ, ABC came into place. Depending on the project, API limitations out of support functionality. If you were in Don Foster's talk earlier, documenting what is in and out of scope for the project and what the project intends to actually do. This is super important because one, people are building production systems on you at this point. You need to tell them if there are any limitations, those are very important to them. Two, when you get to a place where you have multiple contributors, particularly multiple corporate contributors, you need to understand what is out of scope or else people will submit whatever they want, as Don said. And there's a whole bunch more content policies that you can put into place, but you need to start thinking about it at this stage. Automations. Merging approvals and reviewer assignments. Documentation, particularly if you have technical writers on staff, requires a very different kind of review than most projects are used to. The other thing with documentation review as well is that it is something that people feel that they all have an opinion on because it's just English language, right? I know how to write. I know how to talk. Managing reviews so that they don't become pedantic is a super important skill to start thinking about as a large project. I said release notes again. The Kubernetes project automates release notes. At first that was very skeptical, now I'm on board. When you're a really big project, it starts to make a lot of sense to automate your API documentation if you can, as well as your CLI flags. So what we're talking here is really pulling all the CLI flags out of a code file and then pulling whatever help is coming up on that command line interface and dumping it into a table and then dumping it into your documentation. When you're a large project, this makes a lot of sense. When you're a small project, I would encourage you not to do this. Other hard problems, localization and community policies around this. Localization is interesting because it impacts something else on this list which is performance because localization increases the amount of files that you have in your documentation repo exponentially. Every time somebody gets a shiny little thought in their head of localizing it into their favorite language. So all of a sudden out of nowhere you're in a place where your documentation of all things has very poor performance and then it actually makes a lot of sense to put some engineering cycles into that. Containerization starts to make a lot of sense for large projects because, for example, the Kubernetes project tends to have about 150 PRs in documentation open at any given time and a lot of them fail not necessarily because the content is bad, like it's markdown content, it's not that dramatic, y'all. But because the build environments that people are using locally don't match the build environments that the server is using, this is the perfect case for containerization. Versioning docs is another one. Nate Waddington has done a ton of work on this and I'm not gonna talk about it too much. What I would like to talk about, however, are what I consider the NP-hard problems of documentation in open source. These are NP-hard because no individual product can solve them. They are an artifact of open source as it is today. I find these fascinating and that's why I'd like to bring your attention to them. The two that come to mind, how do I get actual technical writers contributing slash my engineers want to write less? How do I make that happen? Two, how do I do large-stale content reorganization in a relatively mature project? This is a Kubernetes problem for sure, the second one. These are my opinions. They are not founded by fact and I don't have all the answers but I would like to open the conversation for you. My engineers want to write less. What do I do? Technical writer absence in open source is a result of misaligned understanding on the part of the organizations that are doing this. Open source is critical infrastructure and it must be staffed in the same way that you would staff literally any other project or product. Right now, open source organizations have an idea that if we staff a bunch of engineers and if we staff some program managers, the project's going to be fine. When you get to the scale of a project like Kubernetes or like Envoy, that doesn't cut it anymore. Think about how many things on your phone run Kubernetes. That is not going to undo itself in the next 10 years and the people working on Kubernetes now will maybe not be working on Kubernetes in 10 years. You need to staff the people who are going to write things down onto the project. You need to staff the QA staff onto the project. You need to staff the release engineers onto the project. It simply will not happen otherwise. The way I sort of think about this is that open source in many ways is the closest that software gets to the sort of same standardization that other engineering practices have where it's like you would never, the I-beams that compose the ceiling of this space are stress tested for certain weights and certain applications and you would never want to use them for anything but that. We don't have that in software. And when you talk to real like actual mechanical engineers, it freaks them out that we build so much stuff with zero standardization. Open source is as close as it gets. You would never develop a vaccine without writing the instructions to go with a vaccine. That would be irresponsible. So the answer to this question in my opinion is to be more responsible and to have a bit of an attitude shift within open source. Okay, content organization and relatively mature projects. I don't have an answer to this one but I can tell you that this is a problem that the Kubernetes project faces. And I think the issue is around the fact that horizontal project concerns, so release engineering is one of them. The other one is documentation. Community is another one. Existing project policies do, oops, do a very poor job of servicing horizontal concerns. So I think the question to ask to maybe start solving this problem, and again, I really don't know the answer here, is how do we better support cross-cutting project functionality like SIG release and SIG docs with better process that doesn't hold them up? Because if you think about reorganizing content in Kubernetes, that potentially has hundreds of stakeholders that need to approve it. And that's why it's unachievable. And I think it's similar of some concerns in SIG release. So I think if we can answer that question we can unblock the actual work. Quickly, docs assessments are off at ours. Go do them if you're a CNCF project. There's some more content for you to consume if you wanna learn how to actually write great documentation. Some other resources. If you need some technical writers and you wanna talk to some people, the write the docs community is where they all hang out. There is both conferences and the Slack. The Slack does in fact have an open source channel pop in there. Google Season of Docs is a program that pairs wannabe technical writers with open source projects for a paid internship. There are more CNCF resources available. If you look at the service desk there is in fact a massive list of the things the documentation team can do for you. The other thing you can do through the CNCF is participate in LFX mentorship where you mentor somebody into the project for documentation or other reasons. Finally, I guess as a resource I go by my name on Twitter, Celeste Horgan. And I very often tweet about docs. This talk, if you enjoyed it, was the result of one of those tweet threads. And I have other talks maybe planned based on other tweet threads. So if this is content you're just consuming, follow me on Twitter. And we have a little bit of time for questions. That's that. Thank you. For things that you have no content for at all right now. Totally. So just the question for those of you who didn't hear, how do you balance high quality docs versus docs that you don't have content for? There is no balance. Always go for completion. Completion is the base state. Documentation at its best documents the truth of a system. And if it is not documenting the full truth of a system it's not doing its job. It can do its job with bad grammar. It can do its job with bad structure. It can do its job if it's a little disorganized. It can't do its job if it's not complete. Always prioritize completion. So the question for those who didn't, there is in fact a mic by the way guys. The question for those who didn't hear it is, can you expand on why you would want to move your docs out of the code repository? How that affects ownership of the documentation? And just expand on it a little bit. Nate and I are just like making eye contact because this is like, this is the struggle of every project. And every project is like, do I have to? Really recommend it. Yes. The problem with keeping your docs in the code repository is at some point they then become a dependency. They become dependent on your code tests passing. Which seems trivial, especially when you're a small project you're like, why wouldn't it my test pass all the time? Whereas anybody who has worked on any sufficiently large and mature code base knows that tests get flaky and then they stay flaky and then that's just what it is. And you know, every large code base I've ever worked on has had a flaky test situation. So that's thing number one. You get into a situation where like really simple markdown changes get held up because the integration tests aren't passing and the person doing the documentation maybe doesn't have the knowledge to unblock that implementation, that flaky test. So as a result, people stop contributing to documentation because they look at the single like, I don't know why this test is broken. I don't want to solve this. I don't know anything about this part of the code base. I don't want to break anything. So I'm just going to close this PR. And then all of a sudden you have a lot of people who aren't contributing to documentation who might. The advantages of keeping it in the code repository are typically that you can do like kind of deeper integrations. Hot take. Almost no open source projects I've seen has actually done those deeper integrations that take advantage of being in the code base. The only one I've seen is maybe Kubernetes and that is frankly arguable. That is also accomplishable by pulling in like specific commits from a different code base. Can do that for you. It's a little complicated but it works. I have seen that deeper integration thing work in private companies. Like for example, Stripe does a bit of this on the back end. But private companies also typically benefit from documentation tooling teams which most open source doesn't. In terms of ownership, I think it just makes it easier if it's in its own repository because then the people who can maintain at that repository level, they don't necessarily need to know everything about the code base. They don't need to know how all of that works. It just makes life easier. So sorry if that was a little long-winded. I think I'm being like summarily kicked off. So thank you so much for your time everybody. I appreciate it. Thank you.