 Kubernetes follows a SIG, or special interest group, model. And that's how Kubernetes development work is done, is through SIGs. There is a SIG for documentation, SIG docs. So when I do these presentations, I like to flip Q&A. And I like to take questions up front to ask specifically what it is that you are hoping to get from this talk, so that I can address that as we move through the content. So just really quickly, what are folks hoping to get from this presentation? I cannot promise satisfaction, possibly disappointment, but I will at least address your concern. Yes, please. Excellent. Yes, I will talk about how we handle workflows for change management with multilingual content. Anyone else before I get rolling? Excellent, yes. I can also talk about that as well. That's built in really nicely to our static site generator. So I will be sure to cover that. All right. Let's see if I, it looks like I have to do this manually. Oh, very good. So we're going to talk about the tool chain and the workflow for Kubernetes documentation. These two links are going to be really helpful for exploring content around this presentation. The first URL here is for the actual documentation. This is the documentation landing page for Kubernetes. And this is the GitHub repository site. Everything that I'm going to talk about today you can see either in action on the site or in code in our repository. Let's also talk about what this presentation does not cover. This is not about Kubernetes functionality. If you came here hoping to learn more about Kubernetes, sorry, that is a much different and much deeper set of talks. This is also not a presentation about how to localize documentation. From a tooling perspective, we're concerned about the output of the localization process. So we're not going to talk so much about how to localize documentation as we're going to talk about what to do with the finished product. So I'm happy to say today that Kubernetes docs natively support multilingual content. This is amazing. This represents the ability to host multilingual content for Kubernetes docs represents a huge technical pivot in less than a year. And it would have taken us even less time than that. But we made a lot of mistakes along the way. And I'm going to share some of those hopefully so you can avoid them if you're considering hosting multilingual content. When we first opened Kubernetes documentation for multilingual content, we were simply not prepared for what that actually meant. None of us, none of the SIG chairs, none of our approvers had ever done an internationalization project before. We were all novices. So we were all inventing things, guessing at things, making huge mistakes. When we first opened the site for multilingual content, our workflow was not scalable. We were suddenly flooded by a significantly greater number of pull requests for multilingual content, and it was full of Chinese content. None of us are fluent in Chinese. So we were receiving requests to approve PRs that we could not read. And as you can guess, requiring fluency for all reviewers across all languages supported is not a scalable solution for reviewing pull requests. So that was not scalable. Our tooling was incredibly fragile. We responded to the overload of pull requests. We said, ah, we can't possibly deal with this. We'd better put all of the different languages, language in its own repository. But we didn't create those new repositories by forking the existing one, which made upstream contributions impossible. So we did that wrong. And of course, that solution, creating different repos, one language per repo, that means repo maintenance costs also scale. They go up with fewer resources with the same number of people to maintain those repositories. So that didn't scale well. And we still didn't answer the fundamental question of how we were going to publish multilingual content using Jekyll as a static site generator. We hadn't answered the most important question. How do we actually publish this content? So who here uses Jekyll or has used Jekyll in the past? It's OK. I got over it too. So some things to know about Jekyll. There are some plugins for Jekyll that support multilingual content, but they are incredibly brittle and they're very stale. At the time when we first looked about two years ago at Jekyll plugins, the freshest, the most recently updated multilingual plugin for Jekyll content had been two years old at that point. None of them had received updates in two years. So if we chose to continue using Jekyll, our tool chain was going to become increasingly brittle. But we also made good choices along the way. Not just bad ones, good ones too. So as we looked at the increasing technical debt associated with trying to flog Jekyll into doing what we wanted it to do, we decided simply to change our static site generator. So we changed from Jekyll to Hugo. And Hugo is great for multilingual content. It's not perfect, but it's great. And it's also really good for build times. Jekyll is an OK tool until you get about 100 pages on your site and then Jekyll's build times start scaling incredibly. The build times for Jekyll start going up a lot. Hugo is great for build times as well. So we resolved some pretty significant pieces of technical debt by changing our static site generators. That was one of the better decisions that we made. And we also solved for the pieces of process fragility, where we were trying to do too many complex technical things with too many people, multiple repos, trying to juggle all of that. Instead, we were able to consolidate all multilingual content and do our single existing repo. And we were able to integrate, we were able to answer some of the pieces of how do we approve pull requests for languages that we don't read. We were able to address that with our CI. And specifically, we'll get to this more towards the end. But Kubernetes, the entire Kubernetes project, uses a custom CI called Prow. And Prow, of course, is in keeping with the nautical pilot theme of Kubernetes. But Prow is a custom CI that's based on some Chromium commands, if you are familiar with the Chromium project, uses the concept of owner's files specifically. And Prow let us solve two of the most difficult pieces of process for multilingual content. Specifically, it let us set subfolder level owner and review permissions for multilingual content. And that means that Chinese localizers were able to create, review, and approve Chinese content. Not content for other languages, but very specifically Chinese content. Same for English, Japanese, Korean. So individual localization teams were able to review and approve their own, oh, so sorry. The other problem with Google Docs. This is my name. Good. We were able to use our CI to handle approval permissions and also some sorting and filtering pieces of the localization process that ended up being kind of thorny. We also made some good process choices. I would say that the best process choice that we made is when we went into this process, the chairs of SIG documentation, thought, we have to solve this all ourselves. We ended up inviting the localization teams themselves to participate in our tooling challenges. And this may seem obvious to you, but it was not to us, at least not initially. So inviting the teams, the localization teams themselves, to participate in the tooling challenges made things much easier. Because all of a sudden, we had a much deeper, a broader and deeper pool of technical knowledge for how to solve these problems. And that openness, that willingness to treat localization teams, not really as stakeholders, but as equal collaborators, that made a huge difference. And I would say that that is our absolutely best choice. But we didn't stop making choices. We didn't make a choice once and then stop. We kept making them over time. So when we first started trying to figure out how to answer all of these questions, how to address all of these concerns, Prow had not yet been implemented. And Hugo was still in its relative infancy. So we didn't just make a choice once and then stop and consider that choice done. We kept on asking ourselves, well, is this the best possible way to do things? And that led over time to reconsidering and looking at better options as they evolved. And sometimes that kind of willingness to revisit decisions ends up like cultural neurosis, like, ah, we didn't make the right thing, and it ends up being sort of paralytic. But for us, it ended up being a very healthy thing, the willingness to constantly revisit and re-examine our decisions. And so our decision-making remained open and that was very helpful. So the two most difficult pieces to solve in all of this were not the technical pieces. Funny enough, Hugo ended up being remarkably helpful for supporting native multilingual content. The two most difficult pieces to solve were CI-related, process-related, and that were permissions review permissions, like we talked about, and also sorting by language or filtering by language. And by permissions, I specifically mean the ability of localization teams to create, review, and merge their own content without requiring approval from non-fluent speakers. The nice thing about this is that this scales. Using Prow, every time that we add a localization team, we simply create permissions in Prow for that localization team to handle their own content, and that scales nicely. The team provides its own review. And by filtering, I mean specifically the ability to sort issues and pull requests by language. When you're looking at the, in GitHub, if you're looking at the list of pull requests for the repository and you can't sort it by language, it's a giant pain in the ass. So the ability to sort by language was very, very helpful. This process was not easy. One of the biggest headaches that we experienced was when we migrated the static site generator, we broke a lot of things. And that's because Jekyll and Hugo use different markdown parsers, crammed down for Jekyll and Black Friday for Hugo, respectively. And they are differently strict. There are things that crammed down says, no, you can't do that, or yeah, we'll give you a pass that are very different from Black Friday. And so it was not only a matter of being differently strict, the parser being differently strict, but it also exposed some lazy habits. So some of the things that we should have been doing correctly all along came to light when they broke suddenly. So that was a difficult piece. I will also say that while we love Hugo, it is not perfect. So with Jekyll, in order to, when you create a site map, let's call it, a site map of page ordering with Jekyll, you have to, Jekyll requires you to create a page map in a single YAML file, subcontent, I believe it's called subcontent.yaml, and you have to, that's where you specify page ordering for the site in Jekyll. In Hugo, you have to specify a page weight in the front matter of every individual markdown file. It's awful, especially when you have a site with greater than 250 pages, like we do. So it's not perfect, that's a work in progress, and given that Hugo just released what, 0.54 this past weekend, it's obviously still in development, but I encourage you to use it, it's really cool. But in the end, we achieved a scalable, sustainable workflow for multilingual documentation, and that's visible on our site today. So let's use the time remaining to cover a little bit of what our stack looks like. We host all of our content here in a GitHub repository, K-website, it's an abbreviation that we use, K stands for Kubernetes, Kubernetes.github.com slash Kubernetes slash website, or K-website. And our two CI pieces are Netlify, which we use for preview builds and publishing, and Prow, which is built out of this repo, K slash test infra, specifically the Prow subdirectory, and that provides our CI for Kubernetes. And these both use GitHub webhooks to manage the CI for individual PRs, and the end result, when all of the approval conditions are met, ends up publishing to kubernetes.io. So this is a very tiny snapshot of what our branching strategy looks like in Git. This is a very small piece of a much larger puzzle. So when you visit kubernetes.io and you go to the main landing page, you are seeing the master branch. We publish from master. And we have, what happens from master is that we have, there's a branch, release-v1 dot, whatever the current released is, right now that's 1.13. So release-1.13. And this is the branch that is most recent. And there's, you see there's a point at which that development, this was the development branch merged to master and we merge periodically into this branch from master. And then we have release-1.next. And in this case, right now it's release-1.14. So we cut this branch from master and then when we're ready to publish from this branch, we'll cut, sorry, we'll merge from master back into this branch. And this repeats itself. And the current release strategy for kubernetes is current plus four. The current version plus the four previous versions. So repeat this four different times and that's basically the branching strategy for kubernetes when you go to the site and experience that live. So let's talk about Netlify a little bit. So Netlify, instead of it's not just one piece, each one of these, master, release-1.current, current plus one, current plus two, three, four. And we use, for the version upcoming, we also have a development branch. Release specifies a release that's actually out there in the world. Dev specifies things that are coming up for the next branch. It's not yet visible or published on the website. So each one of these represents a Netlify deployment. So every time that you go to kubernetes.io and you use the version selector on the site to switch, you're actually not just switching a version, you're switching your deployment. You're switching which Netlify server that content's being published from. And for multi-lingual content specifically, our multi-lingual, our localization teams use a dev-1.version or dot language. So for example, for Korean, the Korean team is really very strict and very scrupulous about their involvement. So it's dev-1.14.co. This k-o is the ISO 639-1, two-letter language code for Korean. So each one of these teams has their own Netlify deployment so that they can get PR, that build previews for their own language. Let's see, okay. So this is, I mentioned the ability to set permissions. This is what that permission setting looks like in the repository. So you see here, this is the repository name and this is the content path. Hugo uses content and then two-letter code for every single language that you want to publish in in internationalization. So English content is hosted in content slash en. And this is the owner's file for English. And there are three things that I want to point out about this file. We have two permissions levels and this is true for the Kubernetes project in general. We break people who can review PRs and people who can merge PRs into two separate groups, reviewers and approvers. And this is obviously an alias. This is an alias that is specified, two different ones, sig-docs-en-reviews and dash owners. And all of these aliases are specified in a top level file called owner's sub aliases. And the reason why we maintain all aliases in a single top level file is so that we don't have to go into each one of these files and update individual names every time one of their involvement or permissions changes. We manage that in one place. But this is the really interesting piece here. This piece called labels. And you see that there is this marker here, language slash en and that matches here to this. So every time that a pull request includes a change to files that are under this sub folder here, any content, any English language content automatically receives the language, the English language applied to it as a label, as a repository label. So that makes it really easy to go through pull requests and you can filter by language. And so this is a filtered view of pull requests that have English language content in it. And you can see that's also possible for Japanese, Korean, Norwegian as a test language and Chinese as well. So that's a little bit about what that looks like. It's really cool. It works. I don't know what to recommend. Prowl works great for us, but implementing Prowl in its entirety for another project may be difficult. So I give that as a content warning, but other than that, that's how we do it and it works really well. Thank you. Yes. They are already published on Penta and they're available as a file attachment. Yep. That is an excellent question. They can, if it doesn't exist yet, if it hasn't, if the file itself has not been translated, they can simply go into content slash two-letter, whatever it is, say Korean as an example, content slash K-O, and they can look in the file, the subfolder tree, and see if that file exists yet as a Korean, as a file in the Korean subtree. If it doesn't exist yet, it hasn't been localized. And that's so, sir, to speak specifically to your question about how we handle fallbacks. It's possible in Hugo in the config.toml file in the top level of the repository to specify a fallback language. So we have, we set English as the fallback language. So if content doesn't exist for that language, unless you specify a fallback, it'll 404. But we specify English as a fallback. So if they go to a page that has not yet been translated, it appears in English and the version selector's nullified for that language, for that page. Yes. Okay, this one's for people. So if you go to the Kubernetes home page, and if you go to doc slash contribute, we have a localization guide. And I know that there, I can't remember whether we've published it there or in the repo wiki, but there is a script that you can run that can say, has this content changed between versions? So there is, but for incomplete translation, if that file exists, that partial content will be published. There's nothing, there's nothing that will only publish, say, the paragraphs that have been completed. The entire file will be published and it'll be mixed content. Absolutely, yeah. Thank you.