 yeah thanks a lot for showing up sorry I'm I have a slight cold so I'm highly doped but yeah please bear with me I totally want to give this talk and I'm really happy that so many of you are coming for such a boring topic documentation is boring right no okay okay you're always already one step ahead of me so quick survey how many of you are involved in open source projects okay how many of you are open so involved in open source projects that don't have any documentation how many of you are trying to change that and how many of you have actually written documentation before whoa whoa didn't expect that okay so this talk is titled writing open source documentation for open source project it's based on our experiences at SUSE but we can apply the lessons that we have learned to other projects as well and I will tell you how to do so first something about me my name is Christoph Wicker as you have already heard I'm a Linux user since 1999 in 2005 I became a contributor to various projects such as the Fedora project where I've been very active if people know me they probably know me from Fedora also I've been involved with XFCE and the LXDE and the one laptop for child initiative in 2010 I was hired by Colab Systems the nice guys downstairs with the Colab Cooper server and yeah early last year I joined SUSE as a technical documentation writer and as you can see here I'm not sure if you can read it this is the invoice from the first ever Linux distribution I bought SUSE Linux 6.1 student version for 79 Deutschmarks in 1999 one of the reasons why I purchased SUSE was well everybody was using it at the time if people were using Linux they were using or in Germany in particular they were using SUSE and one of the reasons for that is it was a German distribution and it was always very strong when it comes to documentation so documentation this big handbook was a great benefit I still use this even when I no longer used open SUSE I've been using other distributions I've been using Debianfit or whatnot but still documentation was the key so that's me who are we that's the open SUSE documentation team at the write the doc doc's conference in Prague in fall last year I think they can see me down there in the back you can see our team lead Marcus Finer probably some of him know him from Linux magazine and yeah that's not even all of us there's more because we have been growing rapidly SUSE has been growing the documentation team has been growing and also the numbers of the number of products that we are to document is also steadily increasing so who are we so the next question obviously is what do we want and we want to document all the things so much for the laughter that's that's the only picture I suck at slides so the rest of this presentation will be boring yeah what do you what do you mean with document all the things here's yeah you probably cannot read it I can't read it either it's too small here you have an overview of the SUSE documentation website as you can see it's quite a lot if you go that's only by products if you go further down into one of the products you see are different kinds of guides like get it started or quick install guides deployment guides user guides administrator guides then certain topic oriented guides like tuning guides virtualization guides and of course with every new release we have release notes all of that sums up to I think at the moment it's 14,000 pages if I'm not mistaken with SUSE Linux Enterprise 12 so that's quite a lot of documentation and the question becomes how do we handle that how do we write it and how do we handle it so we are talking about the tool chain SUSE has basically written a tool chain of its own because we figured at the time we started I mean that predates me joining SUSE but at the time they started they couldn't find any decent tools if you go to if some of you are in documentation and they go to special documentation conferences not like write the docs which is focusing on free software but to like take home the German documentation foundation for technical documentation you will see all of that is proprietary and of course SUSE didn't want to go that way so we did everything on our own and we wrote a fully complied open source tool chain we start off with dog book XML for us it's the language of our choice I know there are others I know that that actually we are like the dinosaur everybody has moved on to something more modern like like just markup or yeah there's there's plenty of stuff but at write the docs I figured that we are one of the few to use doc book but on the other hand I very much like the semantics well there's there's a lot of advantages too much to explain it now I could go on but we really figured it was the only language that catered all our use cases whatever other language came up there was always some feature missing as it's dog book as it's XML it's as people say write only language you can write it but you cannot read it right the the upside is many people are familiar with HTML or XML in general and many of them can you can learn it pretty quickly if you just do a short if you want to submit a short patch or something like that you can if you see para para okay that's probably means paragraph I should put my text into inside a paragraph and you're done so that's pretty pretty trivial and you can see all about doc book doc book dot org next when you have the language you need an editor you know probably know this from classical programmer paintings on tumblr that is the everlasting battle of e-max versus vim so yes you can use any editor you want I'm favoring vim most of the others in the team are using e-max there are proprietary editors there are you can use G edit J edit whatever atomic you know it atom yeah atom so many of them once you've created your code in or your dog book files in the editor you need to somehow publish it and this is where our toolchain kicks in we wrote a tool that is called dubs the dog book dog book authoring and publishing suit this is on github it's open source you can use it so some people say well what is dubs it's basically a rapper around make and I hear some of you complaining oh are you really talking about new make I hate new make no actually new make is the I mean it's the dependency based build tool and that's exactly what you need if you want to generate output files when people say they don't like make it turns out they most of the time they are using they are talking about auto make so they are in fact referring to the auto tools which I don't like either but anyway new make is the tool of our choice it's running in the background and you have one front and which you can use to do all kinds of commands the simplest thing is dubs validate you want to know if your output is valid if you want to generate a PDF file you just type dubs PDF you can see that make is running in the back because the that's actually like the targets for make PDF HTML then you can also pass options like gray scale PDF or HTML single otherwise you get a chunk of files we have lots of formats you can generate man pages plain text files plain text files are not that easy we are using we 3M the browser when it comes to tables you want to have a clear output so that's why we first render it into HTML then use the browser to display it and save it to a text file you can do it up with calibre and from that you can also generate Moby files for your kindle and you can generate the web help that's a special format of HTML it comes with a with a table of contents at the site and with a search bar that is using some JavaScript so if you are using for something standalone for your product and you only want this one single help piece on the internet you probably go for web help but you can not only generate output formats you can also do lots of interesting stuff with it okay here I need to explain something there's a call the minus D parameter refers to the configuration file every document has a configuration file where you specify the document route the starting point every XML element has an XML ID so you start as say from here on I start so you give it the configuration file and then you can say list files I'm looking for the in I have a ton of files in my repository and I'm looking for the file that contains the root ID art for article dubs quick and I can get that without I mean I could get that with grep probably but that would also list of a lot of false positives so we have a command for that or you can give a say generate me a list of all images with the modification date and also show these images in a viewer so it's quite quite a comprehensive tool then we have some SUSE specific commands dubs lock drop so the translation we do we are it's written in English even though most of our team is German which sometimes you can really see it was written by Germans because Germans have a special way of writing or speaking English especially when it comes to the to the words like have to must need their use differently in German and yeah so we write in English and it's then generally translated by translation agency by an external translation agency that's focused on technical documentation in order to have them translate our software we give them basically a big table in a clearly defined format with all the images and so on so that's generated with lock drop and after the translated documents come back we can do unpack lock drop or we can just do the online target dubs online to generate online doc to generate the online documentation that you see on the SUSE website so that's actually a quite powerful tool we are still improving it and we are meanwhile not the only ones that are using it while back we received our first patch by somebody from salando you probably know them so they are obviously using it too next in the tool chain we have the style sheets you have the XML and you need a style sheet to generate the different output formats for example we have standalone books or bigger like the manuals the guides we have single articles which are just a few pages we have just just the HTML and we have different I mean it's the difference if you have a PDF or if you have a website for example on the in the PDF file you want URLs in links to be written out because on paper you cannot click I mean you need to have them written out so this is done with the XSLT style sheets yeah the next we have a style guide that reminds us how to phrase it's not only about phrasing it's well how to spell certain terms recurring terms what tone to use how do you address the reader what's the target audience and all of that so you want to have a consistent voice throughout all your document you want to have consistent namings consistent use of terms and all that's and also the structure and markup needs to be consistently all that's in our style guide we then have a style checker which is a tool that you can manually run on the on an XML file it will output of an HTML file which you can then view in the browser and if you view it in the browser you get this nice these are actually little plus signs where you can just fold the the paragraphs and then you get a list of well notes and warnings of things you should probably fix so much for the style checker we have yet more tools all of them are on github in the open SUSE organization dubs and is something that we just finished recently or it's well it's not finished but we brought it into production this week it's continuous integration with Docker we have dbx included for well in any of you knows doc book you can have includes but you have the problem that if you have XML IDs they need to be unique and if you use an include various time and document the the IDs are no longer unique that's something that we solve with dbx included we have a doc manager that manages meta tags for example is this document going to be translated or not that's something or what path is once it's published on the website what's the URL these are custom tags that we added and they are managed through the doc manager we have a doc stats tool gico doc is our dog books subset we don't use the full set of dog book because it's just way too big we want to actually limit the documentation writer to a certain subset of elements that they can use and also that that allows you to enforce guidelines like you must not use nested lists with x level of nesting or you must not use this element inside the other element and all that's in the gico doc of course you need an editor that can read the schema files and that I mean that not only does syntax like syntax highlighting or autocomplete but that also like the the structure of the semantic syntax thing and also XML diff and G next and G is next generation there was already an XML diff out there if you compare XML files the classical way as you do with git for example I mean even though we try to have very strict with with the dimension dubs auto format earlier so we have common formatting like like indentions and and no white spaces at the end of the line and all that even though sometimes patches look messy and that's why you can use duck duck the XML diff and G to really compare the content of the files not the XML syntax okay so much for the tool chain you can ask questions later now I'm going to talk about the process as I say it's all open everything is on github here you see the doc sly repository so that's the it's no longer the open SUSE organization it's the SUSE organization on github you have doc sly and other repositories all prefixed with doc that's us Sly is SUSE Linux enterprise server and related products here you can see the DC files for the admin guide or for the everything guide and so on yeah it's on github you can submit pull requests you can you can take it as you need it and from the Sly documentation we also build the open SUSE documentation it's basically the same thing except that phrases like SUSE Linux enterprise are replaced with oh the time is actually getting longer oh I was hurrying up and and so now I I have 30 minutes left okay I slow down a bit thanks yeah it's on github you can use it you can fork it you can do whatever you want with it you can even some people even we were surprised a while ago when we saw our documentation on Amazon somebody is obviously generating books from it printing it and selling it we have no idea who it is we have nothing to do with it but it's our documentation it's it's really ours we verify it so you can do that it's open it's all gftl licensed use it as you like so when we are talking about git I need to explain the branching model a bit maybe some of you have read this article the successful git branching model by Vincent Driesen well the schema it's not the diagram is not necessarily the most it looks a bit messy but it's because it's so small so basically we are using the developed branch for development the master branch is the blue one over here it's only used for releases there you have the text for every release and then we have different branches for features bug fixes and maintenance and they also they all follow a consistent naming like feature slash fade 1 2 3 4 fade is the feature and enhancement tracker that is using it's also open source you can find it as open fade it's also used in open Sousa for suggesting new features and discussing feature implementation or bug fix BSC is bugzilla Sousa.com and then the bug number so we are trying to have a very consistent branch naming people work on their own branches and when they are done with it they submit a merge request through github and we are using we recently started using the new review feature from github maybe I can show it later if people don't know it yet that's actually something very nice yeah and we are using this tool called git flow well oh there's a typo it's it's not F but it's V git flow AVH that's a fork from the original git flow tool so git flow is an extension if you have a git repository you type git flow in it in it once it adds some values to the configuration file that you can for example the naming of these branches the prefix feature bug fix maintenance yeah it's a helper to implement this branching model and you can modify the configuration as you like but it makes sure that you have consistent naming when you want to open a new feature or you start working on something you type git flow feature BSC 1234 start when you're done you would use git flow feature finish but we no longer use this but use github's reviews merges instead yeah at github things get reviewed thing and then we try to squash squash all the commits into one so that and then they have a meaningful commit message that you can later cherry pick the whole feature or the whole bug fix without having to cherry pick 20 individual fixes but one one commit only yeah so much for the branching model now the bigger picture so we have three main sources for input either through fate which I already mentioned through Buxilla partners or employees file bugs because they found a bug in the documentation or through a system that is called doc comments I'll get to that later after that we do evaluation and planning to be honest that's why I don't write it out there we are using Trello for the organization a lot of you are probably using it personally I don't like it I don't like it because it's not open source obviously we are or I'm trying to push people to use fabricator instead or something else but yeah there's plenty of tools I mean you can use when it comes to tracking there's there's so many open source tools out there but nowadays I mean I'm old school I use Buxilla or something like that or it in Fedora we use Buxilla a lot but nowadays people want to have these work boards and they want to have nice drag and drop in the browser and whatnot so we use Trello that's important when it comes to planning we assign resources we have we have this team of like 13 and we have these tasks to do and we have an estimation of how much every task is so you pick your Trello cards this is what I'm working on yeah that's really so the resource planning is actually something I forgot here we have gun charts as well but I don't do them and I don't know what the software is called for doing it yeah after the planning comes the research usually when there's something new you need to research it first you cannot just start documenting it straight away I mean if I knew everything I wouldn't be yeah well there's nobody who knows everything actually we do have some documentation writers who know more about certain things they are focused on storage for example and they know they have so much they have accumulated so much knowledge over the time but usually when there's something new you first sit down with a developer for like two hours and he explains this new feature yeah then comes the documentation this is where all the the unicorns are born and and everything yeah it's it's sometimes it's hard work but it's just writing the the stuff and I think I've explained the toolchain already so let's just say it's magic after that we send it to the developers for feedback it's not yet published some of the documents are then translated and then they are published on the website later so one thing but that's actually we are not done once it's published I mean it's it's a constant iteration not only with every new SUSE release but with every release of the documentation sometimes we do publish doc updates after the release and also with every feedback we get and the feedback is something that many people forget when they write documentation so we have two main methods for feedback one is you can report a bug so as you can see these are two screenshots from the SUSE documentation from the online documentation and there's actually a report bug at the bottom at the top for every paragraph which automatically opens bugzilla it fills in all the forms which book is it which chapter which everything and then people just need to fill out their feedback or their criticism or whatever they have in the comment field of bugzilla the other system I quickly briefly mentioned it earlier it's called doc comments this is the system that you see here once again in the background we have the XSLT style sheets working the XSLT style sheets again doc book they allow you to for example on the we have the we have a nightly version of the documentation that's generated every night if you click the bug the feedback bug feedback button there you are directed to bugzilla but on the documentation that is that goes live you're directed to the doc comment system which is yeah more intuitive you just need to fill out your email and your name and can provide feedback so for technical people bugzilla is probably the way to submit feedback for less tech-saving people it's probably documents or other people just send us emails to doc-minus-team at susie.com so that works as well but really don't forget the feedback and don't rely on on people I mean make it easy for people to give feedback don't really rely on if the hurdle is so hard high that you require them to submit pull requests on github you probably won't get a lot of feedback so there needs to be a low entry barrier for feedback and the more feedback you have the easier it is the more feedback you get the more feedback you get the better your documentation gets and in the end ideally all questions are answered and yeah no more feedback needed which never happens but all right but so far that works up pretty well and I think compared to other documentation teams we are doing pretty fine we've seen that at the write the doc conference last year where people were telling us what they are planning to do and we figured like oh we are already doing that we are already doing this so on but of course we do have some problems as well well continuous integration that problem was just solved last week but the biggest problem that I see is we are still doing the traditional waterfall thing we have tried to become more agile or more continuous but that doesn't necessarily work there's always a very short time frame between when a feature becomes ready for testing and for documenting and between the release so after the the beta from beta to release of Susan or open through the product we are very very very busy and and don't have time for anything else sometimes we even have to veto features we can do that if the doc if there's no documentation and the feature is useless it probably gets delayed or we need to provide a documentation update later so again the the workload throughout the year is not constant and yeah that's definitely something that we need to address so we are but it's always a question of I mean of course you can do the whole agile thing and document every single feature and every line of code that the developer does but then I'll get to that just a moment but then yeah well you're in this in this never-ending loop you document something if you have a scrum cycle a sprint and after two weeks you figure out oh our developers rent into the wrong direction and let's do it from scratch you have documented something well that isn't there or you have documented it for yeah no it doesn't work out so you start all over again for example screenshots are typical example right before the release we have to go through all these screenshots in the documentation again to make sure they are up to date even if only a tiny thing in the interface has changed you definitely want the latest screenshot all the checkbox or developers keep renaming options or something like that and you're referring to this option not only in the screenshot but in the text so this can become very cumbersome and if you do this last minute yeah also we are decoupled from development that's both a buck or can we just address your question later after that okay I already address it thank you we are decoupled from development that's both both a feature and a buck well people think a developer is the best person to write documentation but he actually isn't I mean he knows the inner workings of his application but he doesn't know the users expectations for him everything is so obvious why would people even ask questions right so you always need or in a perfect world if your organization is big enough you do have a dedicated documentation team or in your community you have some volunteers who do the documentation and ideally they are not the same ones that are also working on the code we inside Susie can bridge this gap by sitting down with the developers but on the other hand sometimes it's I mean sitting down is always very nice but sometimes it's also emails back and forth back and forth 20 times that is cumbersome as well so the coordination effort is very big and the last problem that I see is so the first two problems can probably be addressed with a more continuous and agile approach we do have for example the cloud guys in the in Susie are working with the agile approach they have a dedicated documentation writer in their team who works with the developers every day who follows their sprints so we will probably switch to to a model like that but on the other hand you still need we have this big big big monolithic as I call it documentation like the reference guide or something that's not something that you write on the drive-by while you're developing that's a huge effort and that's where you need the team of documentation writers so we will try something in the in between between an agile and a waterfall approach let's see where we end up with this and to address this monolithic issue the issue of this big monolithic thing we will probably switch to more topic-oriented documentation topic-oriented is the new buzzword when it comes to documentation that's like you have a single task you have a user story you want to go from A to B want to achieve this or that yeah that should be that should lower or should solve some of the problems but on the other hand the traditional documentation the admin guide or the reference guide is not going away anytime soon because people customers wanted they want to have one point of contact where they have like all the options of this program or so deep so the new topic-oriented formats we are using are more like an addition we recently started a series called suzer best practices which will do yeah the best practices for virtualization the best practices for this or that purpose so that's more topic-oriented and now let's go to the lessons we've learned dogs or it didn't happen I mean you all know pigs or it didn't happen for us that dogs or it didn't happen if there's no documentation for you for your app for I mean even if it's just a tiny app I had an app on my mobile I had a problem with it it turned out to be a buck there was a button missing on a certain screen but how was I supposed to know there was no documentation there was no reference design should there be a button or not so you totally need documentation even if some Mac guys tell you that programs should be so intuitive that they don't require documentation you will need documentation period and don't consider your work done until unless you're done with the documentation a feature is not implemented if there's no documentation for it really that's that's things that that also product managers need to understand of course they want everything and they want everything now or they want everything last week but if there's no qa and if there's no documentation then it's not done period yeah as I already mentioned developers should not write docs we are doing it for a good reason don't write about the stuff you know by heart done you know somebody's taking taking photos of these slides yeah you need feedback channels without feedback it's all useless you're running in circles you have no idea what the users expectations are actually are you can do the you can do the long reference guide maybe it doesn't help maybe your users and expect something completely different so if you don't have feedback channels yeah then it's it's not really worth the effort so think about the feedback before you start writing and try to make the feedback as easy as possible yeah you need tools don't try to spend too much time on tooling I mean we didn't write all these tools because we felt oh we need to write yet another tool when we started out there they were just not there meanwhile there there are other tools for example Fedora has at least has had public publican which is something similar to dubs but is meanwhile orphaned I think it's no no longer used Fedora recently switched away from dog book there's for example pondoc that's something similar to what what we do with dubs that can translate between different documentation format and output PDF or HTML and so on so there are already plenty of tools out there if you have any question about a tool or so you can ask me but make sure you have the right tools for for the job to get the job done yeah don't spend too much time on tooling but on the other hand don't try to know don't you have to spend some time on the tooling otherwise the writing becomes too cumbersome but on the other hand don't reinvent the wheel by by writing yet another oh this is solving all our documentation problems there are some proprietary solutions out there which are really awesome but totally useless well for our use case for our use case useless data or something like this I mean there's company making big money with it but it doesn't cater our use case and when we them I mean we had sales agents who were trying to convince us and we tell them we've written our own and we publish it for free they were like what okay start small don't start with a reference guide I mean that's basically at the very top of the of the documentation you start small start with a certain use case try I mean at the very beginning you probably don't have a community of documentation writers or your company doesn't have anyone employed for writing documentation so at the beginning at least at the beginning it will probably be the developer try to see something from the user's perspective and try to try to think of the typical user tasks that a user that the user is trying to achieve and and this particular topic write it down like how do I connect to the mail server or something like that but don't do the I'm covering all menu options of my program and and and all command line options or something like that that's at the very very end of the documentation if you're done with everything else you can do this and last but not least documentation should be open if you're doing open source your documentation should be open and by open I not only mean have your sources somewhere on github I mean we do have our sources on github you can do that whatever you want as I told you before but also try to have an open process that allows the community and allows other people to to give feedback to participate yeah personally we found it it's difficult to encourage the community to to start working on documentation documentation is something that many people take for granted and I only use it when you have a problem as long as everything is fine you don't need it so that's why probably not a lot of people want to work on it but within the community you can do something I mean start with something small like a weekly yeah be open and that's the message I wanted to bring to all of you if there are any questions we have like eight minutes left or something okay first question so the question was how do we manage the localization of our documentation I mentioned the doc manager earlier the doc manager has so at the head of the document you have some some meta text some some additional doc book elements the custom elements and they define whether or not this documentation is translated so for example the install guide is translated into these 20 languages the quick install while the reference guide is only available in English or the tuning guide or the virtualization guide so I think we have like around 20 20 different languages like what we have like some stuff is not translated at all others is translated into six languages and and if when you do the full languages that's around 20 and that's all defined by these meta text within the doc manager we then give it to the translation agency and from there they they go on to to do the actual translations to be honest so the question was how do they do the actual management to be honest I don't know because I don't do the translation myself that's something that we have outsourced you can do this for example this is what we did at co-op at the co-op group or server with tools like trans effects this online translation tool this is also good because it gives people small chunks I mean it's broken down to strings or to paragraphs and people can start working on it so that's something that I would suggest but yeah I can imagine that if you say I have this file in English and I now I'm doing it in in German or something that is difficult but sorry I cannot really give you the answer I don't know what their the translators workflow is I don't think they start from scratch that but they will probably really take our files and overwrite the text but that's just a wild guess and if you're looking for something for a community I would recommend tools like trans effects so you say you don't like Trello because it's closed source but you are publishing all of your work on github what do you think of the current state of tools available on services like github and would you be tempted to try and share some of your learning so the question the question was if I may make it short why we are using github if it's if it's closed source well on github you have the large user base personally I would like to use get lab or something well that's not actually my question my question is what do you think of the state of tools for documentation on services like github and get lab and would you be willing to share some of your experience improve those tools I don't think that integration of documentation I mean they are focusing on code not on documentation for example they can if you're using markup that works markup can be displayed directly in line in github of course it's well the output depends on the browser or so but you get an idea if you look at an XML file in github it's just XML I would love github to have an XML preview but that's obviously not not trivial because then it needs to know the style sheets and whatnot so yeah but really they don't care about documentation much they care about code but the answer to this is treat documentation like you treat code apply continuous integration apply release engineering tactics that you would use for code as well but certainly the tools could be all the services could be improved you had a question as well we cannot so the question was how do we guarantee that the documentation is up to date and I'm sorry I hate to admit it we can't of course we go through it like when it comes to say the screenshots that's a pretty trivial thing you go over the documentation and make sure the screenshots are still up to date but we don't really rework every guide for every release we implement the new features we implement changes but of course there sometimes is outdated information still in there that's where we rely on feedback where we rely on users to file bugs well in a perfect world you would have quality assurance not only for code but also for documentation that somebody really goes through it and make sure it's still up to date but that's something that you cannot really do because of the effort the second one is a very broad question but okay the second question first why not have technical writers as QA for documentation I'm not saying that I'm against it but it's it's the question whether or not if you can afford it if you have somebody only working on QA of docs and also I think that people who do QA I've been doing QA before they should be see things just like the documentation writer needs to see things from a different perspective as the developer somebody in QA needs to see it from a different perspective so actually maybe it a technical documentation writer wouldn't be the best person for QA on docs I would rather have real-world user testing of docs have grabbed people from the street and and tell them to perform this or that if you have topic-oriented documentation it should be easy to give people tasks that they fulfill and then you see if it really works or not that's more interesting than QA so I think we are running out of time for questions okay final questions for those of you who don't have a chance I will be around on the hallway you can grab me there okay you got the one question do you reach out for upstream sure sure we reach out for ups so the question was if we reach out for upstream yes we try to but sometimes it's it I mean if I find something that we work with upstream but sometimes getting stuff back into upstream is very difficult like I recently had something that I needed to document which was some weird behavior of the linux kernel when it comes to epic routing and so the only documentation out there was a five lines of text in a plain text file in the kernel source tree and some comments in the kernel code itself I would love to fix that because then I can say oh I for commit in the linux kernel but as you can imagine the process of getting something accepted is quite cumbersome and at this point in time I'm not doing it but other we usually try to upstream everything all of our work and Susie is actually contributing upstream like we we are doing the open stack documentation for upstream open open stack all right I think that's all thanks very much for your yeah so use our tools breaks them provide feedback send patches thanks a lot