 you can call me Greg. I'm on the Docs team and the Rust Doc team. I named the Rust Doc team. I named a number of hats and I could guess they'd do some good things. But most importantly, Docs, be too much. And I have this habit of just spewing details in depth about things. So it means a lot to me that we have this tool available to us that lets us render these nice little pages to do so. So just as a kind of a forward for what I'm going to go over, I'm going to talk a little bit about some of the obscure things about the Docs that I think you can use and then more things that you can use to write the Docs and to make them cooler. And then if we have time, I'd like to kind of be able to. How many people in this room recognize this prose near universal? Excellent. My abstract is accurate. So this is basically the output of cargo Docs. It's consistent on every library that you're going to look at, unless they have their own little guide that looks to the side. So it's going to look very much the same for a lot of places. And so kind of getting into a few special things that you can do with it. A lot of the handwritten Docs that you have in there, Rust Doc has the ability to fold those up if in case you just want to see the listing of things. And there is this button off in the corner. We'll fold up everything on the page. And there is not only just the handwritten prose, but also the type declarations, everything that goes into a trade envelope will just really condense so you can see, okay, there are these methods, there are these trades, so you can just catch up the chase. Another really powerful thing that Rust Doc does that I think is really cool is that it packages it up and it's the text highlights the source of the library that is written with, that is running on. So that a lot of these things, if you have this item that you're looking at, there's this SRC button and you can click that and you can see how it's done. And it's everything that has run in Cargo Doc, unless they have like the original setting, has this source behind that you can take a look at and it's hyperlinked straight from the docs. And so I think that's really cool. One kind of smaller feature that is kind of like not only useful, but I think is kind of complete is that whenever Rust Doc outputs like a type signature for like a function more like the listing of fields in a structure or whatever, those are all hyperlinks to the definition of that type or that trade. And something that's like the normally useful part is that they're all colored a bit. Structs have a color, trades have a color, primitives have a color. And so like you can kind of see in a glance like oh this is how this is implemented. So another thing is that very recently, this is a few versions ago, if you just want to have everything in one page and control that through it, there is this button for that on the crate root that you can say see all of here is see all of studs items. And you click it and it's just this big old list. The page is actually really huge. Where you have absolutely everything by name. You can say well I remember this thing was called split or something. You can control that for it if you don't want to use the JavaScript search in there. Speaking of, if you look for a type in the JavaScript search, in the little search bar at the top of the docs, there's actually these links and parameters and return types are tabs that you can use to say like I searched back here and you can say show me all the functions that take VEC as a parameter. Or show me all the functions that return VEC. So it's really nice to be able to say well here's kind of a way that you can see how the type is used, how you can get one of these types. And so a nice fun thing to do. And there's like a bunch of little things that the docs has. I have a hard time keeping track of it all of the times. But here's like just like a handful of things. There's keyboard shortcuts that you can use. There's like settings to automatically fold or unfold things. And like you can search for the operator symbols of traits that correspond to those operators and you can go straight to those traits without having to remember. What was the thing, what's the name of the trait that you use with this star? Well, here it's MullenDref. So I've tried to be cagey with it. But when you run cargo doc much like when it compiles your code with cargo build or cargo run, cargo just kind of like shells out to another tool that comes with the rest distribution. That tool is called rest doc. So much like when you run cargo build it actually builds up command to Rust C, the compiler. When you run cargo doc, it builds up a command to Rust doc so that it can create your documentation out. And so to kind of introduce the next feature, one of the really powerful features I think this hardly counts as obscure, the really powerful features that Rust doc has is that you can, if you write a code sample in your docs, it will treat that as a test. So if you run cargo test or if you limit it with cargo test dash dash doc, Rust doc will scan your documentation, peel out all of the code samples that you've written in and try to compile it with your crate to make sure that you don't have code in your docs that goes stale. And so it's really helpful to have these examples actually work. And if you're running tests, then you can actually verify that yes, my docs actually show you code that works. So it's worth kind of digging into what Rust doc does when it has your doc test. So in the sample there, I had this two-line thing that just instantiated this empty struct, some struct, and then had a print line. And so what Rust doc wants to do is that it wants to take that code snippet that you gave it and turn it into an executable file that it can build and run. And so to that end, it wraps it in a main function if there's not one already and then adds a reference to your crate if it's not there already. And basically, in short, that's basically what it does. There's kind of more things that I get into. Like if you have doc tests that depend on a certain unstable feature, you don't have to write the main function yourself. So if you have this sick rad feature here, that will automatically get placed outside of the main function that Rust doc puts in. And same with when you have to reference multiple crates at once, you can just write that in line with the expectation that, no, you're not going to write them all in one place, but it's kind of assumed that you put those in the scopes where you want them, or I guess in Rust 2018 you don't have external crates statements at all. Then Rust doc will know to put those outside of the main function that runs so that the scopes work out properly. There's one last thing that it does. Because doc tests can kind of hit a few style lints that are pretty common, it adds this allow unused attribute to kind of ignore a handful of lints that are common to hit in doc tests, like in the one that I got. I assigned this my struct variable and I don't use it, so that's an unused variable lint. But say we don't want that. Say we want to make sure, say we have deny warnings on our crate and we want to make sure our doc tests are held to the same standard. There's a way to do that with this mouthful of an attribute here. This doc test adder can basically, it takes the insides of that and replaces that crate attribute to allow unused with whatever you put there. So here I've replaced it with deny warnings and to kind of show it off. This is kind of what comes out of it. And to kind of prove that it happened, I had that same code snippet and tried to run cargo tests on it and naturally it complains with that unused variables lint. So the standard library does this a lot. I think it does deny warnings but allows a handful of deprecated, so that it can document deprecated things. And so it uses this to kind of control to make sure that its doc tests are held to the same standard as its code. So to kind of lead into the next section, it's worth noting that Rust doc leans on the compiler a lot to run through your crate and get all of the information out of it. So it's these doc comments that you may or may not have seen. Those are actually specially recognized by the compiler. They're effectively aliases for these doc attributes here. So the compiler itself will look at these doc comments and say, okay, well, this is just a doc attribute. I'm going to convert it into this form here. And so that bottom section is basically what Rust doc sees when it runs on your crate. And leaning on that, the doc attribute itself has a few kind of overloaded things. I mentioned one earlier with that doc test attribute one. But there's several others that it uses. Just to kind of like list these off. HTML root URL is something you can use to control how other crates link to yours when they use your crate as a dependency. So the standard library does this to make links go to the nightly docs. Several crates use this to link to docs.rs instead of just nothing if their docs aren't made locally. Doc inline and doc no inline are ways to handle re-export statements. So if you've used or seen a pub use statement to sort of publicly relocate something from one portion of your crate to another, Rust doc can handle those based on how the original is visible to either copy in that other thing's docs or not. But you can use these doc inline, doc no inline attributes to tell Rust doc one way or the other, I want this to print a pub use statement or I want this to actually print the type's docs. Doc hidden is kind of a way to control what things are actually visible in your docs. So if you have your code base split up into multiple crates and you have something that's technically an implementation detail and you don't actually want to expose it as public API, you can slap doc hidden on it so that it's still exported and other crates can use it, but it won't appear in your documentation. So you have this sort of unwritten expectation that, okay, this is not meant to be used in that way. So it's just meant for people who work with the code directly. I think the standard library does this on the IO error enum. It has this non-exhaustive variant of it that it slaps doc hidden on to make it appear that you can't match on the IO error enum exhaustively. Things like that, you can say, documentation detail don't depend on this. Doc include is a relatively recent addition. It's still unstable to use, needs a future attribute. And what this does is it makes it so that, say if you have, like the standard library, a whole lot of docs on one thing, it kind of blows up your source files and it has a lot of this empty text that has, it's not relevant to the compilation of your code, but it prevents you from seeing everything on one page. With doc include, you can say, I'm not gonna write this as doc comments. I'm gonna write this in a separate file. Hey, Rust doc, copy paste that in whenever you run. And you can have your documentation in separate files so that you can track it separately and they don't get in the way of the code of your library. And doc CFG is the subject of the next few slides. So if you've rummaged around the standard library for a while, you may have noticed this OS module in it. And on the officially hosted docs, it has these Linux, raw, Unix and Windows submodules on it, some of which are only available on those respective platforms. There's actually a lot more modules in there which kind of gets at the problem behind some of this platform-specific documentation in that because Rust doc leans on the compiler to process code, conditional compilation happens way before Rust doc can get to it. And this is a big problem and an open question is how to really deal with it. So I have this another demo project up here that uses these traitsins.os to get the size of the file. I think the actual metadata struct wraps these, but for the purposes of this demo, I just have these platform-specific things in here. And I ran the docs on Linux and I ran the docs on Windows and as you can see, it's different depending on what system you actually ran the Rust doc executable on. So to get around this, you might first think, well, Rust doc doesn't need to actually fully compile the code. It doesn't need to link against platform libraries. It doesn't need to translate any of the code to machine code or whatever. So let's just make it available to Rust doc whenever Rust doc is running. There's a caveat about the codes featured here. That CFG Rust doc is supposed to be, there's an open PR to make this so that when Rust doc is running, that's set. Right now, the suggested way to do this is by using a cargo feature so that you would type cargo doc dash dash feature docs or whatever and use that as your conditional compilation. But in any case, you tell Rust doc, hey, this thing is available on, or you tell the compiler this thing is available both on Unix and whenever I'm building docs or on Windows and whenever I'm building docs. And if you're just building your docs, you get both things there. Cool, problem solved, right? Kind of. Say you have doc tests on these that use these items. Well, now you need to build and execute those tests as executables, which means they need to link against the platform libraries which may not be available on the platform where you're running Rust doc. So here I made a couple empty doc tests that just called the thing and I ran it on a Linux system and the Windows size function's not available because I'm not running on Windows. So to get around this, that's where the doc CFG attribute comes in. So this was introduced to get this .OS module that I showed earlier to make all these things come together and so this is, the problem that really solves is that yes you can tell Rust doc about all the things all at once but then you can't have doc tests on it but this tells Rust doc, hey, this thing is actually only supposed to run on Unix or on Windows or on little endian systems or on x8664, whatever, anything that you can slap a CFG on. And I put, so in that dummy project I put doc CFG attributes on and ran tests again and it only ran the Unix size test because now it knows I'm not supposed to run the Windows one, I'm not running on Windows. And the other thing that it does is it creates those little banners so that Unix, that square brackets Unix, square brackets Windows and the blue, this is supported on Windows only banner, the doc CFG attribute actually puts those in. So this is a way you can say outside of the text that you write that this is meant for this specific platform, this specific configuration. And so that's, like, it's something really cool that it can do and like I said, like, true handling of additional compilation is a bit of, as I've described it, a holy grail because of how much it leans on the compiler. But this is a good compromise if you can be conscious of when you're running docs. So moving on, in addition to doc attributes that it uses, there are also a handful of CLI flags that you can use. So I've just listed a bunch here. Of note is the document private items attribute, which is really nice if you want to have a separate set of docs for when you're writing on the creating question. What that will do is it, usually when Rust doc runs it actually will remove and not show items that are internal to your great. This is great if you just want to have, if you have a library and you want to say tell users of that library what you can do with it, but it's less useful when you're working on the library itself. So you can hand Rust doc this flag document private items and say, okay, well I'm actually going to look at the internal things of this now. But the subject of the next few slides is actually the HTML and header, HTML before content, HTML after content, because they do kind of silly things depending on what you're doing with them. So I have this another demo project here, and I have just this regular page here, regular Rust doc output, cool. But say you want to add something to your page. Say you have like a top bar, or you want to add like a little signature to the bottom of each page, or you have like a JavaScript library you want to pull in. You can do that. So it's a hand flags to Rust doc via cargo. There's this cargo Rust doc command. And you can use that, and it will run docs on just your crate, and hand these extra attributes that you give it to Rust doc whenever it calls it. So I added this extra file here to Rust, a tiny HTML snippet, and it adds a little positive affirmation to your docs. As far as practical applications, the Curve 255.19.Dollack library, and its internals docs, it actually fully writes out like the mathematical formulas and symbols for like the theory behind what it's doing, and it uses this Cossack library to lay that out, and it uses HTML and header to pull that library in so that all its docs can use it. There's also an impractical application you can do in case you want to plaster your docs with ponies. This is a real URL. If you have a laptop with Wi-Fi, you can actually go to this and see this in action. It's animated, it's fun. So this is kind of what popularized this, I guess. Henry DeValence and Isis Lovecraft kind of put this together. Henry kind of talked to me first when they were putting it together, and it's a shame that they're in the next room talking right now. And basically said, well, this is something that we can use to render Cossack, and then they looked at it and said, well, docs.rs, you can hand that arbitrary arguments in the configuration there and the crate docs of ponies you can see. And, well, how about if we just add in the thing that plasters ponies all over your webpage and do it to your docs? And the ponies crate, if you go to that URL, they give you instructions as to how to do it. And so it kind of started a conversation as to what counts as what they call XSS, cross-site scripting. But in my mind, it's just kind of a novel use of features that are already there. So I think it's kind of funny. So I do have a little bit of time. So let's take a second to kind of look at what goes on under the hood. So Rustdoc has been around for a while. I kind of mentioned that in a minute. But it lives, its code lives in the compiler's repo right alongside the compiler in the standard library. So if you look in the Rust repo's source directory in the middle of the massive list of subfolders there, there's a little one called libRustdoc. And that's where all of Rustdoc's code lives. And so, like, all of the stuff that it adds on top of the compiler lives there. As a side note, Rustdoc is kind of old. Like, this is the PR, I dug it up, that first added the very small barebones things that scraped out doc attributes. This was before I was talking with someone else this morning, apparently this was before doc comments were a thing. So I just have, like, the timeline there. July 2009, Graydon started, like, writing on Rust as a private project. Shortly after it was made public. And then in April 2011, it switched from being written in OCaml to written in Rust itself. So when it started self-hosting. Just a few months later, Rustdoc was added. So Rustdoc is almost as old as the compiler. This means a lot of hands have been placed on it. And it's a little, it's a little cursed, in a sense. I have this joke in a handful of friends that Rustdoc kind of occasionally kind of gets its information by not kosher methods. So first we ask nicely, there's the actual compiler things that the compiler does for what it does. And sometimes we have to go it around that somehow. So I mentioned the doc attribute earlier. That doc attribute is actually fully whitelisted by the compiler. The compiler just ignores the doc attribute. It passes it through, but it doesn't do anything with it. So Rustdoc, I know I've used that personally whenever I've added an extension attribute to Rustdoc to just hack on it and deal with it later. Occasionally we have to break the compiler to get what we want. So I have notes here. There's a feature called intradoc links where you can write your link target to point at a type instead of the actual HTML page. To do that we had to break the compile flow because name resolution happens before the part that Rustdoc usually runs out. So we had to re-architect part of the compiler to be able to hold on to that information. And if things get drastic enough we threaten to break the compiler's friends. So a couple of years ago, maybe a year and a half ago the way that the compiler was built changed and Rustdoc really benefited from that. So it was this complete upheaval in the compiler repo and the net effect of it all was that Rustdoc used to be built as part of the full compiler process which takes like an hour and a half on my system. And now it builds on its own which takes like three minutes on my system. It's great. And then also there is this way you can, there's this function and they're called pretty printing that has a handful of transformations on the code. And one of the things that it does is there's this version called everybody loops that replaces, yes, that replaces all function bodies with a empty loop statement because empty loop statements type check to everything because nothing's coming out of that. So this was done actually because Rustdocs were put in because if you're not referencing anything inside your functions now you no longer have to worry about what's in there. Now you no longer have to worry about accidentally referencing platform stuff. So because of the way I've laid this out the inside joke is that Rustdoc is kind of like the mafia. But it provides an essential community service so no one's really tried to stop it. So I have a couple slides here that I'm gonna actually gloss over. If you read the full slides I have them online that like here's the full compiler process of what it does. Like I mentioned it's the jargon filled version so I'm kind of glossing because we're getting close to time. But here's a few highlights. So if you've looked around this is kind of like circling back to fun things that are in the display of documentation. If you've noticed for a couple versions now there's been these auto-trade implementations. So this is novel because that send-imple doesn't exist anywhere. Because Rustdocs scrapes the text of your crate effectively it used to not even say that VEC implemented send or sync at all. You just kind of had to assume that well if the items in the VEC are sync or send then that transfers to the VEC. Now that's made obvious but to do that we had to break the compiler. So that because we had to basically make up these symbols on the spot. Another fun highlight is that Rustdoc outputs HTML but it's templating engine is a gigantic write macro call. This is... This is literally like in the Rustdoc source there is this file that is just a massive skeleton of the HTML page and everything gets written in there. So everything that needs to be printed implements display and it just slots it there. Internals-wise Rustdoc has its own test suite has its own form of tests to make sure that we don't skip outputting anything that we are supposed to output. So here it's just saying we output fields that are public and instruct and we don't output fields that are not public or are hidden. And there are a couple hundred tests like this I think. There's quite a few and it's kind of fun to be able to say we're actually scanning the output of the HTML there. So to wrap up I just kind of wanted to really mention the Rustdoc team is actually an official Rust team to deal with the maintenance of Rustdoc and so it's us five. Steve's on it, I'm on it and I'm on it for years. And we just kind of hang out and talk about compiler internals and how we should display docs. But it's just kind of fun if you're interested in kind of the gory internals of how Rustdoc how documentation goes from crate to docs. It's pretty fun. But as I close you may or may not want to actually follow me on Twitter but if you'd like to verify that claim otherwise that's all I've got.